Brad L. :verified:<p>There's a lot talk about "ZeroDisk" infrastructure backed by S3. The pitch is "move your data from locally attached NVMe storage to S3 and your applications will scale easier and be more performant!"</p><p>Maybe I'm getting too old for this shit, but I swear to dog this is the 4th such cycle in my career:</p><p>1. NFS<br>2. iSCSI / Fibrechannel<br>3. Hadoop / HDFS<br>4. ZeroDisk with S3</p><p>Am I the only one that's like: "wait, move TBs of data to S3 from NVMe to increase performance? Are you high?"</p><p>It doesn't work, so you scale up. Now you're back to local NVMe "cache disks" running instances as expensive as the locally attached NVMe instances when you add those costs to your S3 bill. The performance is worse because of course it is.</p><p>It always comes back to the two hard problems in computer science: naming things, cache invalidation, and off-by-one errors. 😂 </p><p><a href="https://hachyderm.io/tags/zerodisk" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>zerodisk</span></a> <a href="https://hachyderm.io/tags/s3" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>s3</span></a> <a href="https://hachyderm.io/tags/hadoop" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>hadoop</span></a> <a href="https://hachyderm.io/tags/cache" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>cache</span></a> <a href="https://hachyderm.io/tags/datalake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datalake</span></a> <a href="https://hachyderm.io/tags/GetOffMyLawn" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GetOffMyLawn</span></a></p>
eupolicy.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
This Mastodon server is a friendly and respectful discussion space for people working in areas related to EU policy. When you request to create an account, please tell us something about you.
Administered by:
Server stats:
196active users
eupolicy.social: About · Profiles directory · Privacy policy
Mastodon: About · Get the app · Keyboard shortcuts · View source code · v4.4.3
#hadoop
0 posts · 0 participants · 0 posts today
Peter Czanik<p>No response yet to my <a href="https://fosstodon.org/tags/syslog_ng" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>syslog_ng</span></a> <a href="https://fosstodon.org/tags/HDFS" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>HDFS</span></a> destination question:</p><p><a href="https://www.syslog-ng.com/community/b/blog/posts/deprecating-java-based-drivers-from-syslog-ng-is-hdfs-next" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">syslog-ng.com/community/b/blog</span><span class="invisible">/posts/deprecating-java-based-drivers-from-syslog-ng-is-hdfs-next</span></a></p><p>Most likely it means that we can drop <a href="https://fosstodon.org/tags/Hadoop" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Hadoop</span></a> support from syslog-ng without any complaints. But I rather repeat my question a few more times on my <a href="https://fosstodon.org/tags/socialmedia" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>socialmedia</span></a> accounts...</p>
Interesting Links<p><strong><a href="https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html" rel="nofollow noopener" target="_blank">Command-line Tools can be 235x Faster than your Hadoop Cluster</a></strong><br>"This find | xargs mawk | mawk pipeline gets us down to a runtime of about 12 seconds, or about 270MB/sec, which is around 235 times faster than the Hadoop implementation."</p><p><a href="https://bookmarks.kvibber.com/tagged/complexity" class="mention hashtag" rel="nofollow noopener" target="_blank">#complexity</a> <a href="https://bookmarks.kvibber.com/tagged/ShellTools" class="mention hashtag" rel="nofollow noopener" target="_blank">#ShellTools</a> <a href="https://bookmarks.kvibber.com/tagged/RightToolForTheRightJob" class="mention hashtag" rel="nofollow noopener" target="_blank">#RightToolForTheRightJob</a> <a href="https://bookmarks.kvibber.com/tagged/Hadoop" class="mention hashtag" rel="nofollow noopener" target="_blank">#Hadoop</a> <a href="https://bookmarks.kvibber.com/tagged/computing" class="mention hashtag" rel="nofollow noopener" target="_blank">#computing</a></p>
Francis Hunger<p>Help needed! I'm writing a media theory book chapter about Hadoop, BigTable, ACID, BASE and CAP for a non-technical audience. I'm looking for an _expert_ and practitioner from the field to doublecheck the 11 pages I wrote. Thank you for boosting. <a href="https://dair-community.social/tags/phd" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>phd</span></a> <a href="https://dair-community.social/tags/academia" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>academia</span></a> <a href="https://dair-community.social/tags/Hadoop" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Hadoop</span></a>, <a href="https://dair-community.social/tags/BigTable" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigTable</span></a></p>
Jan Vlug<p>Vacature: Senior <a href="https://mastodon.social/tags/Systeembeheerder" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Systeembeheerder</span></a> <a href="https://mastodon.social/tags/Linux" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Linux</span></a> bij de <span class="h-card" translate="no"><a href="https://social.overheid.nl/@aivd" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>aivd</span></a></span>:</p><p><a href="https://www.werkenvoornederland.nl/vacatures/senior-systeembeheerder-linux-AIVD-2025-0006" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">werkenvoornederland.nl/vacatur</span><span class="invisible">es/senior-systeembeheerder-linux-AIVD-2025-0006</span></a></p><p><a href="https://mastodon.social/tags/Elasticsearch" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Elasticsearch</span></a> <a href="https://mastodon.social/tags/NiFi" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>NiFi</span></a> <a href="https://mastodon.social/tags/Hadoop" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Hadoop</span></a> <a href="https://mastodon.social/tags/Ansible" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Ansible</span></a> <a href="https://mastodon.social/tags/Kafka" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Kafka</span></a> <a href="https://mastodon.social/tags/devops" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>devops</span></a> <a href="https://mastodon.social/tags/OpenSource" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenSource</span></a> <a href="https://mastodon.social/tags/werkenvoornederland" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>werkenvoornederland</span></a></p>
TrendingLive feeds
Mastodon is the best way to keep up with what's happening.
Follow anyone across the fediverse and see it all in chronological order. No algorithms, ads, or clickbait in sight.
Create accountLoginDrag & drop to upload