Thanks to SeaweedFS, I've set up two replicas of the BSD Cafe's media. One is on a backup server (for disaster recovery purposes only, also containing the hourly zfs-autobackup backups), and the other is on a Raspberry Pi 4 running FreeBSD, which is one of my home backup servers - just one meter away from my desk.
I've also configured Nginx in a jail, so when I'm at home, the BSD Cafe's media is served directly from the local replica.
The performance is lightning fast.
#s3
0 posts0 participants0 posts today
After conducting some performance tests, I believe #SeaweedFS will be the new S3-compatible software for BSD Cafe media storage. The results have been promising
There's a lot talk about "ZeroDisk" infrastructure backed by S3. The pitch is "move your data from locally attached NVMe storage to S3 and your applications will scale easier and be more performant!"
Maybe I'm getting too old for this shit, but I swear to dog this is the 4th such cycle in my career:
1. NFS
2. iSCSI / Fibrechannel
3. Hadoop / HDFS
4. ZeroDisk with S3
Am I the only one that's like: "wait, move TBs of data to S3 from NVMe to increase performance? Are you high?"
It doesn't work, so you scale up. Now you're back to local NVMe "cache disks" running instances as expensive as the locally attached NVMe instances when you add those costs to your S3 bill. The performance is worse because of course it is.
It always comes back to the two hard problems in computer science: naming things, cache invalidation, and off-by-one errors. 
OK, Minio has decided to drop all the useful GUI features from the open-source (CE) version, making them exclusive to the paid version. More, they'll only provide "security fixes for severe issues" and no new features/development.
Personally, I prefer software that doesn't offer a certain feature at all, rather than one that, at some point, removes it and reserves it for "paying" customers.
It almost feels like a betrayal, in my eyes.
It's probably time to move the #BSDCafe media over to Garage or Seaweedfs.
Replied in thread
Great to hear that, @hetzner!
And it even works perfectly with Mastodon, as I show in my latest blog article: https://abshoff.dev/blog/2025/07/25/mastodon-und-hetzner-object-storage/ 
abshoff.devMastodon und Hetzner Object StorageSeit 2022 betreibe ich meine eigene Mastodon-Instanz auf einem Hetzner Cloud-Server. Um die Medieninhalte unabhängig vom Dateisystem des Servers zu halten, habe ich mich entschieden, den S3-kompatiblen Hetzner Object Storage als Objektspeicher zu nutzen. Dieser Artikel zeigt Dir, wie Du den Dienst einrichtest und welche Gedanken ich mir dazu gemacht habe.
Mein neuer Blog-Artikel ist online!
Ich zeige, wie Du Mastodon-Medieninhalte flexibel und einfach mit dem S3-kompatiblen Hetzner Object Storage in die Cloud auslagerst, inkl. Migration, Nginx-Konfiguration und Backup mit Synology DiskStation.
Ideal für alle, die mehr Skalierbarkeit und Unabhängigkeit wollen.
Jetzt lesen: https://abshoff.dev/blog/2025/07/25/mastodon-und-hetzner-object-storage/
abshoff.devMastodon und Hetzner Object StorageSeit 2022 betreibe ich meine eigene Mastodon-Instanz auf einem Hetzner Cloud-Server. Um die Medieninhalte unabhängig vom Dateisystem des Servers zu halten, habe ich mich entschieden, den S3-kompatiblen Hetzner Object Storage als Objektspeicher zu nutzen. Dieser Artikel zeigt Dir, wie Du den Dienst einrichtest und welche Gedanken ich mir dazu gemacht habe.
Ok #mastoadmin #selfhost #S3 aficionados. I don’t have any experience with object storage providers. I am looking to move my self-host mastodon account here from local storage on my 2007 #footimac to my account at leaseweb. Could you help me with the answers to their queries? I suggested, completely off the top of my head and without experience, i would need 1.5TB and low bandwidth. Not knowing if that makes any sense.
“ for a 1.5 TB provision with low bandwidth usage, we can provide an estimate once we have more context on access patterns, regions, and any specific requirements.”
Amazon #S3 now supports sort and z-order compaction for #ApacheIceberg tables, promising reduced scan times & lower engine costs.
Available for both S3 Tables and traditional S3 buckets via AWS Glue Data Catalog optimization.
Dive into the details: https://bit.ly/3GyjxWQ
Nowe drogi w Polsce – co powstanie w 2025 r.
Zdjęcie: GettyImages. Druga połowa 2025 roku upłynie pod znakiem otwierania tras szybkiego ruchu. Według planów kierowcy zyskają około…
#Poland #Polska #PL #Europe #Europa #EU #a2 #Aktualności #autostrady #budowadróg #drogiekspresowe #Gdańsk #gddkia #inwestycjedrogowe #metropolia #nowedrogi #Obwodnica #OMT #poland #polska #Polska2025 #S1 #S3 #s7 #Trójmiasto #Warszawa
https://www.europesays.com/2260293/
https://www.europesays.com/2260293/ Nowe drogi w Polsce – co powstanie w 2025 r. #a2 #Aktualności #autostrady #BudowaDróg #DrogiEkspresowe #Gdańsk #gddkia #InwestycjeDrogowe #metropolia #NoweDrogi #Obwodnica #OMT #poland #polska #Polska2025 #S1 #S3 #s7 #Trójmiasto #Warszawa
I don't know if anybody noticed #ZeroFS yet, but it seems there is a completely user space-implementation of #NFS and #blockstorage on top of #S3 #objectstorage: https://github.com/Barre/zerofs
Including a demo running #ZFS on top of it which essentially allows geo-redundant ZFS volumes: https://asciinema.org/a/728234 & https://github.com/Barre/zerofs?tab=readme-ov-file#geo-distributed-storage-with-zfs
I don't see no #FreeBSD port yet, but if that really works it would be absolutely awesome.
GitHubGitHub - Barre/ZeroFS: ZeroFS - The Filesystem That Makes S3 your Primary StorageZeroFS - The Filesystem That Makes S3 your Primary Storage - Barre/ZeroFS
I added a few more notes to my blog post from yesterday about @getkirby backups to @hetzner Object Storage (S3) using @restic
https://kevingimbel.de/blog/kirby-backups-with-restic-and-hetzner-s3
kevingimbel.deKirby backups with restic and Hetzner S3 | kevingimbel.de
More from 
kevin ⁂ (he/him)
Disposable File Links with Quarkus
Ship a self-destructing S3-style file service in fifteen minutes with Quarkus, MinIO, REST, and plain Java.
https://myfear.substack.com/p/quarkus-java-disposable-file-links-minio
#Java #Quarkus #S3 #MiniO
A notary from sweden just got a mail about their open #s3 #bucket with plenty of data. The most documents where related to real estate in #Stockholm and surrounding.
Any current guides/howtos on migrating from local storage to cloud storage on a Debian install?
I think I might need to switch my #selfhost at some point unless I go out and buy a big harddrive.
„Kein Ersatzkonzept“: Mega-Chaos auf beliebter Bahnstrecke vor München bahnt sich an
Lamborghinis, Porsches und BMWs schieben sich Stoßstange an Stoßstange über die Bundesstraße 318 …
#Muenchen #Munchen #Munich #Deutschland #Deutsch #DE #Schlagzeilen #Headlines #Nachrichten #News #Europe #Europa #EU #München #Bahn #BayerischeRegionalbahn #Bayern #brb #Chaos #Deisenhofen #Feiern #Germany #Giesing #Holzkirchen #S3 #Seefeste #Sperrung #Tegernsee #TegernseerTal
https://www.europesays.com/de/245514/
„Kein Ersatzkonzept“: Neues Bahn-Chaos vor München droht
Lamborghinis, Porsches und BMWs schieben sich Stoßstange an Stoßstange über die Bundesstraße 318 zwischen Holzkirchen und War…
#Muenchen #Munchen #Munich #Deutschland #Deutsch #DE #Schlagzeilen #Headlines #Nachrichten #News #Europe #Europa #EU #München #Bahn #BayerischeRegionalbahn #Bayern #brb #Chaos #Deisenhofen #Feiern #Germany #Giesing #Holzkirchen #S3 #Seefeste #Sperrung #Tegernsee #TegernseerTal
https://www.europesays.com/de/243209/
Many of the AWS S3 tutorials for Ruby on Rails ActiveStorage are absolutely wild. Fully public S3 bucket with public read, write and delete permissions. Simple tip: Don't use a bucket policy permission you don't understand, just have a user with inline bucket permission. #Ruby #Rails #AWS #security #IAM #S3
Continued thread
If you made some kind of intercepting HTTP/HTTPS proxy (thinking of a #pentester use case here), you could make it search for these URLs in the streams of HTTP and HTML that are passing through the proxy. Copy down the full URLs and asynchronously issue your own requests for the same URLs and store your own copy of the resulting files. The end user still gets their copy and nobody can tell it's happening. You'd almost certainly be able to do this because the links would surely be valid at the time the proxy sees them, and would work if the proxy immediately issued its request for its own copy.
The only way to really detect this happening is for the bucket owner to look at the S3 object logs in CloudTrail and see more than 1 fetch of that URL. Of course, someone with network connectivity issues could issue the request more than once. But a systematic pattern of duplicate fetches would indicate hijinks. The end user can't detect this happening to them. But, of course, you're MitM'ing their internet connection, so that could be detected.
Continued thread
If you know how these things work, I haven't told you anything new or useful yet. Maybe I won't. But the thing I think is important and frequently overlooked is that expiration time. Too short (5 seconds) and your user might not click the link before it expires. Too long (86400 seconds, i.e., one day) and this file is available far longer than you intended.
So looking at the X-Amz-Expires header in #AWS #S3 is a good #security thing, especially if you're doing a #pentest . Those URLs can be passed from device to device (e.g., you can Slack it to a colleague or SMS it to a friend and it will work). So you want to counsel anyone who uses them to try hard to tune the expiration as short as is reasonably practical. That expiration is all of the security control on that link.
[edit: I left out something important]
I see these URLs with 86400 as the expiration time a lot and often. If you're a developer, look at what you're setting them to. If you're a #pentester, this is a thing to warn your customer about.
3/