Recent searches

No recent searches

Search options

Only available when logged in.
eupolicy.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
This Mastodon server is a friendly and respectful discussion space for people working in areas related to EU policy. When you request to create an account, please tell us something about you.

Administered by:

Server stats:

208
active users

eupolicy.social: AboutProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.1

#lmdb

1 post1 participant0 posts today
Howard Chu @ Symas<p>A new engineer discovers how Cloudflare used <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> for their distributed config <a href="https://xcancel.com/UltraSive/status/1948967694004326605#m" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">xcancel.com/UltraSive/status/1</span><span class="invisible">948967694004326605#m</span></a></p><p>"99th %ile of reads dropped by two orders of magnitude!"</p><p>"LMDB stability has been exceptional. It has been running in production for over three years. We have experienced only a single bug and zero data corruption. Considering we serve over 2.5 trillion read requests and 30 million write requests a day on over 90,000 database instances across thousands of servers, this is very impressive."</p>
Howard Chu @ Symas<p>I've dedicated most of my career to writing the most efficient code.</p><p><a href="https://ngmx.com/@mejofi/114889780968597360" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">ngmx.com/@mejofi/1148897809685</span><span class="invisible">97360</span></a></p><p>But the mention of storing way more data than you're ever going to use also triggered me here.</p><p>It's a common symptom of poorly designed apps looking for write-optimized databases like RocksDB. If your workload is more write heavy than read, then by definition you're writing more than you'll use.</p><p>It's another reason why <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> is read optimized, because we only care about data we'll actually use.</p>
Howard Chu @ Symas<p>lmdb-queue</p><p>A lightning fast disk based queue, meta data is stored in <a href="https://mastodon.social/tags/lmdb" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>lmdb</span></a> </p><p><a href="https://lib.rs/crates/lmdb-queue" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">lib.rs/crates/lmdb-queue</span><span class="invisible"></span></a></p>
Howard Chu @ Symas<p>"This article introduces <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> (an efficient key-value storage database) and the command line tool `lmcmd` developed based on Python. Since LMDB uses binary files for storage, which is inconvenient to manage and debug, `lmcmd` was developed to provide a command line operation interface similar to Redis, supporting database operations, data import and export, and search functions." <a href="https://developer.aliyun.com/article/1672386" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">developer.aliyun.com/article/1</span><span class="invisible">672386</span></a></p>
Howard Chu @ Symas<p>"If I abandon Vercel and switch to VPS, I can abandon Firestore and build my own query layer with <a href="https://mastodon.social/tags/lmdb" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>lmdb</span></a> ...<br> just thinking about it makes me happy"</p><p><a href="https://xcancel.com/apolkingg8/status/1943623985666679152#m" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">xcancel.com/apolkingg8/status/</span><span class="invisible">1943623985666679152#m</span></a></p>
Howard Chu @ Symas<p>"xs-lib is a command-line tool (xs) and a Rust library (xs_lib) for recording and retrieving sequential streams of payloads. It uses Lightning Memory-Mapped Database (<a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a>) for local data storage."</p><p><a href="https://lib.rs/crates/xs-lib" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">lib.rs/crates/xs-lib</span><span class="invisible"></span></a></p><p>I don't understand why you'd need to store a sequentially accessed stream into a database. This looks like an over-engineered `cat` to me.</p>
Howard Chu @ Symas<p>If you can't beat 'em, join 'em. Oracle now uses <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> <a href="https://docs.oracle.com/en/middleware/fusion-middleware/fmwlc/licensing-information.pdf" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">docs.oracle.com/en/middleware/</span><span class="invisible">fusion-middleware/fmwlc/licensing-information.pdf</span></a></p>
Howard Chu @ Symas<p>"I’m experiencing significant slow-downs when converting data from <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> to HDF5 format."</p><p>Solution: don't do that. <a href="https://mastodon.social/tags/HDF5" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>HDF5</span></a> is a pig.</p><p><a href="https://stackoverflow.com/questions/79706522/hdf5-write-performance-degrades-over-time-when-converting-from-lmdb-3-7m-entri" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">stackoverflow.com/questions/79</span><span class="invisible">706522/hdf5-write-performance-degrades-over-time-when-converting-from-lmdb-3-7m-entri</span></a></p>
Howard Chu @ Symas<p>ARTIQ (Advanced Real-Time Infrastructure for Quantum physics) is a leading-edge control and data acquisition system for quantum information experiments. It is maintained and developed by M-Labs and the initial development was for and in partnership with the Ion Storage Group at NIST.<br>...<br>Persistent datasets are now stored in a <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> database for improved performance. </p><p><a href="https://m-labs.hk/artiq/manual-beta.pdf" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">m-labs.hk/artiq/manual-beta.pdf</span><span class="invisible"></span></a></p>
Howard Chu @ Symas<p>Squawk - a compact <a href="https://mastodon.social/tags/XMPP" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>XMPP</span></a> desktop messenger</p><p><a href="https://git.macaw.me/vae/squawk" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">git.macaw.me/vae/squawk</span><span class="invisible"></span></a></p><p>Uses <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a></p>
Howard Chu @ Symas<p><a href="https://mastodon.social/tags/Flutter" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Flutter</span></a> Local DB</p><p>A high-performance local database for Flutter that leverages Rust's <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> embedded database through FFI (Foreign Function Interface). This library provides a robust, efficient, and safe way to store local data in your Flutter applications across multiple platforms.<br><a href="https://pub.dev/packages/flutter_local_db" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">pub.dev/packages/flutter_local</span><span class="invisible">_db</span></a></p>
Howard Chu @ Symas<p>OSM Express: fast data format for OpenStreetMap data, built on <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> <br><a href="https://github.com/bdon/OSMExpress" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">github.com/bdon/OSMExpress</span><span class="invisible"></span></a></p>
Howard Chu @ Symas<p>"Top 8 Embedded SQL Databases in 2025"</p><p><a href="https://www.explo.co/blog/embedded-sql-databases" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">explo.co/blog/embedded-sql-dat</span><span class="invisible">abases</span></a></p><p>Lists a few non-SQL DBs like <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a>, but ok whatever.</p><p>Also includes a couple written in Java, which I'd personally exclude because they're only usable in Java apps, which are categorically too heavyweight to be considered "embedded" IMO.</p>
Howard Chu @ Symas<p>VideoMind: Giving LLMs a Real Memory of the Visual World</p><p><a href="https://www.linkedin.com/pulse/videomind-giving-llms-real-memory-visual-world-atharva-date-htb1f" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">linkedin.com/pulse/videomind-g</span><span class="invisible">iving-llms-real-memory-visual-world-atharva-date-htb1f</span></a></p><p>"What if we could give LLMs structured visual memory? Not just a transcript of captions—but episodic, grounded, object-aware memory that mirrors human understanding?<br>...<br>Captioning BLIPDecoder generates and caches a caption per frame (<a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a>-backed). "</p>
Howard Chu @ Symas<p>Simplifying Synchronization with Lightning Stream for Enterprise</p><p><a href="https://blog.powerdns.com/simplifying-synchronization-with-lightning-stream-for-enterprise" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">blog.powerdns.com/simplifying-</span><span class="invisible">synchronization-with-lightning-stream-for-enterprise</span></a></p><p>"By leveraging <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> backends, Lightning Stream delivers high-speed, scalable replication with one-way and two-way synchronization between authoritative instances across different data centers."</p>
Howard Chu @ Symas<p>Cytoplasm: A general-purpose C library for creating high-level, networked, and multi-threaded C applications.</p><p><a href="https://git.telodendria.io/Telodendria/Cytoplasm" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">git.telodendria.io/Telodendria</span><span class="invisible">/Cytoplasm</span></a></p><p><a href="https://git.telodendria.io/Telodendria/Cytoplasm/pulls/43" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">git.telodendria.io/Telodendria</span><span class="invisible">/Cytoplasm/pulls/43</span></a><br>Supports <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> in addition to some kludgy corruption-prone JSON file writer. But sure, it's important to have the latter?</p>
Howard Chu @ Symas<p>Comprehensive RPC framework with support for C++, CSharp, Java, JavaScript, Python and more.</p><p><a href="https://vcpkg.link/ports/zeroc-ice" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">vcpkg.link/ports/zeroc-ice</span><span class="invisible"></span></a></p><p>Uses <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a></p>
Howard Chu @ Symas<p>MCCD: A Multi-Attribute Chinese Calligraphy Character Dataset Annotated with Script Styles, Dynasties, and Calligraphers</p><p><a href="https://arxiv.org/html/2507.06948v1" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">arxiv.org/html/2507.06948v1</span><span class="invisible"></span></a></p><p>"the total MCCD dataset of 7765 Chinese character annotations and its three subsets, MCCD-Style for 10 calligraphic styles, MCCD-Dynasty for 15 major historical periods, and MCCD-Calligrapher for 142 famous calligraphers. ... To enhance data retrieval efficiency, all training and testing sets are also converted into the <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> format (binary form)."</p>
Howard Chu @ Symas<p>Investigate <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> as HLR storage<br><a href="https://projects.osmocom.org/issues/3649" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">projects.osmocom.org/issues/36</span><span class="invisible">49</span></a></p><p>This Open Source Mobile Comm project appears to be an open source impl of a 3G network's backend software. But they're not using the standard 3GPP protocols, they're using their own home-grown stuff instead.</p><p>Looks like a toy, using SQLite3 as its HLR storage. Real 3G/4G networks use LDAP, lots of OpenLDAP in particular because nothing else scales enough, efficiently enough.</p>
Howard Chu @ Symas<p>A reminder that social media platforms are not <a href="https://mastodon.social/tags/LMDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LMDB</span></a> support channels. And just RTFM, instead of getting misinfo from twitter <a href="https://x.com/cachecrab/status/1942295240628404691" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">x.com/cachecrab/status/1942295</span><span class="invisible">240628404691</span></a></p><p>Obviously yes, the docs say LMDB supports multiprocess concurrency. That has always been true.</p>
TrendingLive feeds
About