So I just fell down the rabbit hole of figuring out how to use SQLite with Ceph (turns out a thing called libcephsqlite[0][1] exists) -- awesome to see this new

So I just fell down the rabbit hole of figuring out how to use SQLite with Ceph (turns out a thing called libcephsqlite[0][1] exists) -- awesome to see this new take on distributed SQLite. The caveats for dqlite and rqlite always felt kind of awkward/risky to me -- in stark contrast to SQLite which is so stable/"built in" that you don't think about it's failure modes. Having to worry about what exactly I ran (ex. RANDOM()) was just a non-starter (IIRC rqlite has this problem but not dqlite? or the other way around -- one replicates at statement level the other at WAL level). That said though, the biggest sticking point with all this SQLite goodness is how to make sure that certain libraries (any popular extension -- vsv, spatialite, libcephsqlite) were loaded for any application using SQLite -- there seem to be only a few options: - calling load_extension[2] from code (this is somewhat frowned upon, but maybe it's fine) - LD_PRELOAD (mvsqlite does this) - Building your own SQLite and swapping out shared libs (mvqslite also does this, because statically compiled sqlite is a nuisance) - Trapping/catching calls to dlopen (also basically requires LD_PRELOAD, but I guess you could go custom kernel or whatever) This is probably the one big wart of SQLite -- it's a bit difficult to pull in new interesting extensions. I also found this hack[3] which looks quite interesting for building something more general/reusable... [EDIT] - Also while I'm here, I think FDB is probably one of the most under-rated massive-scale NoSQL databases right now. It gets nearly no press (to be fair because it went closed then open again), but it's casually a massive force behind Apple's services at scale. [0]: https://docs.ceph.com/en/latest/rados/api/libcephsqlite/ https://docs.ceph.com/en/latest/rados/api/libcephsqlite/ [1]: https://github.com/rook/rook/issues/10689 https://github.com/rook/rook/issues/10689 [2]: https://www.sqlite.org/lang_corefunc.html#load_extension https://www.sqlite.org/lang_corefunc.html#load_extension [3]: https://github.com/cventers/sqlite3-preload https://github.com/cventers/sqlite3-preload