6 ms·
SQLite is likely used more than all other database engines combined
- deleted 2y ago[deleted]
- forwardemail 2y ago[dead]
- paradox460 2y agoIn the past, I've used it as a file format for an online application. Users often wanted to download bits and bobs of data from the app, not to view it or edit it on their desktop, but more as backups or to share a particular configuration Previously we'd used a json+zip pair, but the lack of enforced schema became a problem. Switching over to a gzipped sqlite db with a "custom" file extension worked incredibly well
- quasarj 2y agoMost likely, indeed.
- axlee 2y agoOnce that is said, does OSS move forward? Not an inch, good luck getting any change upstream
- tombert 2y agoIt's still OSS, you're allowed to fork and maintain Sqlite if you don't like the upstream merge requirements.
- AlexErrant 2y agoThe test suite is (partially) proprietary. Good luck maintaining a fork for a DB without tests. https://www.sqlite.org/th3.html https://www.sqlite.org/th3.html see bottom
- oneplane 2y agoYou can buy a license, and maintain your public fork while also letting everyone know you validated your fork with the same TH3 as the origin version. You just won't be able to show people your TH3 since I doubt you can buy a license that allows that.
- gwd 2y agoFor instance: https://github.com/tursodatabase/libsql https://github.com/tursodatabase/libsql
- spencerchubb 2y agoOSS does not mean anyone can contribute. It means anyone can view source
- doubloon 2y agoi find it ironic one of the most gatekeepy, closed, walled garden companies in the history of the planet, Bloomberg, is a sponsor of one of the most free most open piece of software ever.
- TZubiri 2y agoBloomberg sells pacakaged public information, they rely on commonwealth and transparent regulation disclosures, they are agents of such transparency. They gatekeep their product sure, but you need to do that if you want to sell it.
- jameshart 2y agoThe majority of these SQLite use cases that the article is celebrating are... as the internal data store in an app where it is locking up user data in a proprietary, closed, gate kept wall garden. This irony has layers. Like an onion.
- out_of_protocol 2y agoDo you use music player? Most, including open source are using SQLite inside. There are open source tiling maps formats like MBTiles, based on SQLite. Just dig around, you can find plenty of examples. It's other way around since database format is well known with tooling available
- jameshart 2y agoI’m not remotely claiming ‘I never use software that has SQLite inside it’. I use loads of programs that do. And that’s precisely the kind of walled garden I’m talking about. Keeping your playlists in a specific music player app’s SQLite database forces you to use that app to search or delete or share those playlists. Heck, some apps use SQLite blob storage to store images or even music files. Open software should use open file formats and store data in the file system where other programs can access it.
- 2y ago
- TZubiri 2y agoIn terms of installations, sure. In terms of total data, probably not. In terms of important data, 100% not. Important data needs guarantees, and its privacy is second to its longevity. Server side and centralized beats client side decentralized for all but sensitive consumer data.
- TheRealPomax 2y agoI'm not sure I buy the "important data" one: my Lightroom catalog, to me, constitutes incredibly important data. Losing it and just having the base files would destroy over a decade of work. It also happens to be a SQLlite3 database. And the same is true for a slew of other applications that (quite rightly) use SQLite databases as their file format. You might be thinking of things like financial transactions, or medical records, but that's not the only kind of important data there is.
- hehdhdjehehegwv 2y agoThe number of societal-wide problems that occur if your Lightroom database gets corrupted is zero. The amount of hell that would be unleashed if the financial systems layers upon layers of database transactions got broken is impossible to comprehend. So if you mean “important” as “necessary for society to function”, then no, your browser bookmark files, contact list, or the other two dozen things your laptop and phone use SQLite for are not important.
- tibbar 2y agoThis is a false equivalence, as one user’s particular usage of SQLite is not comparable to all financial instructions’ databases. Better to compare to the prospect of all smartphones being irreversibly corrupted at once.
- hehdhdjehehegwv 2y agoLiterally all smartphones dying at once is preferable because they are all backed up into heavy duty databases. You can restore every phone from the cloud in this thought experiment. You can not restore a data center from a pile of phones.
- cmur 2y agoI’m curious, is the prevalence of SQLite similar to how PHP is widely used because it’s behind many WordPress sites?
- TheRealPomax 2y agoI'm trying to understand this sentence: it seems to suggest that PHP is only popular because of Wordpress, which just... makes no sense? Wordpress uses PHP because PHP existed, and was good enough to build Wordpress on. And it's certainly not just clinging onto life, hanging around just because Wordpress is built on top of it, today? So it's hard to understand which similarity you're asking about, but SQLite is popular because it does one thing, and does that one thing really well, and that thing also happens to be a perfect fit for complex file formats.
- pkaye 2y agoCouple of major companies like Adobe, Apple, Google, Mozilla use it in their products. That would add up quick. https://www.sqlite.org/famous.html https://www.sqlite.org/famous.html
- duskwuff 2y agoChrome's use of SQLite for history, cookies, and local storage would probably put it in first place on its own. Its extensive use in iOS and Android just guarantees the "win".
- ethagnawl 2y agoIt's also the default database used when creating new projects using many web frameworks, like Rails or (IIRC) Django.
- darby_nine 2y agoI think Sqlite's accessibility and ease of use definitely help. However, the fact you can distribute an entire db with a single file is something that shouldn't be overlooked, nor sqlite's legendary reliability and insane test coverage. Finally the fact you can embed it in memory makes for an amazing testing story.
- qazxcvbnmlp 2y agoSQLite solves a lot of problems well enough that you can focus on other things. Have an mvp you’re not sure you’ll ever have more that 2 users - yep. Storing a little data for an application on the disk and don’t want to write your own schema. Want to teach someone how databases work without setting up a sever - sure.
- achillean 2y agoA few others: - Want to distribute data to users that don't want to manage a server? A lot of people don't want to manage a server and don't need the best possible performance. - Want to take data with you on a thumb drive and work with it offline? It's extremely convenient to be able to use SQLite for an app that has to work offline. - Does the app mostly just read from the database and fit in memory? It's undervalued to just put the entire database into memory so you don't hit the disk and don't introduce network latency. For example, the following website does all enrichment with in-memory SQLite databases: https://shdn.io/analyze?target=ycombinator.com https://shdn.io/analyze?target=ycombinator.com At Shodan, we distribute versions of our datasets as SQLite and they're a popular way to consume the data without having to manage infrastructure.
- talkingtab 2y agoFor niche uses of database I am sure this is true. But if you are not in that niche, then Postgresql is usually a much better choice. For example Firefox uses SQLite, because it wants to store lots of application specific data. It is good for that. Personally, I find the tooling, documentation, familiarity and quality of Postgresql makes it my choice even in situations where SQLite might work.
- mapcars 2y agoThe article is about number of installations, instances across all types of devices
- saulpw 2y agoI think the headline statistic negates the claim that these are "niche" uses of database.
- 0xcde4c3db 2y agoAt some point, such local usages were sufficiently non-niche that the temp file extension was changed from "sqlite" to "etilqs" to stem the flood of angry emails about local application databases taking up too much disk space. (cf. Daniel Stenberg getting angry emails because everybody and their dog embeds libcurl, sometimes multiple times in the same app)
- simonw 2y agoHere's the commit from 18 years ago explaining that change: https://github.com/sqlite/sqlite/commit/fd288f3549a1ab9a309a9e120d46319d42adea29#diff-d7c6125271929f399b173406a7151cd2734ceca66f1045b77f1795a37dc8c5da https://github.com/sqlite/sqlite/commit/fd288f3549a1ab9a309a... ** 2006-10-31: The default prefix used to be "sqlite_". But then ** Mcafee started using SQLite in their anti-virus product and it ** started putting files with the "sqlite" name in the c:/temp folder. ** This annoyed many windows users. Those users would then do a ** Google search for "sqlite", find the telephone numbers of the ** developers and call to wake them up at night and complain. ** For this reason, the default name prefix is changed to be "sqlite" ** spelled backwards. So the temp files are still identified, but ** anybody smart enough to figure out the code is also likely smart ** enough to know that calling the developer will not help get rid ** of the file.
- foobarian 2y agoI wonder what happened to bdb. I guess maybe authors lost interest/sold to Oracle and the license changed development just didn't keep up with the times. Bdb used to be the sqlite before sqlite existed and ended up used by various embedded applications or even graphical interfaces e.g. to index file thumbnails and such. I'm not sure but I think MacOS at one point used it this way.
- lm411 2y agoI haven't even thought of Berkeley DB in years. Used to use it quite a bit but most notably perhaps with apache mod_auth_db to handle authentication and subscriber information on a fairly high traffic site (yes, this was long long ago ;) ).
- sargun 2y agoI love SQLite. It’s pluggable interfaces and concise C code make it enjoyable to use (and sometimes to debug - as long as you don’t have to go through the single file version!). Unfortunately, I wish there were multiple implementations of it, and that the file format was documented and stable.
- ankitrgadiya 2y agoThe sqlite file format is documented and probably has the most insane backwards compatibility out there. The developers pledged to keep it backwards compatible until 2050. In fact Library of Congress has recognised it as one of the few data formats for long term archival. https://sqlite.org/fileformat2.html https://sqlite.org/fileformat2.html https://sqlite.org/lts.html https://sqlite.org/lts.html https://sqlite.org/locrsf.html https://sqlite.org/locrsf.html
- albertopv 2y agoThere are far more ants in the world than human beings. So what?
- mharig 2y agoAstonishing no commenter mentions that SQLite unleashes the power (and pain) of SQL for what were in ancient times dark binary blobs or obscure text formats. I.e. guaranteed consistency, easy versioning with better up/down compatibility, excellent complex retrievals, ...