3 ms·
Something really neat about SQLite is how self contained the database file is.. You can take say, a large book or log output, then copy it into a table with ful
by Rapzid 4y ago
Something really neat about SQLite is how self contained the database file is.. You can take say, a large book or log output, then copy it into a table with full text search. Optimize the DB to get rid of empty space, then ship that out to S3 or something. Now you can fetch that down and have indexed and fts access to the contents.
Cool way to store data with indexes into it in a way that makes it as easy to use as copying around and opening a file; no need for dumps or restores or server processes.
- encryptluks2 4y agoYou can also do the same with a file and you can even open it in plain text and read it too.
- Rapzid 4y agoI mean, the database IS a file. The key here is that it colocates indexes and other metadata that can be used to efficiently access large data sets without having to hand-roll indexes and/or process them on-the-fly everytime you load the data up...
- VHRanger 4y agoSQLite is made to replace "dread" by being generally better than storing the data as a json blob or csv There's still an argument for columnar formats like arrow, but compatibility is iffy compared to SQLite which "just works"
- bob1029 4y agoMost problems can be neatly packed into 1 sqlite database. Even a big AAA video game could benefit from this type of representation on disk. I am also now starting to look at SQLite as a sort of application framework / operating system. The SQLite VM is running on billions of devices right now without any issues. Seems like a safe bet if I wanted to use the relational/SQL model to represent most of my application state & logic. Anything else I want to bring to the party I can inject by way of application-defined functions.
- thristian 4y agoEver since I read the "Out of the Tar Pit" paper[1], I've been designing my data structures in a more relational form. [1]: https://github.com/papers-we-love/papers-we-love/blob/master/design/out-of-the-tar-pit.pdf https://github.com/papers-we-love/papers-we-love/blob/master...
- bob1029 4y agoThis paper was a very big deal for us. I thought “oh this is nice” and then I got to chapter 9... SQL is so much more powerful than most developers are aware. It's not just some annoyance to trod through in order to get to your data. It is an entire data-as-computation paradigm that also happens to be a domain specific language. SQL is functional. It is also relational. Figuring out how to combine both of these properties across your entire domain is peak elegance. We got pretty damn close on our current iteration. The walls of the matrix are a bit messy, but from inside a given SQL expression you can't tell. It all looks very clean from the inside. SQLite represents the most stable and portable foundation upon which to construct one of these systems. Nothing else comes close.
- chousuke 4y agoI like it just because it's a single file (most of the time). I can put pretty much anything in it without having to think about how to store data on disk in a sane manner, and SQLite can handle a lot of data. You get so many good things for free: transactionality, durability, concurrent access, and an easy consistent backup mechanism. Even if JSON blobs happen to be the most convenient way to represent your data, you can still just put those blobs in an SQLite database and maybe worry about converting them into something relational later.