3 ms·

You'd need something like a "Sqlite standard" then, with a clear path to how to evolve it taking all stakeholder interests into account. Seems like a job at lea

You'd need something like a "Sqlite standard" then, with a clear path to how to evolve it taking all stakeholder interests into account. Seems like a job at least as difficult as writing the sofware itself. (E.g., lack of a standard is why alternative Python implementations never went anywhere, even though the original cPython is, frankly, dreck from a software quality point of view.)

Scaevolus 6y ago
There is a standard though-- the database file format. You can read and write database files without using any SQLite source code by conforming to the expected format: https://www.sqlite.org/fileformat.html https://www.sqlite.org/fileformat.html It's not even that exotic: a bunch of rows in a simple binary format stored in a btree. The rollback journal / WAL are trickier, but aren't strictly necessary. Implementing arbitrary SQL queries on top of the file format is obviously _much_ harder. Here's an example project doing this-- a pure Go sqlite reader: https://github.com/alicebob/sqlittle https://github.com/alicebob/sqlittle
- pbronez 6y ago
  I wonder if there’s a Presto connector for this file format, such that you could dump a bunch of SQLite files in object storage and run analytics on them like they were parquet. Of course, any such driver should probably just link SQLite itself...
ncmncm 6y ago
SQLite has millions of lines of test code. It would actually be difficult to re-implement it in such a way that it is passes the tests, but is incompatible. Would it be as robust? Absolutely not, unless all the OS interaction code were transcribed directly. But Rust has no compiler that targets anywhere near the number of platforms SQLite is deployed on, and never will. The best that could be hoped for is that most such platforms would no longer be used, by the time the transcription is done.

Scaevolus 6y ago

There is a standard though-- the database file format. You can read and write database files without using any SQLite source code by conforming to the expected format: https://www.sqlite.org/fileformat.html https://www.sqlite.org/fileformat.html It's not even that exotic: a bunch of rows in a simple binary format stored in a btree. The rollback journal / WAL are trickier, but aren't strictly necessary. Implementing arbitrary SQL queries on top of the file format is obviously _much_ harder. Here's an example project doing this-- a pure Go sqlite reader: https://github.com/alicebob/sqlittle https://github.com/alicebob/sqlittle

pbronez 6y ago
I wonder if there’s a Presto connector for this file format, such that you could dump a bunch of SQLite files in object storage and run analytics on them like they were parquet. Of course, any such driver should probably just link SQLite itself...

ncmncm 6y ago

SQLite has millions of lines of test code. It would actually be difficult to re-implement it in such a way that it is passes the tests, but is incompatible. Would it be as robust? Absolutely not, unless all the OS interaction code were transcribed directly. But Rust has no compiler that targets anywhere near the number of platforms SQLite is deployed on, and never will. The best that could be hoped for is that most such platforms would no longer be used, by the time the transcription is done.