5 ms·
Standards like OpenDocument are expected to have multiple independent implementations. I'm not sure such a thing is feasible with SQLite (this is why WebSQL was
by wmf 6y ago
Standards like OpenDocument are expected to have multiple independent implementations. I'm not sure such a thing is feasible with SQLite (this is why WebSQL was rejected).
I'm personally sympathetic to the idea that certain implementation-defined monocultures are OK but I don't think that idea has critical mass.
- usefulcat 6y agoI can think of reasons why re-implementing SQLite might lack appeal, buy why would it be infeasible? Even if the format is poorly- or under- specified (note: I have no idea whether that is true), you could definitely check the source.
- otabdeveloper4 6y agoYou'd need something like a "Sqlite standard" then, with a clear path to how to evolve it taking all stakeholder interests into account. Seems like a job at least as difficult as writing the sofware itself. (E.g., lack of a standard is why alternative Python implementations never went anywhere, even though the original cPython is, frankly, dreck from a software quality point of view.)
- Scaevolus 6y agoThere is a standard though-- the database file format. You can read and write database files without using any SQLite source code by conforming to the expected format: https://www.sqlite.org/fileformat.html https://www.sqlite.org/fileformat.html It's not even that exotic: a bunch of rows in a simple binary format stored in a btree. The rollback journal / WAL are trickier, but aren't strictly necessary. Implementing arbitrary SQL queries on top of the file format is obviously _much_ harder. Here's an example project doing this-- a pure Go sqlite reader: https://github.com/alicebob/sqlittle https://github.com/alicebob/sqlittle
- pbronez 6y agoI wonder if there’s a Presto connector for this file format, such that you could dump a bunch of SQLite files in object storage and run analytics on them like they were parquet. Of course, any such driver should probably just link SQLite itself...
- ncmncm 6y agoSQLite has millions of lines of test code. It would actually be difficult to re-implement it in such a way that it is passes the tests, but is incompatible. Would it be as robust? Absolutely not, unless all the OS interaction code were transcribed directly. But Rust has no compiler that targets anywhere near the number of platforms SQLite is deployed on, and never will. The best that could be hoped for is that most such platforms would no longer be used, by the time the transcription is done.
- otabdeveloper4 6y agoLack of multiple independent implementations means we can't ever switch to a different programing language or radically different system architecture. Nobody is feasibly going to rewrite Sqlite in Rust. So yeah, it is important.
- kristianp 6y agoThat's very interesting. The use case described is for quite a limited subset of sqlite. I imagine a limited query language and kv-store could suffice for the usage described in the article. (Probably could be done in a weekend! wink.)
- GaelFG 6y agoNobody is feasibly going to rewrite Sqlite in Rust. Per (Genuine) curiosity why ? I know (good) database systems are hard to make but do sqlite have particulary 'hard' part which are near impossible to replicate ? Especially since we now have a (I assume) well documented base implementation and an extensive test suite/history of error to avoid ? Is making à file based database that hard,compared to creating new programming language for exemple ?
- fpoling 6y agoThe problem is to write a reliable file database. Firefox got a lot of bugs related to history file corruption until it was replaced with SQLite. The same story was with Subversion where they replaced Berkeley key DB due to reliability issues. This reliability comes from a very deep knowledge of OS API and all their corner cases. Creating a programming language typically involves less corner cases and the bugs in the compiler/runtime are much simple to reproduce (again, typically).
- merijnv 6y ago> Is making à file based database that hard,compared to creating new programming language for exemple ? Yes. I can hack together a trivial compiler in a weekend. A serious (non-optimising) compiler for a complex language is a task you could do in a few months. If you gave me two year I'm not sure I could implement a filesystem API that correctly did something as simple as "atomically append to a file". See also Dan Luu's writing on the topic of filesystems: https://danluu.com/deconstruct-files/ https://danluu.com/deconstruct-files/