5 ms·

The problem is that SQLite already exists, and is public domain. It's a fantastic piece of software engineering that has had a lot of time and effort put into m

The problem is that SQLite already exists, and is public domain. It's a fantastic piece of software engineering that has had a lot of time and effort put into making it great. The pitch for these SQLite alternatives is: - SQLite is public domain so there's no problem with us rewriting it - We're going to rewrite it in Rust because that's going to make it inherently better (and don't question this) - We're going to MIT license our rewrite because there's various reasons why people would rather use MIT-licensed code than public domain code (but SQLite being public domain means we get to do all of this rewriting and re-licensing to begin with) - Also we're going to extend SQLite to be “cloud-capable” because that's our business use-case, even though it's completely at odds with SQLite's intended use-case - Also we're going to foster a “community” around our rewrite(-in-progress) (because people still think this is something desirable for some reason, as though a nonzero part of SQLite's greatness is that it operates entirely without having do deal with “nerd drama” that such “communities” inevitably breed) - Also, we replaced that pesky, unsavory-to-our-sensibilities “Code of Ethics” with a bog-standard “Code of Conduct”—because, again: “community” - But we're not going to even attempt to replicate the rigorous, arduous, and closed-source testing that goes into developing SQLite (which really makes up the bulk of its engineering effort) - We've made some progress toward all of this, but it's nowhere near done yet - But we're announcing it now anyway, because “hype” and “community” and “modern” - Also, look at our microbenchmarks that show that our unfinished SQLite rewrite is already faster than SQLite (because we haven't reimplemented everything from SQLite yet) (and also we don't plan to reimplement everything from SQLite anyway) I find it really odd that I can only seems to find criticism of any of this here on HN, and in a couple of reddit threads. It's kind of like when there was that Godot controversy awhile back, and some people made a fork and announced it, despite being identical aside from the Code of Conduct. Merely announcing a fork/rewrite of existing open-source software as a means of "building hype" and “creating a community”, while benefiting from untold man-hours of work done by others, strikes me personally as rather untoward behavior—regardless of whether the reasoning behind said fork/rewrite is ideological (Redot) or business (libSQL/“Limbo”). Software—especially software used to create other software—should be lauded for its engineering efforts, rather than its ability to “build hype” or “build and foster online ‘community’ ‘engagement’”. If SQLite was abandonware, then it would be a somewhat different story, but SQLite is an amazing piece of software—perhaps the most widely-used on the planet—that is still being actively developed and supported. So, piggybacking on its success, without having concrete, demonstrable improvements to its functionality, comes across as… well, I can't think of another polite, applicable adjective than “untoward”.

nindalf 2y ago
> we're not going to even attempt to replicate the rigorous, arduous, and closed-source testing Almost everything you've said is wrong, but this one is trivially easy to prove wrong. You say "not even going to attempt" but they plan to take testing seriously from the get go. They say: > Since this is a reimplementation, doesn’t that mean that testing is now even harder? The reality is that it is the other way around. Since we are reimplementing it from scratch, we are doing it with Deterministic Simulation Testing (DST) built-in from the get-go. We have both added DST facilities to the core of the database, and partnered with Antithesis to achieve a level of reliability in the database that lives up to SQLite’s reputation. > Deterministic Simulation Testing is a paradigm made famous by the folks at TigerBeetle, that we at Turso already dipped our toes into with our server-side code. With DST, we believe we can achieve an even higher degree of robustness than SQLite, since it is easier to simulate unlikely scenarios in a simulator, test years of execution with different event orderings, and upon finding issues, reproduce them 100% reliably. Did you make your false claim in ignorance of their announcement post (https://turso.tech/blog/introducing-limbo-a-complete-rewrite-of-sqlite-in-rust https://turso.tech/blog/introducing-limbo-a-complete-rewrite...), or did you make it knowing full well that they're trying hard to make a well tested library? --- But fuck it, I got time. I'll tackle the other dubious claims you've made as well. > Also, we replaced that pesky, unsavory-to-our-sensibilities “Code of Ethics” They haven't replaced it because Limbo is a new project. LibSQL is a fork of an existing project but again, they haven't replaced the "Code of Ethics" in the repo (https://sqlite.org/src/dir?ci=trunk https://sqlite.org/src/dir?ci=trunk) because it never existed in the repo, just on the sqlite.org website. So they "replaced" nothing. They simply added their own code of conduct, which (I agree with you) is bog standard for a project on Github. --- > Announcing before it's ready Yes? That's normal? You're objecting to a project even existing on Github if it's not 100% done? They specifically explain why they've done it. They're building in the open, they're seeing excitement from outside contributors who want to contribute. So they're announcing to make more potential contributors aware this project exists. Only someone with a warped mind could find something objectionable about starting small and iterating with the community. --- > Also we're going to extend SQLite to be “cloud-capable” because that's our business use-case, even though it's completely at odds with SQLite's intended use-case Nothing you've said makes me think you understand the social side of building software, but this makes me think you don't understand the technical side either. Perhaps you missed the link on this thread where they explain why SQLite works well in the cloud. I'll explain for you though, in case you're unable to find the link at the top of this page. SQLite relies on random access from disk, which made it a poor fit for cloud workloads when the cloud used HDDs exclusively. HDDs work much better for sequential access, which is why LSM trees were invented. So rather than reading from a random part of the local disk (10-20ms) you'd rather pay the network cost (0.5-2ms) and read from a database on a different machine that has optimised it's data storage for sequential read. But SSDs change the game. A random read of 4k from a local SSD is 150 microseconds (https://gist.github.com/jboner/2841832 https://gist.github.com/jboner/2841832), always less than the network round trip. Local SQLite running on an SSD is suddenly the lowest latency database. For infrequently changing data that needs to be served with low latency, SQLite is the best choice in the cloud. If you don't follow this, it's alright. I didn't expect you to. --- > SQLite is public domain so there's no problem with us rewriting it and We're going to MIT license our rewrite because there's various reasons why people would rather use MIT-licensed code than public domain code Yes, this is true. I don't think this was an objection from you. --- > We're going to rewrite it in Rust because that's going to make it inherently better (and don't question this) What should they be writing it in? I notice you don't say. You reckon they should start a new C code base in 2024? --- > Redot comparison Redot is a fork created solely because the creators didn't like a tweet about Godot being welcoming to LGBT game developers. LibSQL was created because they wanted to add features to the SQLite codebase and weren't allowed. In any case the comparison is wrong. You can see the two commit histories of redot (https://github.com/Redot-Engine/redot-engine/commits/master/)and https://github.com/Redot-Engine/redot-engine/commits/master/... libsql (https://github.com/tursodatabase/libsql/commits/main/ https://github.com/tursodatabase/libsql/commits/main/) and see which of these is actually a community creating something new and which is a dead project simply merging in commits from upstream. --- No one disagrees that SQLite is highly useful, rock-solid reliable software. But I disagree with you that it is perfect in every way and no one should ever second guess any decision made by the maintainers. Turso wants to try something different. Nothing wrong with trying.

nindalf 2y ago

> we're not going to even attempt to replicate the rigorous, arduous, and closed-source testing Almost everything you've said is wrong, but this one is trivially easy to prove wrong. You say "not even going to attempt" but they plan to take testing seriously from the get go. They say: > Since this is a reimplementation, doesn’t that mean that testing is now even harder? The reality is that it is the other way around. Since we are reimplementing it from scratch, we are doing it with Deterministic Simulation Testing (DST) built-in from the get-go. We have both added DST facilities to the core of the database, and partnered with Antithesis to achieve a level of reliability in the database that lives up to SQLite’s reputation. > Deterministic Simulation Testing is a paradigm made famous by the folks at TigerBeetle, that we at Turso already dipped our toes into with our server-side code. With DST, we believe we can achieve an even higher degree of robustness than SQLite, since it is easier to simulate unlikely scenarios in a simulator, test years of execution with different event orderings, and upon finding issues, reproduce them 100% reliably. Did you make your false claim in ignorance of their announcement post (https://turso.tech/blog/introducing-limbo-a-complete-rewrite-of-sqlite-in-rust https://turso.tech/blog/introducing-limbo-a-complete-rewrite...), or did you make it knowing full well that they're trying hard to make a well tested library? --- But fuck it, I got time. I'll tackle the other dubious claims you've made as well. > Also, we replaced that pesky, unsavory-to-our-sensibilities “Code of Ethics” They haven't replaced it because Limbo is a new project. LibSQL is a fork of an existing project but again, they haven't replaced the "Code of Ethics" in the repo (https://sqlite.org/src/dir?ci=trunk https://sqlite.org/src/dir?ci=trunk) because it never existed in the repo, just on the sqlite.org website. So they "replaced" nothing. They simply added their own code of conduct, which (I agree with you) is bog standard for a project on Github. --- > Announcing before it's ready Yes? That's normal? You're objecting to a project even existing on Github if it's not 100% done? They specifically explain why they've done it. They're building in the open, they're seeing excitement from outside contributors who want to contribute. So they're announcing to make more potential contributors aware this project exists. Only someone with a warped mind could find something objectionable about starting small and iterating with the community. --- > Also we're going to extend SQLite to be “cloud-capable” because that's our business use-case, even though it's completely at odds with SQLite's intended use-case Nothing you've said makes me think you understand the social side of building software, but this makes me think you don't understand the technical side either. Perhaps you missed the link on this thread where they explain why SQLite works well in the cloud. I'll explain for you though, in case you're unable to find the link at the top of this page. SQLite relies on random access from disk, which made it a poor fit for cloud workloads when the cloud used HDDs exclusively. HDDs work much better for sequential access, which is why LSM trees were invented. So rather than reading from a random part of the local disk (10-20ms) you'd rather pay the network cost (0.5-2ms) and read from a database on a different machine that has optimised it's data storage for sequential read. But SSDs change the game. A random read of 4k from a local SSD is 150 microseconds (https://gist.github.com/jboner/2841832 https://gist.github.com/jboner/2841832), always less than the network round trip. Local SQLite running on an SSD is suddenly the lowest latency database. For infrequently changing data that needs to be served with low latency, SQLite is the best choice in the cloud. If you don't follow this, it's alright. I didn't expect you to. --- > SQLite is public domain so there's no problem with us rewriting it and We're going to MIT license our rewrite because there's various reasons why people would rather use MIT-licensed code than public domain code Yes, this is true. I don't think this was an objection from you. --- > We're going to rewrite it in Rust because that's going to make it inherently better (and don't question this) What should they be writing it in? I notice you don't say. You reckon they should start a new C code base in 2024? --- > Redot comparison Redot is a fork created solely because the creators didn't like a tweet about Godot being welcoming to LGBT game developers. LibSQL was created because they wanted to add features to the SQLite codebase and weren't allowed. In any case the comparison is wrong. You can see the two commit histories of redot (https://github.com/Redot-Engine/redot-engine/commits/master/)and https://github.com/Redot-Engine/redot-engine/commits/master/... libsql (https://github.com/tursodatabase/libsql/commits/main/ https://github.com/tursodatabase/libsql/commits/main/) and see which of these is actually a community creating something new and which is a dead project simply merging in commits from upstream. --- No one disagrees that SQLite is highly useful, rock-solid reliable software. But I disagree with you that it is perfect in every way and no one should ever second guess any decision made by the maintainers. Turso wants to try something different. Nothing wrong with trying.