4 ms·

I think beyond the historical reasons why C was the best choice when SQLite was being developed, or the advantages it has today, there's also just no reason to

I think beyond the historical reasons why C was the best choice when SQLite was being developed, or the advantages it has today, there's also just no reason to rewrite SQLite in another language. We don't have to have one implementation of a lightweight SQL database. You can go out right now and start your own implementation in Rust or C++ or Go or Lisp or whatever you like! You can even make compatible APIs for it so that it can be a drop-in replacement for SQLite! No one can stop you! You don't need permission! But why would we want to throw away the perfectly good C implementation, and why would we expect the C experts who have been carefully maintaining SQLite for a quarter century to be the ones to learn a new language and start over?

turtletontine 1y ago
Thanks for this, I fully agree. One frustration I have with the modern moment is the tendency to view anything more than five years old with disdain, as utterly irrelevant and obsolete. Maybe I’m just going old, but I like my technology dependable and boring, especially software. Glad to see someone express respect for the decades of expertise that have gone into things we take for granted.
- friendly_wizard 1y ago
  I think we owe an equally proportionate measure of respect to the authors of boring old sqlite who, given the depth of their experience, recognize that there may in fact be benefits to be gained from the rewrite and are open to exploring the possibility. The blockers as stated, I have no doubt, were carefully considered with a level of insight few of us could match. If and when the time is right, if they choose to undertake the effort I'm sure the juice will be worth the squeeze. The fact that they're not jumping in blindly today is telling. Even more telling will be if they do eventually go that route.
AdamJacobMuller 1y ago
One good reason is that people have written golang adapters, so that you can use sqlite databases without cgo. I agree to what I think you're saying which is that "sqlite" has, to some degree, become so ubiquitous that it's evolved beyond a single implementation. We, of course, have sqlite the C library but there is also sqlite the database file format and there is no reason we can't have an sqlite implementation in golang (we already do) and one in pure rust too. I imagine that in the future that will happen (pure rust implementation) and that perhaps at some point much further in the future, that may even become the dominant implementation.
- zimpenfish 1y ago
  > One good reason is that people have written golang adapters, so that you can use sqlite databases without cgo. There's also the Go-wrapped WASM build of the C sqlite[0] which is handy. [0] https://github.com/ncruces/go-sqlite3 https://github.com/ncruces/go-sqlite3
glandium 1y ago
And, in fact, these implementations exist. At least in Rust, there's rqlite and turso.
- otoolep 1y ago
  rqlite[1] author here. Just to be clear, rqlite is not SQLite but rewritten in Go. rqlite uses the vanilla C code, and calls it from Go[2]. I consider that an important advantage over other approaches -- rqlite gets all the benefits of rock-solid[3] SQLite. As result there are no questions about the quality of the database engine. [1] https://rqlite.io https://rqlite.io [2] https://rqlite.io/docs/design/ https://rqlite.io/docs/design/ [3] https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html
jacquesm 1y ago
> But why would we want to throw away the perfectly good C implementation, and why would we expect the C experts who have been carefully maintaining SQLite for a quarter century to be the ones to learn a new language and start over? Because a lot of language advocacy has degraded to telling others what you want them to do instead of showing by example what to do. The idea behind this is that language adoption is some kind of zero sum game. If you're developing project 'x' in language 'y' then you are by definition not developing it in language 'z'. This reduces the stature of language 'z' and the continued existence of project 'x' in spite of not being written in language 'z' makes people wonder if language 'z' is actually as much of a necessity as its proponents claim. And never mind the fact that if the decision in what language 'x' would be written were to be revisited by the authors of 'x' that not only language 'z' would be on the menu, but also languages 'd', 'l', 'j' and 'g'.
- waterTanuki 1y ago
  Given the common retort for why not try X project in Y new language is "it's barely used in other things. Let's wait and see it get industry adoption before trying it out" it's hard to see it as anything OTHER than a zero-sum game. As much as I like Rust I recognize some things like SQLite are better off in C. But the reason you see so much push for some new languages is because if they don't get and maintain regualr adoption, they will die off.
  - jacquesm 1y ago
    Plenty of programming languages gained mass adoption without such tactics.
  - john_the_writer 1y ago
    Yeah.. I always remind myself of the netscape browser. A lesson in "if it's working to mess with it" My question is always the reverse. Why try it in Y new language. Is there some feature that Y provides that was missing in X? How often do those features come up. Company I worked for decided to build out a new microservice in language Y. The whole company was writing in W and X, but they decided to write the new service in Y. When something goes wrong, or a bug needs fixing, 3 people in the company of over 100 devs know Y. Guess what management is doing.. Re-writing it in X.
eusto 1y ago
I think that if SQLite would suddenly have to add a bunch of new features, the discussion about rewriting it would be very relevant. I think we like to fool ourselves that decisions like these are based on performance considerations or maintainability or whatever, but in reality they would be based on time to market and skill availability in the areas where the team is being built. At the end of the day, SQLite is not being rewritten because the cost of doing so is not justifiable
- RhysU 1y ago
  These guys are, after all, running a business. If they thought the best thing for their business was a rewrite, they'd do it.
biohazard2 1y ago
> there's also just no reason to rewrite SQLite in another language. […] But why would we want to throw away the perfectly good C implementation, and why would we expect the C experts who have been carefully maintaining SQLite for a quarter century to be the ones to learn a new language and start over? The SQLite developers are actually open to the idea of rewriting SQLite in Rust, so they must see an advantage to it: > All that said, it is possible that SQLite might one day be recoded in Rust. Recoding SQLite in Go is unlikely since Go hates assert(). But Rust is a possibility. Some preconditions that must occur before SQLite is recoded in Rust include: […] If you are a "rustacean" and feel that Rust already meets the preconditions listed above, and that SQLite should be recoded in Rust, then you are welcomed and encouraged to contact the SQLite developers privately and argue your case.
- yomismoaqui 1y ago
  My theory is they wrote this just to get the ‘rewrite everything in Rust’ crowd off their backs.
  - rirze 1y ago
    I think it’s the opposite. They want to atleast explore rewriting in Rust but are afraid of backlash. Hence why they’re open to private discussion. I can imagine they are split internally.
  - biohazard2 1y ago
    I find it a bit too specific, because it won't get rid of the `rewrite everything in (Go|Zig|…)` crowds. But who knows…?
etruong42 1y ago
To build excitement in a project and potentially release new versions with new features, all more safely than adding C code.
wolvesechoes 1y ago
> there's also just no reason to rewrite SQLite in another language But think about all those karma points here and on Reddit, or GitHub stars!

turtletontine 1y ago

Thanks for this, I fully agree. One frustration I have with the modern moment is the tendency to view anything more than five years old with disdain, as utterly irrelevant and obsolete. Maybe I’m just going old, but I like my technology dependable and boring, especially software. Glad to see someone express respect for the decades of expertise that have gone into things we take for granted.

friendly_wizard 1y ago
I think we owe an equally proportionate measure of respect to the authors of boring old sqlite who, given the depth of their experience, recognize that there may in fact be benefits to be gained from the rewrite and are open to exploring the possibility. The blockers as stated, I have no doubt, were carefully considered with a level of insight few of us could match. If and when the time is right, if they choose to undertake the effort I'm sure the juice will be worth the squeeze. The fact that they're not jumping in blindly today is telling. Even more telling will be if they do eventually go that route.

AdamJacobMuller 1y ago

One good reason is that people have written golang adapters, so that you can use sqlite databases without cgo. I agree to what I think you're saying which is that "sqlite" has, to some degree, become so ubiquitous that it's evolved beyond a single implementation. We, of course, have sqlite the C library but there is also sqlite the database file format and there is no reason we can't have an sqlite implementation in golang (we already do) and one in pure rust too. I imagine that in the future that will happen (pure rust implementation) and that perhaps at some point much further in the future, that may even become the dominant implementation.

zimpenfish 1y ago
> One good reason is that people have written golang adapters, so that you can use sqlite databases without cgo. There's also the Go-wrapped WASM build of the C sqlite[0] which is handy. [0] https://github.com/ncruces/go-sqlite3 https://github.com/ncruces/go-sqlite3

glandium 1y ago

And, in fact, these implementations exist. At least in Rust, there's rqlite and turso.

otoolep 1y ago
rqlite[1] author here. Just to be clear, rqlite is not SQLite but rewritten in Go. rqlite uses the vanilla C code, and calls it from Go[2]. I consider that an important advantage over other approaches -- rqlite gets all the benefits of rock-solid[3] SQLite. As result there are no questions about the quality of the database engine. [1] https://rqlite.io https://rqlite.io [2] https://rqlite.io/docs/design/ https://rqlite.io/docs/design/ [3] https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html

jacquesm 1y ago

> But why would we want to throw away the perfectly good C implementation, and why would we expect the C experts who have been carefully maintaining SQLite for a quarter century to be the ones to learn a new language and start over? Because a lot of language advocacy has degraded to telling others what you want them to do instead of showing by example what to do. The idea behind this is that language adoption is some kind of zero sum game. If you're developing project 'x' in language 'y' then you are by definition not developing it in language 'z'. This reduces the stature of language 'z' and the continued existence of project 'x' in spite of not being written in language 'z' makes people wonder if language 'z' is actually as much of a necessity as its proponents claim. And never mind the fact that if the decision in what language 'x' would be written were to be revisited by the authors of 'x' that not only language 'z' would be on the menu, but also languages 'd', 'l', 'j' and 'g'.

waterTanuki 1y ago
Given the common retort for why not try X project in Y new language is "it's barely used in other things. Let's wait and see it get industry adoption before trying it out" it's hard to see it as anything OTHER than a zero-sum game. As much as I like Rust I recognize some things like SQLite are better off in C. But the reason you see so much push for some new languages is because if they don't get and maintain regualr adoption, they will die off.
- jacquesm 1y ago
  Plenty of programming languages gained mass adoption without such tactics.
- john_the_writer 1y ago
  Yeah.. I always remind myself of the netscape browser. A lesson in "if it's working to mess with it" My question is always the reverse. Why try it in Y new language. Is there some feature that Y provides that was missing in X? How often do those features come up. Company I worked for decided to build out a new microservice in language Y. The whole company was writing in W and X, but they decided to write the new service in Y. When something goes wrong, or a bug needs fixing, 3 people in the company of over 100 devs know Y. Guess what management is doing.. Re-writing it in X.

eusto 1y ago

I think that if SQLite would suddenly have to add a bunch of new features, the discussion about rewriting it would be very relevant. I think we like to fool ourselves that decisions like these are based on performance considerations or maintainability or whatever, but in reality they would be based on time to market and skill availability in the areas where the team is being built. At the end of the day, SQLite is not being rewritten because the cost of doing so is not justifiable

RhysU 1y ago
These guys are, after all, running a business. If they thought the best thing for their business was a rewrite, they'd do it.

biohazard2 1y ago

> there's also just no reason to rewrite SQLite in another language. […] But why would we want to throw away the perfectly good C implementation, and why would we expect the C experts who have been carefully maintaining SQLite for a quarter century to be the ones to learn a new language and start over? The SQLite developers are actually open to the idea of rewriting SQLite in Rust, so they must see an advantage to it: > All that said, it is possible that SQLite might one day be recoded in Rust. Recoding SQLite in Go is unlikely since Go hates assert(). But Rust is a possibility. Some preconditions that must occur before SQLite is recoded in Rust include: […] If you are a "rustacean" and feel that Rust already meets the preconditions listed above, and that SQLite should be recoded in Rust, then you are welcomed and encouraged to contact the SQLite developers privately and argue your case.

yomismoaqui 1y ago
My theory is they wrote this just to get the ‘rewrite everything in Rust’ crowd off their backs.
- rirze 1y ago
  I think it’s the opposite. They want to atleast explore rewriting in Rust but are afraid of backlash. Hence why they’re open to private discussion. I can imagine they are split internally.
- biohazard2 1y ago
  I find it a bit too specific, because it won't get rid of the `rewrite everything in (Go|Zig|…)` crowds. But who knows…?

etruong42 1y ago

To build excitement in a project and potentially release new versions with new features, all more safely than adding C code.

wolvesechoes 1y ago

> there's also just no reason to rewrite SQLite in another language But think about all those karma points here and on Reddit, or GitHub stars!