10 ms·

I strongly suspect this article was written as a response against C++, Java, or C#. Especially given SQLite being started in 2000, C really was undeniably the b

• Sir_Cmpwn 9y ago
  They are both weak contenders, for the reasons laid out in the article. Go and Rust have large runtimes with lots of dependencies, and Go is not as performant as C. Literally every system has a C compiler, except maybe for a very small number of very old and niche systems. Assuming it cound fit on the ROM, sqlite could probably be ported to my z80 calculator, which has an 8 bit processor and 128 kiolobytes of RAM - trivially! C runs literally everywhere, which is something no other programming language can lay claim to. For a tool like sqlite there is no other choice, period.
  • steveklabnik 9y ago
    Rust has an equivalent amount of runtime to C. Our platform support is limited by LLVM though, this is a good reason for sure.
    • Sir_Cmpwn 9y ago
      Fair, I forgot that Rust has good freestanding support. I still don't think it'd be a good candidate for sqlite, though, for one reason in particular: Rust is not competitive with C on portability.
      • dom96 9y ago
        Which is why Nim would be a good candidate. Compiles to C so has the benefits of its portability, interoperability and speed.
        • Sir_Cmpwn 9y ago
          Oh please please no! Nim is almost impossible to debug at a low level and you have no fine-grained control over its performance or behavior. It's also far less well understood than C, which is more boring and the subject of decades of conservative development. I'm also not sure what kind of support nim has for freestanding programs.
          • dom96 9y ago
            Can you elaborate? I've never had problems debugging it, you can use lldb or gdb to do it. You have a hell of a lot of control over its performance and behaviour. Are there instances where you found that wasn't the case? As far as the understanding of C goes, I can agree with that. But we are discussing rewriting sqlite in something else after all.
            • Sir_Cmpwn 9y ago
              Maybe you can reach out via email, it's in my profile. I'd rather not tear into Nim in this thread, we're getting pretty off-topic.
              • dom96 9y ago
                Done.
        • fiedzia 9y ago
          Doesn't it come with GC? That would limit embedding.
          • dom96 9y ago
            That's true. You do have the option to disable it though. Seeing as Go was suggested above (which is also garbage collected), I figured I would mention Nim too.
        • rwbt 9y ago
          Interesting. I love programming in Nim but I'm not sure if I would consider developing something like SQLite in Nim. It still has a few rough edges IMHO.
      • steveklabnik 9y ago
        Yes, agreed. It's not a real answer yet, but mrustc can compile Rust to C, so if this becomes a really huge blocker, we could put in some elbow grease and make it a real answer. The question is mostly "is that worth the time or not?"
    • falcolas 9y ago
      I was under the impression that the support for unrecoverable errors via panic! creates a runtime structure (allowing for the explicit release of resources) which is not typically present in C programs (unless manually included).
      • steveklabnik 9y ago
        Honestly, "runtime" is just a complex word in general, like, most people refer to C as having "no runtime" even though it does. Regardless, those are called "landing pads", and you can compile with an option to turn them off. Many do.
• agentultra 9y ago
  You would have to start at languages that are demonstrably better than C at being C for the reasons listed as to why C is still the best language for SQLite. In the present day context of vulnerabilities it's tempting to blame C. Yet it's still a good choice for many reasons. It's not wise to suggest that everything written in C must be re-written in some other language because of hand-waving reasons like buffer-overflows or off-by-one errors. Maybe you could prove Rust is a good choice by writing your own SQLite implementation feature for feature? Until then I don't think there's going to be a compelling reason to re-write SQLite because it's written in C.
  • kyberias 9y ago
    But... who proposed a re-write?
    • stochastic_monk 9y ago
      Exactly. At this point, the strongest reason for SQLite to remain in C is that it is in C.
  • peatmoss 9y ago
    > Maybe you could prove Rust is a good choice by writing your own SQLite implementation feature for feature? A person undertaking that task would already have a head start in that they could likely use the extensive test suite for SQLite ;-)
  • imtringued 9y ago
    >It's not wise to suggest that everything written in C must be re-written in some other language because of hand-waving reasons like buffer-overflows or off-by-one errors. These security vulnerabilities are trivial to fix at the language level but cause extremely negative consequences if exploited successfully. Yet the default stance of C programmers is to not adress them at all. A competitor like Rust absolutely becomes necessary because of the complacency of C programmers. They will then will strongly critizise the newcomer that is built on strong fundamentals as "language of the week" that is chosen by "dumber programmers" that are trying to ride the latest hype train.
• ex3ndr 9y ago
  Go and Rust is too heavy and require more dependencies for SQLite. SQLite runs even on the smallest exotic embedded systems.
  • nickez 9y ago
    You can run Rust on atmegas. It doesn't get much more small, but it can get a lot more exotic. https://github.com/avr-rust/rust https://github.com/avr-rust/rust
    • jeffreyrogers 9y ago
      But then you're forced to use the rust compiler. Sqlite can be included in any project that can use a decent C compiler.
      • eropple 9y ago
        Why would you be forced to use the Rust compiler? You can generate C headers for a static Rust image.
        • jeffreyrogers 9y ago
          It brings the Rust compiler in as a dependency. I'm not saying you have to use Rust for the rest of the project, but it complicates things in a way that having C source code that you drop into your project doesn't.
          • steveklabnik 9y ago
            I think you two are talking at cross-purposes. You are correct that building the project would require Rust. Your parent is correct that, with a pre-built binary, you don't need any Rust-specific stuff installed to make this work. Building from source is usual, of course, given the lack of a stable ABI.
    • baldfat 9y ago
• bringtheaction 9y ago
  Probably originally written as a response against C++, Java or C# yes. However it’s also been kept up to date because it mentions Swift, so I think they don’t want to rewrite it in Rust or Go. But I agree that Rust might have been a good fit if SQLite had been created today. There is the point that sibling commenter made about small embedded systems though. But some people work on making Rust useful even for microcontrollers so who knows maybe a SQLite in Rust could have been highly portable too?
  • jeffreyrogers 9y ago
    Sqlite is often just dropped into a project as source code. You can't do that if it were written in Rust.
    • nicoburns 9y ago
      You can't do that with C code either unless your project is in C (or C++?).
      • jeffreyrogers 9y ago
        Sure, but for many embedded systems you have to use C. Plus C interoperates well with every other mainstream language. Every platform supports C, so by writing it in C you can support every platform (with very few caveats).
• mitchty 9y ago
  > That being said, Rust and possibly even Go would be strong contenders to make a new SQLite-like library/program today. At least on the Rust side, the C bindings are excellent too. Doubtful, rust and go do not run on nearly as many platforms as c and sqlite does. I get everyone wants to use other languages, but this incessant "rewrite in rust/go/$THING" is getting annoying. I'm going to try to coin a new online discussion rule of programming language posts: - At some point someone is going to suggest any problems for one language can be solved by rewriting in another language. In this case "because c" "therefore rust". Its basically godwins law for programming language discussion.
  • lukebitts 9y ago
    Keyword is "SQLite-like", you are the one bringing "rewrite in rust" up.
    • mitchty 9y ago
      Seems like a distinction without a difference to rewrite "sqlite in c" to "not sqlite but sqlite-like in rust or go".
      • lukebitts 9y ago
        How is it not different? It would be a completely new library. I don't see what the argument against that is.
        • mitchty 9y ago
          You're free to do so, as is anyone. But you'll have to overcome and provide a good reason for why anyone using sqlite should move to your new unproven shiny. If anything sqlite is a poster child for c done (and tested/validated) right. You'll have to demonstrate a lot more than you can implement some things better in the new language.
          • lukebitts 9y ago
• bluejekyll 9y ago
  Lets not create a false equivalency between Go and C. Go is a good language, one of its core pieces is to increase memory safety through the use of a garbage collector. Much of what the SQLite post states may not be safe operations in Go. Rust would be a safer alternative that meets many of the SQLite requirements. The thing is, SQLite is bulletproof at this point, so do we need to replace it?
  • chungy 9y ago
    > The thing is, SQLite is bulletproof at this point, so do we need to replace it? No, probably not, it is fine as it is. Still, even SQLite started out as a "for fun" project. Who knows what might happen in the future to displace it? ;)
    • tangue 9y ago
      Sqllite didn’t start for fun it was built for navy destroyers and submarines. (1) that’s probably why parent called it bulletproof. (1) https://en.wikipedia.org/wiki/SQLite#History https://en.wikipedia.org/wiki/SQLite#History
      • mzzter 9y ago
        It could also be a reference to the comprehensive SQLite test suite (1) or the fact that it has been in production systems for many years, though it’s initial purpose does make bulletproof quite apt. (1) https://sqlite.org/testing.html https://sqlite.org/testing.html
      • bluejekyll 9y ago
        I admit that I was merely being metaphorical and referencing the comprehensive test suite, wide distribution and usage. I wish I had intended that pun embedded in the comment, but from now on ;)
      • weberc2 9y ago
        > Sqllite didn’t start for fun it was built for navy destroyers and submarines. Those aren't mutually exclusive. I for one find destroyers to be an absolute riot.
• petercooper 9y ago
  I've seen a couple of efforts to attempt to port SQLite to Go, usually with the idea of 'mechnically' porting the existing C code to Go rather than starting from scratch. Someone has already ported the SQLite3 shell: https://github.com/cznic/sqlite3shell https://github.com/cznic/sqlite3shell .. you can tell the code is transpiled, however, as it's nothing like what a Go developer would write riddled with gotos and artificial variable names :-) Personally, I think it'd be better to just start afresh with new implementations, not necessarily under the auspices of the SQLite project. Being 'compatible' is reasonable enough and you can just ditch the lesser used or out of date features if you're not claiming to be a 100% clone.
  • mikepurvis 9y ago
    There's a lot of first-party tooling for that, though, since it's how the go compiler itself has been slowly rewritten into go.
  • tmzt 9y ago
    The shell might be the least neccessary to mechanically port as it could be easily implemented in the target language and probably doesn't change that much. The core is where the real compatibility issues lie and where it makes the most sense to translate, as well as where the core vulnerabilities that can be remotely exploited lie.
• amluto 9y ago
  I don’t believe that Go is useful for creating a widely-consumable library.
• m_fayer 9y ago
  I'm an unabashed enthusiast for high-level languages that require complicated runtimes and tooling. I thank the heavens sqlite is written in C and if anything is to ever supplant it, I hope that's then also written in C.
  • peatmoss 9y ago
    What, you don't want to try and figure out how to embed the Erlang VM into your Python project? ;-) EDIT: Jokes aside, Rust sounds like the most plausible alternative, though it still lags C due to being LLVM-only, whereas C has gcc and... others.
• tsavola 9y ago
  I suspect Go will never be an ideal choice for creating libraries which are supposed to be used from any programming language. Even while the C-compatible-library-story of Go is developing (which is great), it brings runtime machinery that just isn't there with languages like C, C++ (especially with exceptions disabled) and Rust.
  • weberc2 9y ago
    I agree. I'm curious though--is it true that Go brings its runtime? I'm of the impression that the runtime is only compiled in if you actually use it, but if your library is just `func Add(a, b int) int { return a + b }`, would linking against it still bring in the runtime? And to reiterate, this is only a curiosity. Even if you could take care to avoid importing the runtime, I wouldn't think that it's worth the while.
• saosebastiao 9y ago
  It's still not there yet, but getting close. And Rust is really the only language that has come close. In terms of performance, it's really the only language out there that can claim to be as fast as C/C++. It has a minimal runtime, with the option of no runtime. It has pretty great FFI and can produce easily callable libraries for other runtime-heavy languages. It's not, however, as broadly compatible as C. I don't think that's a problem for most cases, as most programming is now done for (MIPS|ARM|X86|X86_64), and it can handle those well enough. But microcontrollers and OS-less embedded devices still have a ways to go before Rust beats out C. And stability is just not there. IMO, that's a good thing. Rust is the best thing to happen to systems programming in a really long time, and there are still tons of ideas with amazing potential benefits. The Rust community has been exceptional at guiding this development. I'm sure some day it will level off, but until then, I'm happy with it changing pretty rapidly.
  • pcwalton 9y ago
    I don't think "stability" and "changing rapidly" are incompatible notions. Stability implies that code written today continues to work years from now on later versions of the compiler and language. Rust has always (well, since 1.0) striven to provide stability in that sense, despite the rapid pace of language improvement.
  • cat199 9y ago
    > most programming is now done for (MIPS|ARM|X86|X86_64) but not necessarily posix/windows/etc having C as a least-common-denominator toolchain which almost all platforms will provide is still useful for the embedded world where sqlite has a huge number of applications..
    • tmzt 9y ago
      Rust has a complete #nostd mode where most of it's ergonomics are still usable without any runtime. It also supports pluggable allocators, as well as replacing most of the builtin functionality of other language components (lang items). It's story around panic and debug handling is also improving, including cutting out formatting code for things like println! and debug! when targetting embedded platforms.
• jcranmer 9y ago
  In 2000, the only feasible language really was C. C++ suffered from severe portability issues, which have only arguably gone away in the past few years (that said, you still do see systems with pre-libstdc++-4.9, which is still a headache for portability). Managed languages generally suffer embedability problems, and still do to this day. The conclusion that C remains the best language is a lot harder to support. Certainly, and especially with the level of infrastructure that SQLite developed to harden its implementation, the alternatives are not so much better as to be worth the cost of migration.
  • saagarjha 9y ago
    > you still do see systems with pre-libstdc++-4.9 Such as macOS ;)
• masklinn 9y ago
  > Rust and possibly even Go would be strong contenders to make a new SQLite-like library/program today. Go absolutely would not be, one of the purposes of sqlite is to be embeddable in any and all software.
  • _ph_ 9y ago
    You can create shared libraries in Go, which can be used as any C-implemented shared library. This would make a Go-implemented SQLite equally usable by other programming languages as a C written one.
• bachmeier 9y ago
  I'd actually say D would be best suited as a replacement because of -betterC mode. You could replace one piece, run all the tests, and then move on to the next piece. Most of the C code could even be reused with minor modifications. In fact, this is why -betterC was introduced. The only reason you wouldn't want to do this is that D is limited to the platforms targeted by LLVM.
  • pornel 9y ago
    > Most of the C code could even be reused with minor modifications I've done such conversions with C to Rust converter (https://gitlab.com/citrus-rs/citrus https://gitlab.com/citrus-rs/citrus), but quickly found out that the style of writing idiomatic in C is part of the problem. To replace code function by function you're generally forced to keep the same structs and APIs (often even internal ones) for most of the time, and these require you to erase the extra type safety, degrade smart pointers and slices to plain pointers, etc. If you just do all the same wonky stuff that C does, but only with a slightly different syntax and compiler, you don't gain that much. The value comes from using idioms of a safer language, and that's much more work, and it's especially hard if your hands are tied by the rest of the program being C-like.
    • bachmeier 9y ago
      I agree. The important thing for a port is to be able to do it one small chunk at a time, so that you know you're not changing behavior. Then after the port is complete, you can start adding all the bells and whistles of the new language. There's no benefit to the port if you don't take the last step.
• zombieprocesses 9y ago
  > Especially given SQLite being started in 2000, C really was undeniably the best choice, and the existing code in C is a strong argument in favor against rewrites. That's not much of an argument considering the CLR and the JVM are written in C.
• oscargrouch 9y ago
  Go has a bad fitness to this kind of problem as any language with non-deterministic GC would. Candidates may be: C, C++, Rust and maybe Swift(if the deterministic ref-counting doesnt get in the way). Of course you can do it in other langs, but it wont be able to compete with products made in languages with better fitness to this. Eg. CouchDB vs. MongoDB. The first invented the concept, but the later used a language with better fitness for the kind of problem, and therefore was able to create a better product.