3 ms·

I'm not impressed: - if you're not passing SQLite's open test suite, you didn't build SQLite - this is a "draw the rest of the owl" scenario; in order to tran

by camgunz 8mo ago

I'm not impressed: - if you're not passing SQLite's open test suite, you didn't build SQLite - this is a "draw the rest of the owl" scenario; in order to transform this into something passing the suite, you'd need an expert in writing databases These projects are misnamed. People didn't build counterstrike, a browser, a C compiler, or SQLite solely with coding agents. You can't use them for that purpose--like, you can't drop this in for maybe any use case of SQLite. They're simulacra (slopulacra?)--their true use is as a prop in a huge grift: tricking people (including, and most especially, the creators) into thinking this will be an economical way to build complex software products in the future.

gf000 8mo ago
Also, the very idea is flawed. These are open-source projects and the code is definitely part of the training data.
- tux3 8mo ago
  That's why our startup created the sendfile(2) MCP server. Instead of spending $10,000 vibe-coding a codebase that can pass the SQLite test suite, the sendfile(2) MCP supercharges your LLM by streamlining the pipeline between the training set and the output you want. Just start the MCP server in the SQLite repo. We have clear SOTA on re-creating existing projects starting from their test suite.
- viraptor 8mo ago
  This would be relevant if you could find matching code between this and sqlite. But then that would invalidate basically any project as "not flawed" really - given GitHub, there's barely any idea which doesn't have multiple partial implementations already.
  - criemen 8mo ago
    Even if was copying sqlite code over, wouldn't the ability to automatically rewrite sqlite in Rust be a valuable asset?
    - scott_w 8mo ago
      Not really because it's not possible for SQLite written in Rust to pass SQLite's checks. See https://www.sqlite.org/whyc.html https://www.sqlite.org/whyc.html
      - jodrellblank 8mo ago
        That doesn't seem to support your claim; guessing you mean: > "2. Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy." 'Safe' languages don't need to do that, if they can verify the array access is always in bounds at compile time then they don't need to emit any code to check it. That aside, it seems like they are saying: for (int i=0; i<10; i++) { foo(array[i]); } in C might become the equivalent of: for (int i=0; i<10; i++) { if (i >= array_lower && i < array_higher) { foo(array[i]); } else { ??? // out of bounds, should never happen } } in a 'safe' language, and i will always be in inside the array bounds so there is no way to test the 'else' branch? But that can't be in SQLite's checks as you claim, because the C code does not have a branch there to test? Either way it seems hard to argue that a bounds test which can never fail makes the code less reliable and less trustworthy than the same code without a bounds test, using the argument that "you can't test the code path where the bounds check which can never fail, fails" - because you can use that same argument "what if the C code for array access which is correct, sometimes doesn't run correctly, you can't test for that"?
        scott_w 8mo ago
        Correct, that's what I mean. I trust SQLite's devs to know more about this, so I trust what they wrote. There are parts of Rust code that are basically: do_thing().expect(...); This branch is required by the code, even if it can't be reached, because the type system requires it. It's not possible to test this branch, therefore 100% coverage is impossible in those cases.
9dev 8mo ago
Well--given a full copy of the SQLite test suite, I'm pretty sure it'd get there eventually. I agree that most of these show-off projects are just prop pieces, but that's kind of the point: Demonstrate it's technically possible to do the thing, not actually doing the thing, because that'd have diminishing returns for the demonstration. Still, the idea of setting a swarm of agents to a task, and, given a suitable test suite, have them build a compliant implementation, is sound in itself.
- layer8 8mo ago
  If you minimax for passing the SQLite test suite, I’m still not sure you’ll have a viable implementation. You can’t prove soundness of code through a test suite alone.
- groundzeros2015 8mo ago
  > I'm pretty sure it'd get there eventually. Why? The combinatorics of “just try things until you get it right” makes this impractical.
- kevinsync 8mo ago
  Sure, but that presumes that you have that test suite written without having a single line of application code written (which, to me, is counterintuitive, unrealistic, and completely insane) SQLite apparently has 2 million tests! If you started only with that and set your agentic swarm against it, and the stars aligned and you ended up with a pristine, clean-room replica that passes everything, other than proof that it could be done, what did you achieve? You stood on the shoulders of giants to build a Bizarro World giant that gets you exactly back to where you began? I'd be more interested in forking SQLite as-is, setting a swarm of agents against it with the looping task to create novel things on top of what already exists, and see what comes out. [0] https://en.wikipedia.org/wiki/SQLite#Development_and_distribution https://en.wikipedia.org/wiki/SQLite#Development_and_distrib...
  - mcculley 8mo ago
    You think an implementation of SQLite in another language, with more memory safety, has no value? I agree that this current implementation is not very useful. I would not trust it where I trust SQLite. Regardless, the potential for having agents build clean room implementations of existing systems from existing tests has value.
stavros 8mo ago
I'm generally not this pedantic, but yeah, "I wrote an embedded database" is fine to say. If you say "I built SQLite", I expected to at least see how many of the SQLite tests your thing passed.
wseqyrku 8mo ago
> tricking people (including, and most especially, the creators), I believe it's an ad. Everything about it is trying so hard to seem legit and it's the most pointless thing I have ever seen.
kyars 8mo ago
sorry for misleading, added an update stating that this is a simulacra of sqlite

by camgunz 8mo ago

gf000 8mo ago

Also, the very idea is flawed. These are open-source projects and the code is definitely part of the training data.

tux3 8mo ago
That's why our startup created the sendfile(2) MCP server. Instead of spending $10,000 vibe-coding a codebase that can pass the SQLite test suite, the sendfile(2) MCP supercharges your LLM by streamlining the pipeline between the training set and the output you want. Just start the MCP server in the SQLite repo. We have clear SOTA on re-creating existing projects starting from their test suite.
viraptor 8mo ago
This would be relevant if you could find matching code between this and sqlite. But then that would invalidate basically any project as "not flawed" really - given GitHub, there's barely any idea which doesn't have multiple partial implementations already.
- criemen 8mo ago
  Even if was copying sqlite code over, wouldn't the ability to automatically rewrite sqlite in Rust be a valuable asset?
  - scott_w 8mo ago
    Not really because it's not possible for SQLite written in Rust to pass SQLite's checks. See https://www.sqlite.org/whyc.html https://www.sqlite.org/whyc.html
    - jodrellblank 8mo ago
      That doesn't seem to support your claim; guessing you mean: > "2. Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy." 'Safe' languages don't need to do that, if they can verify the array access is always in bounds at compile time then they don't need to emit any code to check it. That aside, it seems like they are saying: for (int i=0; i<10; i++) { foo(array[i]); } in C might become the equivalent of: for (int i=0; i<10; i++) { if (i >= array_lower && i < array_higher) { foo(array[i]); } else { ??? // out of bounds, should never happen } } in a 'safe' language, and i will always be in inside the array bounds so there is no way to test the 'else' branch? But that can't be in SQLite's checks as you claim, because the C code does not have a branch there to test? Either way it seems hard to argue that a bounds test which can never fail makes the code less reliable and less trustworthy than the same code without a bounds test, using the argument that "you can't test the code path where the bounds check which can never fail, fails" - because you can use that same argument "what if the C code for array access which is correct, sometimes doesn't run correctly, you can't test for that"?
      - scott_w 8mo ago
        Correct, that's what I mean. I trust SQLite's devs to know more about this, so I trust what they wrote. There are parts of Rust code that are basically: do_thing().expect(...); This branch is required by the code, even if it can't be reached, because the type system requires it. It's not possible to test this branch, therefore 100% coverage is impossible in those cases.

9dev 8mo ago

Well--given a full copy of the SQLite test suite, I'm pretty sure it'd get there eventually. I agree that most of these show-off projects are just prop pieces, but that's kind of the point: Demonstrate it's technically possible to do the thing, not actually doing the thing, because that'd have diminishing returns for the demonstration. Still, the idea of setting a swarm of agents to a task, and, given a suitable test suite, have them build a compliant implementation, is sound in itself.

layer8 8mo ago
If you minimax for passing the SQLite test suite, I’m still not sure you’ll have a viable implementation. You can’t prove soundness of code through a test suite alone.
groundzeros2015 8mo ago
> I'm pretty sure it'd get there eventually. Why? The combinatorics of “just try things until you get it right” makes this impractical.
kevinsync 8mo ago
Sure, but that presumes that you have that test suite written without having a single line of application code written (which, to me, is counterintuitive, unrealistic, and completely insane) SQLite apparently has 2 million tests! If you started only with that and set your agentic swarm against it, and the stars aligned and you ended up with a pristine, clean-room replica that passes everything, other than proof that it could be done, what did you achieve? You stood on the shoulders of giants to build a Bizarro World giant that gets you exactly back to where you began? I'd be more interested in forking SQLite as-is, setting a swarm of agents against it with the looping task to create novel things on top of what already exists, and see what comes out. [0] https://en.wikipedia.org/wiki/SQLite#Development_and_distribution https://en.wikipedia.org/wiki/SQLite#Development_and_distrib...
- mcculley 8mo ago
  You think an implementation of SQLite in another language, with more memory safety, has no value? I agree that this current implementation is not very useful. I would not trust it where I trust SQLite. Regardless, the potential for having agents build clean room implementations of existing systems from existing tests has value.

stavros 8mo ago

I'm generally not this pedantic, but yeah, "I wrote an embedded database" is fine to say. If you say "I built SQLite", I expected to at least see how many of the SQLite tests your thing passed.

wseqyrku 8mo ago

> tricking people (including, and most especially, the creators), I believe it's an ad. Everything about it is trying so hard to seem legit and it's the most pointless thing I have ever seen.

kyars 8mo ago

sorry for misleading, added an update stating that this is a simulacra of sqlite