5 ms·

I'm excited to see how this compares to DuckDB. More local/embedded analytics databases will be good for the industry.

• lainga 3y ago
  Well, if Wikipedia's anything to go by, it seems to be the first in-mem column-oriented DB written in Zig
  • Iridescent_ 3y ago
    As far as I can tell, the linked github repository is written only in C. Zig is used only as build system EDIT: nvm, the github stats seem to be completely wrong? It shows 96% C to me... Found it, I believe this is the result of including code from SQLite in the repo
    • lr1970 3y ago
      Yes, this is because it includes SQlite source (in C) that dwarfs the size of columnar storage extension (in Zig).
    • dgllghr 3y ago
      Including the sqlite source is for the convenience of running unit tests without needing to rely on sqlite as an external dependency :)
      • justinclift 3y ago
        Ahhh. We (sqlitebrowser.org) used to do something along those lines in one of our repos too. We moved to automatically grabbing the SQLite source as a step in our unit tests instead so we always have the latest SQLite release. One less manual task that needs doing every time there's a new SQLite release. It turned out to be really easy to do, as the SQLite download page has hidden tags in it just for this purpose: https://github.com/sqlitebrowser/sqlitebrowser/blob/7a31ef971a5f878383a43bd193c7d0c5d458a77a/.github/workflows/build-appimage.yml#L27-L35 https://github.com/sqlitebrowser/sqlitebrowser/blob/7a31ef97... In theory (!) it should be pretty easy to just copy the above code snippet into your unit testing setup, which would also reduce the line count of C code in your repo. :) In the above code I'd probably use "make -j `nproc`" instead of the hard coded number of cpus (-j2) though, so it automatically uses however many are available. Should probably update that. :)
        • dgllghr 3y ago
          That looks great, thanks for sharing! Given that the zig build system uses zig and zig just got an http client in the standard library, I think it would be feasible to port into stanchion's build. Out of curiosity, do you run unit tests locally using a container that is set up from the CI config you linked (using something like `act`)? Or do you just require sqlite to be installed when you run locally?
          • justinclift 3y ago
            We have a spread of different GitHub Actions based workflows that do stuff whenever a PR is proposed or merged: https://github.com/sqlitebrowser/sqlitebrowser/tree/master/.github/workflows https://github.com/sqlitebrowser/sqlitebrowser/tree/master/.... Most of those are oriented around building packages for various OS's (Linux, macOS, Windows) so people can try the latest code. While there are some tests, they're more like extremely basic sanity checks and they don't use containers. Those tests rely on whichever version of SQLite was downloaded and compiled into the GUI (as per above code snippet). --- That being said, that's for the client side GUI application. There's a server side of things too (https://github.com/sqlitebrowser/dbhub.io https://github.com/sqlitebrowser/dbhub.io -> dbhub.io) that does use containers (docker) for it's automated tests: https://github.com/sqlitebrowser/dbhub.io/tree/master/.github/workflows https://github.com/sqlitebrowser/dbhub.io/tree/master/.githu... Those are integration tests though (eg "make sure we didn't bust communication with our cli", "make sure our go library still works 100% with the server"), and a reasonably decent set of End to End (E2E) tests of the web interface using Cypress. --- Does that help? :)
• BiteCode_dev 3y ago
  DuckDB can read from several heterogenous sources, but is read only. SQLite is limited to its own format but can write. So very different use cases.
  • speedgoose 3y ago
    Actually DuckDB has its own database format for persistence and can support writes. It’s not obvious from their documentation. People told me that on HN not long ago, as I thought the same than you.
    • BiteCode_dev 3y ago
      Oh, thanks, TIL.
    • remram 3y ago
      It can also write Parquet files and other formats.
  • orlp 3y ago
    DuckDB isn't optimized for writing speed, but it is not read-only.
  • jitl 3y ago
    I’m looking at some DuckDB code right here that does `INSERT INTO … ON CONFLICT (id) DO UPDATE SET` so it can definitely do upserts :)
    • BiteCode_dev 3y ago
      Ok, I retract.