13 ms·
I'm surprised by your benchmark results. I've considered building this exact thing before (I think I've talked about it on HN even), but the reason I didn't bu
by Retr0id 8mo ago
I'm surprised by your benchmark results.
I've considered building this exact thing before (I think I've talked about it on HN even), but the reason I didn't build it was because I was sure (on an intuitive level) the actual overhead of the SQL layer was negligible for simple k/v queries.
Where does the +104% on random deletes (for example) actually come from?
- swaminarayan 8mo agoFair skepticism — I had the same intuition going in. The SQL layer overhead alone is probably small, you're right. The bigger gain comes from a cached read cursor. SQLite opens and closes a cursor on every operation. SNKV keeps one persistent cursor per column family sitting open on the B-tree. On random deletes that means seek + delete on an already warm cursor vs. initialize cursor + seek + delete + close on every call. For deletes there's also prepared statement overhead in SQLite — even with prepare/bind/step/reset, that's extra work SNKV just doesn't do. I'd genuinely like someone else to run the numbers. Benchmark source is in the repo if you want to poke at it — tests/test_benchmark.c on the SNKV side and https://github.com/hash-anu/sqllite-benchmark-kv https://github.com/hash-anu/sqllite-benchmark-kv for SQLite. If your results differ I want to know.
- Retr0id 8mo agoWhat does "column family" mean in this context?
- swaminarayan 8mo agoA named key space within the same database file — keys in "users" don't collide with keys in "sessions" but both share the same WAL and transaction.
- bflesch 8mo agoDid you measure the performance impact of having multiple trees in a single file vs. having one tree per file? I'd assume one per file is faster, is that correct?
- swaminarayan 8mo agono dont know about it. I will check it out.
- d1l 8mo agoAre you using ai for the comment replies too?!
- derwiki 8mo agoEveryone knows the emdash is a giveaway, and they are being left in
- altmanaltman 8mo agoare you reading what you're writing?
- d1l 8mo agoIt's a nonstop slop funnel as far as I can tell. Only ashamed I've been here for more than 5 minutes.
- swaminarayan 8mo agoyes
- nightfly 8mo agoIt "only" doubles performance so the overheads aren't that heavy
- snowhale 8mo ago[dead]
- swaminarayan 8mo agoI have used this https://github.com/hash-anu/sqllite-benchmark-kv/blob/main/tests/sqlite-kv-benchmark.c https://github.com/hash-anu/sqllite-benchmark-kv/blob/main/t..., in which I called sqlite3_exec for only setting PRAGMA settings, for main operations I never used it. I have used prepared/bound/step/reset functions only. so the gap isn't statement compilation overhead and the comparison is fair