3 ms·
Still halfway through reading, but what you've made can unlock a lot of use cases. > I tried SQLite first, but its extension API is limited and write performan
by aljgz 7mo ago
Still halfway through reading, but what you've made can unlock a lot of use cases.
> I tried SQLite first, but its extension API is limited and write performance with custom storage was painfully slow
For many use cases, write performance does not matter much. Other than the initial import, in many cases we don't change text that fast. But the simpler logistics of having a sqlite database, with the dual (git+SQL) access to text is huge.
That said, for the specific use case I have in mind, postgres is perfectly fine
- ImGajeed76 7mo agosounds great yes. maybe an SQLite version will come in the future
- nasretdinov 7mo agoAlso SQLite in WAL/WAL2 mode is definitely not amy slower for writing than Postgres either.
- hrmtst93837 7mo ago[flagged]
- brigandish 7mo agoFor a distributed VCS, what would be the need for such things? Even if it were a really big project, how many writes could be going on that this becomes a bottleneck? I don't see it but maybe you have a situation in mind.
- ImGajeed76 7mo agoThe problem i faced is mostly importing large repos. But normal use should be fine.
- lelanthran 7mo agoIn the current environment, even a distributed VCS may have concurrent agents modifying it on different branches.
- ps12 7mo ago[dead]
- babarot 7mo agoThe single-file simplicity of SQLite is a huge win for self-hosted apps. I've been using SQLite in WAL mode for a single-user app and it handles concurrent reads from the API while background workers write without issues. Backup is just cp. For anything that doesn't need multi-user concurrent writes, it's hard to justify the operational overhead of Postgres. ko
- ImGajeed76 7mo agoYeah, I get that, and I'm fully on your side. SQLite would have been a nice fit. The only downside is the delta compression problem. Creating an extension for SQLite works, but it's slow. I had two options: 1) Do the delta compression and caching and so on on the pgit side and lose SQL queryability (or I need to do my own), or 2) Use postgres
- swaminarayan 7mo agoif you want to use key value store using sqlite then you can try : https://github.com/hash-anu/snkv https://github.com/hash-anu/snkv in which i am directly accessing b-tree layer and not consuming query layer. for kv workloads it is much faster compare to sql. and yeah you will get same benefits of sqlite storage engine.