4 ms·
The problem with a queue on SQLite is that every successful read implies a write. SQLite is fast enough that it may not matter, though.
by zamalek 4y ago
The problem with a queue on SQLite is that every successful read implies a write. SQLite is fast enough that it may not matter, though.
- jacobr1 4y agoWe used sqlite for a queue at a prior startup - enabling the WAL was critical to maintain disk throughput.
- mst 4y agoI once saw WAL turned off to make a sqlite queue perform better. But that was ... gremlins. More than one person tried to figure out wtf was going one and eventually it was better business wise to declare 'gremlins' and everybody involved in the incident has been annoyed about not figuring it out since.
- simplotek 4y ago> The problem with a queue on SQLite is that every successful read implies a write. SQLite also supports in-memory databases with shared cache. https://www.sqlite.org/inmemorydb.html https://www.sqlite.org/inmemorydb.html If a message queue does not require any form of persistence, writes don't sound like an issue.
- avinassh 4y agoBut this is purely in-memory with no disk persistence. That’s quite risky
- michaelcampbell 4y agoNot to be coy, but it only is if it is. For this application I'd agree, but there are plenty of apps that want queues for in-memory use, and if the thing that holds the memory for the queue dies, chances are the whole ecosystem has. Sometimes this is fine.
- nerpderp82 4y agoI just did a quick test, I am seeing over 1k commits per second when being driven from Python 3.11 on an old macbook pro.