4 ms·
Things worked well at the outset, especially in local development against my NVMe drive for my small CRUD application. Then, with a little traffic, things cont
by settrans 4y ago
Things worked well at the outset, especially in local development against my NVMe drive for my small CRUD application.
Then, with a little traffic, things continued to go well in production. But as traffic scaled up (to 1-5 QPS, roughly 25% writes), they fell apart. Hard. Because my production environment was spinning rust, IO contention was a real issue and totally absent from development. This manifested as frequent database timeouts, both from reads and writes.
Echoing another commenter's sentiment: things would have gone much more smoothly from the beginning had I started with PostgreSQL, but after having written many thousands of lines of direct SQL taking intimate advantage of SQLite's surprisingly rich featureset, migrating was less than totally appealing.
The mitigation strategy, which ultimately worked out, was to implement backpressure for writes to SQLite: queuing and serializing all writes to each database in the application, failing loudly and conspicuously in the case of errors (thus forcing the client to retry), and gracefully handling the rare deadlock by crashing the process completely with a watchdog timer.
- justsomeuser 4y agoSo you were using an HDD, not an SSD? Would an SSD in production have solved the timeouts by increasing your write throughput?
- settrans 4y agoAccording to my testing, SSDs would greatly reduce the contention but were not a complete fix.