2 ms·
I used to work at a company that had a management interface that used sqlite as database, its multi-node / fallover approach was also just... copying the file a
by Cthulhu_ 2y ago
I used to work at a company that had a management interface that used sqlite as database, its multi-node / fallover approach was also just... copying the file and rsyncing it. I did wonder about data integrity though, what if the file is edited while it's being copied over? But there's probably safeguards in place.
Anyway I don't think the database file size was really an issue, it was a relatively big schema but not many indices and performance wasn't a big consideration - hence why the backend would concatenate query results into an XML file, then pass it through an xml->json converter, causing 1-2 second response times on most requests. I worked on a rewrite using Go where requests were more like 10-15 milliseconds.
But, I still used sqlite because that was actually a pretty good solution for the problem at hand; relatively low concurrency (up to 10 active simultaneous users), no server-side dependencies or installation needed, etc.
- MPSimmons 2y ago>I did wonder about data integrity though, what if the file is edited while it's being copied over? But there's probably safeguards in place. You could do a filesystem snapshot and copy from that, but neither a cp or rsync is atomic.
- formerly_proven 2y agosqlite3 has a backup API for this, which you can invoke using the .backup command in the sqlite3 CLI. And then there is also https://www.sqlite.org/rsync.html https://www.sqlite.org/rsync.html
- rollcat 2y agoSQLite has a write-ahead log (WAL). You can use Litestream on top of that. You get single RW, multiple readers (you lose the C in CAP), and can promote a reader when the writer fails.