4 ms·
Sqlite isn't necessarily easier to backup than a filesystem. I've got a fairly large (~60GB) sqlite that I'm somewhat eager to get off of. If I'd stuck with pur
by webstrand 2y ago
Sqlite isn't necessarily easier to backup than a filesystem. I've got a fairly large (~60GB) sqlite that I'm somewhat eager to get off of. If I'd stuck with pure filesystem then backing up only the changeset would be trivial, but with sqlite I have to store a new copy of the database.
I've tried various solutions like litestream and even xdelta3 (which generates patches in excess of the actual changeset size), but I haven't found a solution I'm confident in other than backing up complete snapshots.
- fanf2 2y agoYou might like the new sqlite3_rsync command https://www.sqlite.org/rsync.html https://www.sqlite.org/rsync.html
- simonw 2y agoYeah that looks ideal for this exact problem, because it lets you stream a snapshot backup of a SQLite over SSH without needing to first create a duplicate copy using .backup or vacuum. My notes here: https://til.simonwillison.net/sqlite/compile-sqlite3-rsync https://til.simonwillison.net/sqlite/compile-sqlite3-rsync
- webstrand 2y agoMaybe that tool just doesn't fit my use-case, but I'm not sure how you'd use it to do incremental backups? I store all of my backups in S3 Glacier for the cheap storage, so there's nothing for me to rsync onto. I can see how you'd use it for replication though.
- simonw 2y agoIf you want incremental backups to S3 I recommend Litestream.
- deleted 2y ago[deleted]
- hedgehog 2y agoWhat do you do about compaction?