3 ms·
This is awesome. Especially for sqlite db’s that are read only from a website user perspective. My use case would be an sqlite DB that would live on S3 and ge
by indigodaddy 11mo ago
This is awesome. Especially for sqlite db’s that are read only from a website user perspective. My use case would be an sqlite DB that would live on S3 and get updated by cron or some other task runner/automation means (eg some other facility independent of the website that is using the db), and the website would use litestream vfs and just make use of that “read only” (the website will never change or modify the db) db straightup. Can it be used in this described fashion? Also/if so, how will litestream vfs react to the remote db updating itself within this scenario? Will it be cool with that? Also I’m assuming there is or will be Python modules/integration for doing the needful around Litestream VFS?
Currently on this app, I have the Python/flask app just refreshing the sqlite db from a Google spreadsheet as the auth source (via dataframe then convert to sqlite) for the sqlite db on a daily scheduled basis done within the app.
For reference this is the current app: (yes the app is kinda shite but I’m just a sysadmin trying to learn Python!)
https://github.com/jgbrwn/my-upc/blob/main/app.py https://github.com/jgbrwn/my-upc/blob/main/app.py
- benbjohnson 11mo agoAuthor here. Litestream VFS will automatically poll for new back up data every second so it keeps itself up to date with any changes made by the original database. You don't need any additional code (Python or otherwise) to use the VFS. It will work on the SQLite CLI as is.
- indigodaddy 11mo agoOk, yeah I think litestream vfs isn’t suitable to do as I described in the intended scenario.
- Eikon 11mo agoPerhaps you would find ZeroFS [0] useful. It works great out the box with SQLite [1] and only depends on S3 as an external service. [0] https://github.com/Barre/ZeroFS https://github.com/Barre/ZeroFS [1] https://github.com/Barre/ZeroFS?#sqlite-performance https://github.com/Barre/ZeroFS?#sqlite-performance
- indigodaddy 11mo agoYes this approach might be better. Sounds like litestream vfs won’t really do what I wanted in my described scenario
- indigodaddy 11mo agoForgot to say, thanks for posting this, looks quite useful for various projects that have been on my mind. At one point I was looking for a git vfs for Python (I did find one for caddy static serving specifically, but I needed it for Python) but couldn’t find much that wasn’t abandoned—- an s3 vfs might do the trick for a lot of use cases though.
- Eikon 11mo agoMy pleasure, this project has been a lot of fun :).