3 ms·
My first popular web app kept all state in a python dict that was dumped to disk every few minutes. Fastest api you’ve ever seen. When we moved to mongo perform
by QuadmasterXLII 3y ago
My first popular web app kept all state in a python dict that was dumped to disk every few minutes. Fastest api you’ve ever seen. When we moved to mongo performance never recovered. And yet, when I make a website today I do not reach for “pickledb”
- sanderjd 3y agoHonestly, I think more people should consider at least starting with an in-memory-with-snapshotting architecture than a transactional database architecture. Less so for request/response style user interaction, but this is, IMO, less common than it should be for incremental (/batch) processing of large static or replayable streaming data.
- JonChesterfield 3y agosqlite as a replacement for fopen is the middle ground
- klabb3 3y agoI think SQLite is a probably a better approach than manually snapshotting. How often to snapshot? You will get significantly less data loss and architecturally very similar operational simplicity. You can also handle disk sized data sets vs memory size.
- 0cf8612b2e1e 3y agoNow I all wondering, can you use the LiteStream online backup tool against an in memory database? Or does that machinery only work against a materialized database?
- klabb3 3y agoGood Q. I don’t think so? But you could tune SQLite to not wait for disk writes and have a larger memory use, which would make it extremely fast for memory-sized data sets. If you’re not intend on using disk at all, then not too much point having backups?