4 ms·
Good writeup on using LiteStream at this early stage, though I wouldn't say it's production-ready based on a skim, though it might be a good fit for read-heavy
by pcthrowaway 4y ago
Good writeup on using LiteStream at this early stage, though I wouldn't say it's production-ready based on a skim, though it might be a good fit for read-heavy apps which don't have a lot of data.
One thing that's not clear to me is whether Litestream+SQLite works in-memory. If Litestream wrote to disk on the primary node, but kept the full DB in-memory for facilitating reads, I can definitely see how it would be a lot faster than Postgres (as the author claims).
Though I'd also like to know if any RDBMSs have been designed for that use case, as I'd imagine traditional DB optimizations have been made for high incidence of disk access that would no longer hold true if the entire DB was in-memory
- nickcox 4y agoEven reading from disk ought to be way faster than the typical Postgres use case of going over the network, shouldn't it?
- chillfox 4y agoIt's faster even if the PostgreSQL instance is on the same box. If PostgreSQL is on a different box, then it's a lot faster.
- pcthrowaway 4y agoI would expect Postgres to be faster for complex operations over large datasets (of course with a helping of "It depends!"), but very interested in knowing if this is generally untrue.
- mrkurt 4y agoThis is generally untrue, depending on what you mean by "faster". The way most web apps use DBs, sqlite is incredibly fast. Because most web apps make multiple queries, often sequentially, and they're very sensitive to DB latency. The network overhead adds up: https://twitter.com/benbjohnson/status/1514969870529560580 https://twitter.com/benbjohnson/status/1514969870529560580
- mhio 4y agoLiteFS replicates the underlying file system writes by storing the file on a Fuse file system that understands sqlite writes/transactions, it doesn't look like it changes how SQLite runs on top of that file system. The underlying OS page cache is still available, as long as the fuse file system doesn't mess with it [1] so I think this is regular sqlite performance with a fuse overhead to reads/writes. - [0] https://github.com/superfly/litefs/blob/main/docs/ARCHITECTURE.md https://github.com/superfly/litefs/blob/main/docs/ARCHITECTU... - [1] https://github.com/superfly/litefs/issues/149 https://github.com/superfly/litefs/issues/149
- fmajid 4y agoI’d wait until the VFS version (non-FUSE) is ready before giving it a whirl.