4 ms·
Litestream/LiteFS author here. Litestream still exists and it works as a disaster recovery system. That hasn't changed. LiteFS (aka bizarro FUSE filesystem) is
by benbjohnson 4y ago
Litestream/LiteFS author here. Litestream still exists and it works as a disaster recovery system. That hasn't changed.
LiteFS (aka bizarro FUSE filesystem) is for distributing databases geographically. It's admittedly still early stage (for a database) but we're continuing to focus on stability & correctness in order to make it less "bizarro". We're also going to release it as a regular SQLite VFS if that helps ease any fears.
The original Litestream code had several limitations in terms of ensuring correctness in a distributed system because it had little control over the interaction with SQLite. To distribute in a consistent way, it really needs to act as a shim in the file system layer which is how we landed on the FUSE/VFS design for LiteFS.
One last point I'll add is that all databases have a similar adoption cycle and LiteFS is no exception. They start out being used for non-critical infrastructure and gradually move into more systems as the community gains confidence. For example, we use LiteFS internally at Fly.io as an easy way to distribute some status information. It's been working well overall and it gives us a real-world test case. It's also data that can be rebuilt if we hit an issue with LiteFS. As we gain confidence, we'll add it to more and more systems.
- infamia 4y agoBen, thanks for your awesome work on Litestream/LiteFS! I've just started benchmarking Litestream with a mind to replace Postgres with Litestream on a new project. Any update on when WAL mode support might be available in LiteFS?
- benbjohnson 4y agoThanks! WAL support is in the `main` branch and we're just doing some testing and bug squashing right now. I'm hoping we can release v0.3.0 next week with WAL support.
- fmajid 4y agoSorry for misspelling the name of your project. Yes, VFS rather than FUSE would make a big difference in confidence, as would having it be independently tested by Jepsen.io.
- benbjohnson 4y agoNo problem. Jepsen is something we're interested in once the project reaches a certain level of maturity. The replication is pretty basic primary/replica async streaming replication so I'm not sure there are too many surprises on that side of it. One unique thing it has is that it does handle divergent primaries since it uses leasing rather than distributed consensus.