3 ms·

Be interested to hear a comparison between this lib and litestream/litefs, which seem to be actively developed by fly.io for a similar use case https://litestr

by sho 3y ago

Be interested to hear a comparison between this lib and litestream/litefs, which seem to be actively developed by fly.io for a similar use case https://litestream.io/ https://litestream.io/ https://github.com/superfly/litefs https://github.com/superfly/litefs

AlexErrant 3y ago
cr-sqlite [0] is another option. It has the added benefit of being able to run in the browser (persisting to IndexedDB a la absurd-sql, though it uses a fork of wa-sqlite [1]). In theory it could also persist to the Origin Private Filesystem via SQLite's official impl, though switching doesn't seem to be strictly better [2]. A con is that it doesn't enforce foreign key constraints (though of course you can still do joins). Cr-sqlite also has better documentation about what CRDTs it supports than Mycelite [3], which is something I brought up in a recent thread [4](which might have triggered this post). I'm personally using cr-sqlite in an Anki clone I'm building [5] and have only positive things to say about cr-sqlite and its maintainer. [0] https://github.com/vlcn-io/cr-sqlite/ https://github.com/vlcn-io/cr-sqlite/ [1] https://github.com/rhashimoto/wa-sqlite/ https://github.com/rhashimoto/wa-sqlite/ [2] https://github.com/rhashimoto/wa-sqlite/discussions/63 https://github.com/rhashimoto/wa-sqlite/discussions/63 [3] https://github.com/vlcn-io/cr-sqlite/#how-does-it-work https://github.com/vlcn-io/cr-sqlite/#how-does-it-work [4] https://news.ycombinator.com/item?id=36473772 https://news.ycombinator.com/item?id=36473772 [5] https://github.com/AlexErrant/Pentive https://github.com/AlexErrant/Pentive
- sho 3y ago
  Thank you for the link, very interesting. A lot of work going on in this space right now!
tedunangst 3y ago
Litestream lets you pick which servers you replicate your data to.
benbjohnson 3y ago
Litestream/LiteFS author here. It's been a while since I looked at Mycelial but IIRC it does eventually consistent replication using CRDTs. That's great for systems that need multi-master but there are consistency tradeoffs that you have to watch out for. Litestream & LiteFS are both physical, async replication tools so they actually work at the page level. When a transaction commits, the changed pages are shipped off to a replication destination. By replaying those changes in the same order, you can rebuild the current state of the database (or any point-in-time state in between). Both of these systems require a single primary node at a time so multi-master doesn't work with them. Litestream only replicates to "dumb storage" (e.g. S3, SFTP) whereas LiteFS replicates to other live nodes so you can have live replicas.

by sho 3y ago

AlexErrant 3y ago

cr-sqlite [0] is another option. It has the added benefit of being able to run in the browser (persisting to IndexedDB a la absurd-sql, though it uses a fork of wa-sqlite [1]). In theory it could also persist to the Origin Private Filesystem via SQLite's official impl, though switching doesn't seem to be strictly better [2]. A con is that it doesn't enforce foreign key constraints (though of course you can still do joins). Cr-sqlite also has better documentation about what CRDTs it supports than Mycelite [3], which is something I brought up in a recent thread [4](which might have triggered this post). I'm personally using cr-sqlite in an Anki clone I'm building [5] and have only positive things to say about cr-sqlite and its maintainer. [0] https://github.com/vlcn-io/cr-sqlite/ https://github.com/vlcn-io/cr-sqlite/ [1] https://github.com/rhashimoto/wa-sqlite/ https://github.com/rhashimoto/wa-sqlite/ [2] https://github.com/rhashimoto/wa-sqlite/discussions/63 https://github.com/rhashimoto/wa-sqlite/discussions/63 [3] https://github.com/vlcn-io/cr-sqlite/#how-does-it-work https://github.com/vlcn-io/cr-sqlite/#how-does-it-work [4] https://news.ycombinator.com/item?id=36473772 https://news.ycombinator.com/item?id=36473772 [5] https://github.com/AlexErrant/Pentive https://github.com/AlexErrant/Pentive

sho 3y ago
Thank you for the link, very interesting. A lot of work going on in this space right now!

tedunangst 3y ago

Litestream lets you pick which servers you replicate your data to.

benbjohnson 3y ago

Litestream/LiteFS author here. It's been a while since I looked at Mycelial but IIRC it does eventually consistent replication using CRDTs. That's great for systems that need multi-master but there are consistency tradeoffs that you have to watch out for. Litestream & LiteFS are both physical, async replication tools so they actually work at the page level. When a transaction commits, the changed pages are shipped off to a replication destination. By replaying those changes in the same order, you can rebuild the current state of the database (or any point-in-time state in between). Both of these systems require a single primary node at a time so multi-master doesn't work with them. Litestream only replicates to "dumb storage" (e.g. S3, SFTP) whereas LiteFS replicates to other live nodes so you can have live replicas.