4 ms·

If you need multiple writers and can handle eventual correctness, you should really be using cr-sqlite[1]. It'll allow you to have any number of workers/clients

by matlin 3y ago

If you need multiple writers and can handle eventual correctness, you should really be using cr-sqlite[1]. It'll allow you to have any number of workers/clients that can write locally within the same process (so no network overhead) but still guarantee converge to the same state. [1] https://github.com/vlcn-io/cr-sqlite https://github.com/vlcn-io/cr-sqlite

eternityforest 3y ago
I don't see any timestamps in the data. If two peers write to the same row, does it not use latest-wins logic?
- jeromegn 3y ago
  There’s a col_version column in a clock table used for last-write-wins. In case a tie, the “biggest” value wins.
  - eternityforest 3y ago
    Oh nice. Looks like on closer inspection they're using Lamport Clocks, which track causation, but if ignore time, although time is mentioned somewhere as a possibility in hybrid systems someday, if I'm understanding it? Looks like only a 2MB binary for the extension, so you could in theory just pack it with your app too. I'm particularly interested because it seems like(For very small databases) you could use SyncThing as the sync backend by just periodically dumping your data to files(And making a new one once the file got too big). I don't know how you could ever garbage collect the old files aside from some kind of manual "Delete everyone else's stuff and output your own big merged log" command, but it would be really cool to be able to make apps with P2P sync. It also seems like you could put them in an http server and use it like an RSS feed. Or even serve them via torrents.
    - jeromegn 3y ago
      We’re using cr-sqlite as part of our distributed state propagation system. It is indeed easy to bundle in the app! https://github.com/superfly/corrosion https://github.com/superfly/corrosion It would be possible to distribute cr-sqlite changes in many different ways (like you said, http or torrents, etc.) since any change can be applied out of order.

by matlin 3y ago

eternityforest 3y ago

I don't see any timestamps in the data. If two peers write to the same row, does it not use latest-wins logic?

jeromegn 3y ago
There’s a col_version column in a clock table used for last-write-wins. In case a tie, the “biggest” value wins.
- eternityforest 3y ago
  Oh nice. Looks like on closer inspection they're using Lamport Clocks, which track causation, but if ignore time, although time is mentioned somewhere as a possibility in hybrid systems someday, if I'm understanding it? Looks like only a 2MB binary for the extension, so you could in theory just pack it with your app too. I'm particularly interested because it seems like(For very small databases) you could use SyncThing as the sync backend by just periodically dumping your data to files(And making a new one once the file got too big). I don't know how you could ever garbage collect the old files aside from some kind of manual "Delete everyone else's stuff and output your own big merged log" command, but it would be really cool to be able to make apps with P2P sync. It also seems like you could put them in an http server and use it like an RSS feed. Or even serve them via torrents.
  - jeromegn 3y ago
    We’re using cr-sqlite as part of our distributed state propagation system. It is indeed easy to bundle in the app! https://github.com/superfly/corrosion https://github.com/superfly/corrosion It would be possible to distribute cr-sqlite changes in many different ways (like you said, http or torrents, etc.) since any change can be applied out of order.