4 ms·

There's a whole database, Datomic, that works roughly this way -- not the UDP multicast but the idea of having a slightly more nuanced consistency/availability/

There's a whole database, Datomic, that works roughly this way -- not the UDP multicast but the idea of having a slightly more nuanced consistency/availability/partition tolerance tradeoff by having a thin transaction organizer which is not partition tolerant and officially states which one came first and second, upstream of the replicas that grant normal availability. I would have liked the discussion about Raft/Paxos that they said they'd leave out of this episode though :(

michael_j_ward 5y ago
Sounds similar to xtdb https://github.com/xtdb/xtdb https://github.com/xtdb/xtdb
- tekacs 5y ago
  Yeah as I understand it XTDB was built by Juxt after using Datomic for a long while, to make a different set of trade-off choices. Whenever one comes up, the other is likely to be mentioned close at hand: https://docs.xtdb.com/resources/faq/#comparisons https://docs.xtdb.com/resources/faq/#comparisons
refset 5y ago
CORFU and Tango are interesting research implementations in this general area, with emphasis on a dedicated "sequencer" - see https://rebeccabilbro.github.io/the-shared-log-abstraction/ https://rebeccabilbro.github.io/the-shared-log-abstraction/ Facebook's LogDevice had a similar sequencer component too https://engineering.fb.com/2017/08/31/core-data/logdevice-a-distributed-data-store-for-logs/ https://engineering.fb.com/2017/08/31/core-data/logdevice-a-...
- jasonwatkinspdx 5y ago
  Yeah, and the key idea here is that the sequencer is soft state that can be recovered from the log at any time. It acts to improve throughput while it's up, but if it's down the system can still make progress. The key to this is that the log is the ultimate ground source of truth, and individual entries in the log are write once. This means racing writers can detect and work around each other in the absence of a sequencer, via the hole filling protocol.
- bitbckt 5y ago
  FaunaDB (fauna.com) is a commercial system based on Calvin. Disclosure: I work on FaunaDB.
runevault 5y ago
What's the state of Datomic these days? I stopped keeping up with it a while back when I stopped really paying attention to the Clojure space in general. Last I remember hearing it was still a purely pay to use their instances sort of deal.
- capableweb 5y ago
  > What's the state of Datomic these days? Continuous, minor improvements over time (just like Clojure). Nothing too groundbreaking lately AFAIK, but still worthy to use if you have the use cases for it. Changelog can be found here: https://docs.datomic.com/on-prem/changes.html https://docs.datomic.com/on-prem/changes.html
- fulafel 5y ago
  There's on-prem which is the original model, and then the newer Datomic Cloud (came about in ~2018) which is installed from AWS Marketplace and billed through AWS. There are some feature differences between them. With on-prem there are free and $$ licensing options. Might be that the on-prem gratis licensing options were more limited before. Some of the cool kids in Clojure world seem to have moved over to XTDB these days.

michael_j_ward 5y ago

Sounds similar to xtdb https://github.com/xtdb/xtdb https://github.com/xtdb/xtdb

tekacs 5y ago
Yeah as I understand it XTDB was built by Juxt after using Datomic for a long while, to make a different set of trade-off choices. Whenever one comes up, the other is likely to be mentioned close at hand: https://docs.xtdb.com/resources/faq/#comparisons https://docs.xtdb.com/resources/faq/#comparisons

refset 5y ago

CORFU and Tango are interesting research implementations in this general area, with emphasis on a dedicated "sequencer" - see https://rebeccabilbro.github.io/the-shared-log-abstraction/ https://rebeccabilbro.github.io/the-shared-log-abstraction/ Facebook's LogDevice had a similar sequencer component too https://engineering.fb.com/2017/08/31/core-data/logdevice-a-distributed-data-store-for-logs/ https://engineering.fb.com/2017/08/31/core-data/logdevice-a-...

jasonwatkinspdx 5y ago
Yeah, and the key idea here is that the sequencer is soft state that can be recovered from the log at any time. It acts to improve throughput while it's up, but if it's down the system can still make progress. The key to this is that the log is the ultimate ground source of truth, and individual entries in the log are write once. This means racing writers can detect and work around each other in the absence of a sequencer, via the hole filling protocol.
bitbckt 5y ago
FaunaDB (fauna.com) is a commercial system based on Calvin. Disclosure: I work on FaunaDB.

runevault 5y ago

What's the state of Datomic these days? I stopped keeping up with it a while back when I stopped really paying attention to the Clojure space in general. Last I remember hearing it was still a purely pay to use their instances sort of deal.

capableweb 5y ago
> What's the state of Datomic these days? Continuous, minor improvements over time (just like Clojure). Nothing too groundbreaking lately AFAIK, but still worthy to use if you have the use cases for it. Changelog can be found here: https://docs.datomic.com/on-prem/changes.html https://docs.datomic.com/on-prem/changes.html
fulafel 5y ago
There's on-prem which is the original model, and then the newer Datomic Cloud (came about in ~2018) which is installed from AWS Marketplace and billed through AWS. There are some feature differences between them. With on-prem there are free and $$ licensing options. Might be that the on-prem gratis licensing options were more limited before. Some of the cool kids in Clojure world seem to have moved over to XTDB these days.