3 ms·

What happens in your scenario if operation B arrives hours later? It's hard to provide reliable ordering in practice because networks aren't dependable and thin

by thdxr 9y ago

What happens in your scenario if operation B arrives hours later? It's hard to provide reliable ordering in practice because networks aren't dependable and things that happened earlier may show up later. I work on a realtime database and we heavily rely on CRDTs for communication. We can't have a central server because our goals are fault tolerance and high throughout. If you're more curious about the subject look into Riak. Afaik that is the biggest project built with CRDTs

marknadal 9y ago
@thdxr, which realtime database do you work on? I know some people at Riak/Basho who are now at MIT, they are helping us formalize our CRDT ( https://github.com/amark/gun/wiki/Conflict-Resolution-with-Guns https://github.com/amark/gun/wiki/Conflict-Resolution-with-G... ). We were chatting just last night and he said that Riak actually lacked proper CRDT in many settings (for example, secondary indices would produce non-deterministic results depending on order).
burntsushi 9y ago
> What happens in your scenario if operation B arrives hours later? It's hard to provide reliable ordering in practice because networks aren't dependable and things that happened earlier may show up later. I guess I'm trying to figure out why this is a problem. If operation B shows up hours later, then the client simply doesn't know about B until then. But since my stated design requires that dependent operations are delivered in order, we have that any other operations are necessarily concurrent with B, so it shouldn't matter when B is "evaluated." > I work on a realtime database and we heavily rely on CRDTs for communication. We can't have a central server because our goals are fault tolerance and high throughout. > If you're more curious about the subject look into Riak. Afaik that is the biggest project built with CRDTs Yeah, I'm definitely aware of Riak. I guess I'm more interested in the systems layered on top of these components. i.e., What problems are they trying to solve? For instance, what if you could relax your requirements? Maybe: throughput/latency only need to be good enough for web response time to end users.
- im_down_w_otp 9y ago
  For the most part CRDTs of all types solve primarily one problem. The "lost update" problem. When I worked with CRDTs---even inventing a new one based on R*-trees---they were most useful for objects/structures that needed constant mutating by lots and lots of participants concurrently where it made sense to trade constant coordination/protection overhead for lazy-evaluation/merging. Like in a settlement ledger, or massive leaderboard, or indexing structure, etc.
  - burntsushi 9y ago
    Interesting, thanks for sharing that!

by thdxr 9y ago

marknadal 9y ago

@thdxr, which realtime database do you work on? I know some people at Riak/Basho who are now at MIT, they are helping us formalize our CRDT ( https://github.com/amark/gun/wiki/Conflict-Resolution-with-Guns https://github.com/amark/gun/wiki/Conflict-Resolution-with-G... ). We were chatting just last night and he said that Riak actually lacked proper CRDT in many settings (for example, secondary indices would produce non-deterministic results depending on order).

burntsushi 9y ago

> What happens in your scenario if operation B arrives hours later? It's hard to provide reliable ordering in practice because networks aren't dependable and things that happened earlier may show up later. I guess I'm trying to figure out why this is a problem. If operation B shows up hours later, then the client simply doesn't know about B until then. But since my stated design requires that dependent operations are delivered in order, we have that any other operations are necessarily concurrent with B, so it shouldn't matter when B is "evaluated." > I work on a realtime database and we heavily rely on CRDTs for communication. We can't have a central server because our goals are fault tolerance and high throughout. > If you're more curious about the subject look into Riak. Afaik that is the biggest project built with CRDTs Yeah, I'm definitely aware of Riak. I guess I'm more interested in the systems layered on top of these components. i.e., What problems are they trying to solve? For instance, what if you could relax your requirements? Maybe: throughput/latency only need to be good enough for web response time to end users.

im_down_w_otp 9y ago
For the most part CRDTs of all types solve primarily one problem. The "lost update" problem. When I worked with CRDTs---even inventing a new one based on R*-trees---they were most useful for objects/structures that needed constant mutating by lots and lots of participants concurrently where it made sense to trade constant coordination/protection overhead for lazy-evaluation/merging. Like in a settlement ledger, or massive leaderboard, or indexing structure, etc.
- burntsushi 9y ago
  Interesting, thanks for sharing that!