7 ms·

> When partitions heal, you might have to make ugly decisions: which version of your customer’s data to you choose to discard? If two partitions received update

> When partitions heal, you might have to make ugly decisions: which version of your customer’s data to you choose to discard? If two partitions received updates, it’s a lose-lose situation. When partitions heal you simply merge all versions through conflict-free replicated data types. No ugly decisions, no sacrificing neither latency nor consistency. We call it strong eventual consistency [1] nowadays. And it's exactly like CDNs, except more reliable. I'm wondering, since CockroachDB keeps lying about and attacking eventual consistency in these PR posts, the whole "consistency is worth sacrificing latency" mantra might not work in practice after all. People just don't buy it, they want low latency, they want something like CDNs, something fast and reliable, something that just works. Something that CockroachDB can never deliver. [1] https://en.wikipedia.org/wiki/Eventual_consistency#Strong_eventual_consistency https://en.wikipedia.org/wiki/Eventual_consistency#Strong_ev...

imtringued 9y ago
My application is just plain old CRUD. What if two users want to change e.g. the telephone number of an existing record during a network partition. There just is no obvious way to merge a telephone number. One of them is correct, the other is incorrect. Can CRDTs solve my simple problem?
- merb 9y ago
  I think the issue is solved (while still complex) with an event sourcing system. The chances are really high that the users did not push / updated the number at the exact same time. (so they are at least 1ns apart, even when not it's not that bad). So you can actually restore to a sane state by reapplying the log from scratch. i.e. this is solved with CQRS and Event Sourcing and it probably works in like all databases. It's quite complex but pretty reliable, I'm pretty sure that everybody already built at least a extremly simple append only event log.
  - elvinyung 9y ago
    Does this imply that there's a single place that log entries get ingested at? Doesn't that make it a single point of failure?
    - merb 9y ago
      well you can either use a single database for log entries, or cassandra or a mysql cluster or a pg cluster. that doesn't matter.. basially instead of having a table where you update/delete entries you basically append only to a table. so insert only. you can still have another table that will be aggregated from your log table. where you store (i.e. which database system, cluster whatever) your stuff doesn't matter.
      - elvinyung 9y ago
        That doesn't answer my question. If you're appending to the end of a table, regardless of system you still need one machine somewhere to ingest new records for the same primary key, right?
    - cube2222 9y ago
      No, you can just have partitioned replicated topics like kafka has, just make sure that all the events for one user get put into the same partition, so that they get totally ordered.
      - elvinyung 9y ago
        So I guess the specific issue that grandparent raised was if multiple users wanted to edit the same record. So even if the queue is sharded on some primary key, then for each partition, either a) you only have one machine ingesting and thus a SPOF, or b) you ingest data using multiple machines, which means you need consensus/conflict resolution somewhere, right?
        Joeri 9y ago
        You ingest the log partition in all machines to reconstruct the state from the log. You do this even on the machine that writes to the log, treating the log as the source of truth. Since every user would belong to one partition there would be a globally agreed ordering of their events in the log. Basically though that makes a CP tradeoff, since a network partition that hides the primary kafka replica from some of the writers causes writes to fail. There’s simply no way to have a globally distributed system that’s always available and always consistent, either you get inconsistency or you drop writes.
  - lukev 9y ago
    1ns according to whom? Clock skew is still a thing :)
- xj9 9y ago
  sort of all CRDT will give you a deterministic outcome and provide enough information to allow a user to decide if this outcome is acceptable
- CMCDragonkai 9y ago
  Last write wins in that case?
- al2o3cr 9y ago
  I've only tinkered with CRDTs, but one common theme in those applications is changing the data structure to accommodate the CRDT. In your example, that might mean changing the single "phone number" field to many "phone numbers", so that merging the two writes results in a customer record with two phone numbers. This preserves the data, but pushes conflict resolution (which number should be used?) out into the consumers.
riku_iki 9y ago
> they want something like CDNs, something fast and reliable, something that just works. There are other DB systems with such properties "they" can use, e.g. Cassandra. But CockroachDB also gives ACID transactions, which may be important for others.

imtringued 9y ago

My application is just plain old CRUD. What if two users want to change e.g. the telephone number of an existing record during a network partition. There just is no obvious way to merge a telephone number. One of them is correct, the other is incorrect. Can CRDTs solve my simple problem?

merb 9y ago
I think the issue is solved (while still complex) with an event sourcing system. The chances are really high that the users did not push / updated the number at the exact same time. (so they are at least 1ns apart, even when not it's not that bad). So you can actually restore to a sane state by reapplying the log from scratch. i.e. this is solved with CQRS and Event Sourcing and it probably works in like all databases. It's quite complex but pretty reliable, I'm pretty sure that everybody already built at least a extremly simple append only event log.
- elvinyung 9y ago
  Does this imply that there's a single place that log entries get ingested at? Doesn't that make it a single point of failure?
  - merb 9y ago
    well you can either use a single database for log entries, or cassandra or a mysql cluster or a pg cluster. that doesn't matter.. basially instead of having a table where you update/delete entries you basically append only to a table. so insert only. you can still have another table that will be aggregated from your log table. where you store (i.e. which database system, cluster whatever) your stuff doesn't matter.
    - elvinyung 9y ago
      That doesn't answer my question. If you're appending to the end of a table, regardless of system you still need one machine somewhere to ingest new records for the same primary key, right?
  - cube2222 9y ago
    No, you can just have partitioned replicated topics like kafka has, just make sure that all the events for one user get put into the same partition, so that they get totally ordered.
    - elvinyung 9y ago
      So I guess the specific issue that grandparent raised was if multiple users wanted to edit the same record. So even if the queue is sharded on some primary key, then for each partition, either a) you only have one machine ingesting and thus a SPOF, or b) you ingest data using multiple machines, which means you need consensus/conflict resolution somewhere, right?
      - Joeri 9y ago
        You ingest the log partition in all machines to reconstruct the state from the log. You do this even on the machine that writes to the log, treating the log as the source of truth. Since every user would belong to one partition there would be a globally agreed ordering of their events in the log. Basically though that makes a CP tradeoff, since a network partition that hides the primary kafka replica from some of the writers causes writes to fail. There’s simply no way to have a globally distributed system that’s always available and always consistent, either you get inconsistency or you drop writes.
- lukev 9y ago
  1ns according to whom? Clock skew is still a thing :)
xj9 9y ago
sort of all CRDT will give you a deterministic outcome and provide enough information to allow a user to decide if this outcome is acceptable
CMCDragonkai 9y ago
Last write wins in that case?
al2o3cr 9y ago
I've only tinkered with CRDTs, but one common theme in those applications is changing the data structure to accommodate the CRDT. In your example, that might mean changing the single "phone number" field to many "phone numbers", so that merging the two writes results in a customer record with two phone numbers. This preserves the data, but pushes conflict resolution (which number should be used?) out into the consumers.

riku_iki 9y ago

> they want something like CDNs, something fast and reliable, something that just works. There are other DB systems with such properties "they" can use, e.g. Cassandra. But CockroachDB also gives ACID transactions, which may be important for others.