3 ms·

Full disclosure: I work for a non-competing (EC vs SC) database - which also has an ugly name "gunDB". Super excited to hear about CockroachDB's progress - and

Full disclosure: I work for a non-competing (EC vs SC) database - which also has an ugly name "gunDB". Super excited to hear about CockroachDB's progress - and their decision to add SQL support I think was smart. However, I can't go without critiquing their marketing of "Scalable" and "Strongly Consistent", especially when they reference horizontal scaling. Strongly Consistent systems bottleneck around a centralized master, or around complex consensus algorithms like PAXOS or Raft. CockroachDB has previously written this off with "we're using Google's Spanner algorithm" but this makes them vulnerable to a Split-Brain failure (which contradicts their other goal of "Survivability"), for anybody interested in this line of research I recommend googling Kyle Kingsbury "Aphyr" "Call Me Maybe" Jepsen Tests. With that said, you can't just toss horizontally "Scalable" and "Strongly Consistent" into the same blog because they are fundamentally at odds with each other - the physics just don't work. To achieve this you only have two options... either (A) vertically scale the Master, or (B) make incredibly technical sharding decisions (where no two pieces of data that depend upon each other can be in separate shards). The first (A) is not the definition of being scalable, because a single machine can only scale so much (limits of the physical system). And (B) cannot be generalized, which would require every company building on top of CockroachDB to either have the expert knowledge of predicting the exact sharding schema in advance (maybe why they are moving to SQL?), or have to depend upon expensive consulting options (perhaps provided by CockroachDB, after a startup has already built itself around using the free open source product?). The third option is that they are over-generalizing their database trade-offs, and that you cannot actually get horizontal "Scalability" and "Strong Consistency". That said, I know the CockroachDB guys are smart, so I'd love to hear the more nuanced/detailed explanation. I'm just trying to keep your guys' marketing and promises in check with the reality.

teraflop 11y ago
CockroachDB's replication protocol is based on Raft, which is not susceptible to split-brain. (Neither is Spanner, but Spanner requires specialized hardware for clock synchronization.) And contrary to your assertion, it allows consistent snapshots and transactions that cross shard boundaries. The rest of your comment ignores a couple decades or so of very extensive research into scalable distributed transactions. Design documentation is here if you care to read it: https://github.com/cockroachdb/cockroach/blob/master/docs/design.md https://github.com/cockroachdb/cockroach/blob/master/docs/de...
- theseoafs 11y ago
  From what I can tell, gunDB completely ignores all distributed systems research up to this point, so that makes sense.
- marknadal 11y ago
  Raft isn't, but that is extremely different from whether an implementation of Raft is or isn't. Kyle Kingsbury's work shows this to be the case quite often - see http://jepsen.io/ http://jepsen.io/ . I have read the design document, and know how Spanner works. This isn't the first time I've asked about this, I had the 2nd highest upvoted comment on the initial CockroachDB announcement ( https://news.ycombinator.com/item?id=9660339 https://news.ycombinator.com/item?id=9660339 ) which means it is many many more people than just me that are concerned about these claims. Unless you can show me exactly what research contradicts my critique, your response has no merit and doesn't address my concerns. You can't handwave magic appeals to authorities without citing them. So my questions still stand.
  - phpnode 11y ago
    If an implementation of raft is susceptible to split brain then it is not raft, it's something else. Please read the literature rather than dismissing it, no one owes you any further explanations and no one is handwaving or appealing to authority.

teraflop 11y ago

CockroachDB's replication protocol is based on Raft, which is not susceptible to split-brain. (Neither is Spanner, but Spanner requires specialized hardware for clock synchronization.) And contrary to your assertion, it allows consistent snapshots and transactions that cross shard boundaries. The rest of your comment ignores a couple decades or so of very extensive research into scalable distributed transactions. Design documentation is here if you care to read it: https://github.com/cockroachdb/cockroach/blob/master/docs/design.md https://github.com/cockroachdb/cockroach/blob/master/docs/de...

theseoafs 11y ago
From what I can tell, gunDB completely ignores all distributed systems research up to this point, so that makes sense.
marknadal 11y ago
Raft isn't, but that is extremely different from whether an implementation of Raft is or isn't. Kyle Kingsbury's work shows this to be the case quite often - see http://jepsen.io/ http://jepsen.io/ . I have read the design document, and know how Spanner works. This isn't the first time I've asked about this, I had the 2nd highest upvoted comment on the initial CockroachDB announcement ( https://news.ycombinator.com/item?id=9660339 https://news.ycombinator.com/item?id=9660339 ) which means it is many many more people than just me that are concerned about these claims. Unless you can show me exactly what research contradicts my critique, your response has no merit and doesn't address my concerns. You can't handwave magic appeals to authorities without citing them. So my questions still stand.
- phpnode 11y ago
  If an implementation of raft is susceptible to split brain then it is not raft, it's something else. Please read the literature rather than dismissing it, no one owes you any further explanations and no one is handwaving or appealing to authority.