5 ms·

I can't find any documentation on how, exactly, ACID is achieved across all these partitions. It's impossible to reason about performance characteristics in the

I can't find any documentation on how, exactly, ACID is achieved across all these partitions. It's impossible to reason about performance characteristics in the absence of documentation. For example, if they use two-phase commit, what is the granularity of locks ( table, row, etc )during an open transaction? Table locks during two-phase commit with, say, 50ms coast-to-coast latency would be an immediate dealkiller. Or, if it's eventually consistent as most asynchronously distributed DBs are, they can't claim AC. I really wish Cockroach had better documentation - not so much how to use it, but how it works so as to let engineers reason about performance characteristics.

deleted 9y ago
[deleted]
andreimatei1 9y ago
Besides the blog, you can find interesting stuff in the source repo, in the technotes and RFC dirs: https://github.com/cockroachdb/cockroach/tree/master/docs/tech-notes https://github.com/cockroachdb/cockroach/tree/master/docs/te... https://github.com/cockroachdb/cockroach/tree/master/docs/RFCS https://github.com/cockroachdb/cockroach/tree/master/docs/RF... There's also a bunch of technical tech-talks linked from our company's website: https://www.cockroachlabs.com/community/tech-talks/ https://www.cockroachlabs.com/community/tech-talks/
chubot 9y ago
Not much detail here, but: https://www.cockroachlabs.com/docs/stable/frequently-asked-questions.html#how-is-cockroachdb-both-highly-available-and-strongly-consistent https://www.cockroachlabs.com/docs/stable/frequently-asked-q... Last I heard CockroachDB is like "Spanner without atomic clocks". Not sure if that helps but at least the WAN latency is not an afterthought. https://www.cockroachlabs.com/docs/stable/frequently-asked-questions.html#do-transactions-in-cockroachdb-guarantee-acid-semantics https://www.cockroachlabs.com/docs/stable/frequently-asked-q... https://www.cockroachlabs.com/docs/stable/frequently-asked-questions.html#when-is-cockroachdb-not-a-good-choice https://www.cockroachlabs.com/docs/stable/frequently-asked-q...
knz42 9y ago
(working for cockroach labs) As explained in the blog (as linked by another response) and the documentation (https://www.cockroachlabs.com/docs/stable/strong-consistency.html https://www.cockroachlabs.com/docs/stable/strong-consistency... and https://www.cockroachlabs.com/docs/stable/distributed-transactions.html https://www.cockroachlabs.com/docs/stable/distributed-transa...) CockroachDB does not use locks, and write intents in the concurrency control scheme are expressed by K/V pair i.e. per table cell. Meanwhile, I acknowledge that CockroachDB does not yet convey well its performance model so that engineers can get an intuition about it. There is documentation, but it's either too detailed (like the design doc) or too high-level (most of the docs site). This needs a new sort of documentation, which we are working on.
- daxorid 9y ago
  Thanks!
- eternalban 9y ago
  From the first link: > Employs Raft, a popular successor to Paxos. I don't believe that is correct and that is not a good meme to promote. Successor implies the 'deprecation' of Paxos, which is certainly not the case. For the architectural sweet spot where both protocols can be used, it is fair to say Raft provides the more accessible approach, and that for those who insist on rolling their own consensus mechanism. Paxos and Raft serve distinct architectural concerns in the shared context of providing a distributed state with formally defined consistency guarantees. And to the point, it is entirely reasonable to see a distributed system that utilizes both to address system component specific concerns. CockroachDB, ironically (in context of your OP), in fact could benefit from a vendor lock-in were the candidate cloud provider to provide atomic clocks and GPS cards in their offerings.
  - ddorian43 9y ago
    What are the GPS cards for ?
    - infogulch 9y ago
occultist_throw 9y ago
The trend Ive been seeing is "AP with eventual C". It comoes down to the idea that we have a planet, with datacenters in locations on said planet. Light down fiber or copper only goes so fast. The "eventual consistency" part is simply acknowledging that we can't beat C when our databases are in North America, South America, Europe, Africa, Australia, and Asia. One-way comms takes time. 2 way comms, even more so. The tradeoff is, that changes can come in, and 'eventually' will propagate to the rest of the DB nodes. It does sound not ideal initially, however this type of operation would allow a localDB running as AP to sync when back online. The examples I'm deriving come from recent DBs, like GUN http://gun.js.org/#step1 http://gun.js.org/#step1
- dastbe 9y ago
  cockroachdb and spanner are "CP with high probability of A". There are situations where the db (or possibly a subset, haven't read too deeply into the papers) can become unavailable due to a loss of quorum, but these situations are highly unlikely.

deleted 9y ago

[deleted]

andreimatei1 9y ago

Besides the blog, you can find interesting stuff in the source repo, in the technotes and RFC dirs: https://github.com/cockroachdb/cockroach/tree/master/docs/tech-notes https://github.com/cockroachdb/cockroach/tree/master/docs/te... https://github.com/cockroachdb/cockroach/tree/master/docs/RFCS https://github.com/cockroachdb/cockroach/tree/master/docs/RF... There's also a bunch of technical tech-talks linked from our company's website: https://www.cockroachlabs.com/community/tech-talks/ https://www.cockroachlabs.com/community/tech-talks/

chubot 9y ago

Not much detail here, but: https://www.cockroachlabs.com/docs/stable/frequently-asked-questions.html#how-is-cockroachdb-both-highly-available-and-strongly-consistent https://www.cockroachlabs.com/docs/stable/frequently-asked-q... Last I heard CockroachDB is like "Spanner without atomic clocks". Not sure if that helps but at least the WAN latency is not an afterthought. https://www.cockroachlabs.com/docs/stable/frequently-asked-questions.html#do-transactions-in-cockroachdb-guarantee-acid-semantics https://www.cockroachlabs.com/docs/stable/frequently-asked-q... https://www.cockroachlabs.com/docs/stable/frequently-asked-questions.html#when-is-cockroachdb-not-a-good-choice https://www.cockroachlabs.com/docs/stable/frequently-asked-q...

knz42 9y ago

(working for cockroach labs) As explained in the blog (as linked by another response) and the documentation (https://www.cockroachlabs.com/docs/stable/strong-consistency.html https://www.cockroachlabs.com/docs/stable/strong-consistency... and https://www.cockroachlabs.com/docs/stable/distributed-transactions.html https://www.cockroachlabs.com/docs/stable/distributed-transa...) CockroachDB does not use locks, and write intents in the concurrency control scheme are expressed by K/V pair i.e. per table cell. Meanwhile, I acknowledge that CockroachDB does not yet convey well its performance model so that engineers can get an intuition about it. There is documentation, but it's either too detailed (like the design doc) or too high-level (most of the docs site). This needs a new sort of documentation, which we are working on.

daxorid 9y ago
Thanks!
eternalban 9y ago
From the first link: > Employs Raft, a popular successor to Paxos. I don't believe that is correct and that is not a good meme to promote. Successor implies the 'deprecation' of Paxos, which is certainly not the case. For the architectural sweet spot where both protocols can be used, it is fair to say Raft provides the more accessible approach, and that for those who insist on rolling their own consensus mechanism. Paxos and Raft serve distinct architectural concerns in the shared context of providing a distributed state with formally defined consistency guarantees. And to the point, it is entirely reasonable to see a distributed system that utilizes both to address system component specific concerns. CockroachDB, ironically (in context of your OP), in fact could benefit from a vendor lock-in were the candidate cloud provider to provide atomic clocks and GPS cards in their offerings.
- ddorian43 9y ago
  What are the GPS cards for ?
  - infogulch 9y ago

occultist_throw 9y ago

The trend Ive been seeing is "AP with eventual C". It comoes down to the idea that we have a planet, with datacenters in locations on said planet. Light down fiber or copper only goes so fast. The "eventual consistency" part is simply acknowledging that we can't beat C when our databases are in North America, South America, Europe, Africa, Australia, and Asia. One-way comms takes time. 2 way comms, even more so. The tradeoff is, that changes can come in, and 'eventually' will propagate to the rest of the DB nodes. It does sound not ideal initially, however this type of operation would allow a localDB running as AP to sync when back online. The examples I'm deriving come from recent DBs, like GUN http://gun.js.org/#step1 http://gun.js.org/#step1

dastbe 9y ago
cockroachdb and spanner are "CP with high probability of A". There are situations where the db (or possibly a subset, haven't read too deeply into the papers) can become unavailable due to a loss of quorum, but these situations are highly unlikely.