34 ms·
I have not worked with NDB, nor read any research papers or documentation about it. So that's why I didn't have it in mind when I wrote that post. But I'm a li
by abadid 8y ago
I have not worked with NDB, nor read any research papers or documentation about it. So that's why I didn't have it in mind when I wrote that post.
But I'm a little confused by your comment: How is it possible to partition consensus without having more than leader? To me, the definition of "partitioned consensus" is that there is more than one consensus group, which means more than one leader.
Also, FYI, Calvin does not serialize transactions. It processes transactions in parallel. But it guarantees equivalence to a predetermined serial order. That distinction is important. As far as scalability, I discussed that in my previous post. Calvin doesn't have any scalability constraints that can be reached by known real-world workloads.
- jamesblonde 8y agoI should have been more clear - Calvin serializes cross-partition transactions. NDB does not. There is a Transaction Coordinator (TC) on every node. TCs can execute cross-partition transactions in parallel, but programmers need to write "lock-aware" programs (more late). TCs can fail-over if one fails - so, it blocks for just a few seconds (Transaction inactive timeouts are typically just a couple of seconds). There are no leaders in each partition, because every node is a potential TC. There are primary TCs for each partition, but failure does not require a leader election. NDB has "lock-aware" programming - you don't get "global consensus". You decide, as a programmer, that this row could be accessed concurrently by another process, so you lock it, with either a read of write lock. Linearizability is easily implemented by acquiring a lock on a well-known row, but, of course, kills scalability. In our Usenix FAST paper on HopsFS on Spotify's Hadoop workload, we had 1m ops/sec on HDFS, which was about 10m ops/sec on NDB. We ran out of hardware. There are workloads that big. [edited for clarity]
- abadid 8y agoPlease, please read the Calvin paper http://www.cs.umd.edu/~abadi/papers/calvin-sigmod12.pdf http://www.cs.umd.edu/~abadi/papers/calvin-sigmod12.pdf. The assumption that Calvin serializes cross-partition transactions is a common misunderstanding is 100% inaccurate. The paper shows how Calvin gets better parallelism on cross-partition transactions than traditional systems.
- jamesblonde 8y agoOk, sorry about that if it wasn't correct. But are you still not globally ordering every cross-partition transaction - "every scheduler to piece together its own view of a global transaction order by interleaving (in a deterministic, round-robin manner) all sequencers’ batches for that epoch". Even if the sequencers are distributed and execute the transactions in parallel, they need to agree on a total order. This contrast with the lock-aware programming model in NDB, where programmers can allow cross-partition transactions to proceed immediately in parallel if they are sure that they don't conflict.
- abadid 8y agoYes, that's what I refer to as "unified consensus". Batching helps to overcome the scalability challenges (i.e. order batches rather than each individual xact). I talk about this briefly at: http://dbmsmusings.blogspot.com/2018/09/newsql-database-systems-are-failing-to.html http://dbmsmusings.blogspot.com/2018/09/newsql-database-syst...