3 ms·
90% of the article is a satisfying analysis of problems two-phase commit. The remaining 10% of the article gives me no confidence that some alternative system i
by klodolph 8y ago
90% of the article is a satisfying analysis of problems two-phase commit. The remaining 10% of the article gives me no confidence that some alternative system is around the corner. Two-phase commit has the nice property that we know it's "correct", and there is a long history of alternative proposals that either aren't based on reasonable assumptions (e.g. we can make the network ordered), don't provide properties that make it easy to build systems on top of them (e.g. don't provide consistency), or aren't correct.
So I'm not holding my breath until someone writes a paper about it. And even then, I would like to see someone build a system with it.
- freels 8y agoFaunaDB already does this. It provides distributed transactions based on Calvin, which does not rely on classic 2PC. See https://fauna.com/blog/acid-transactions-in-a-globally-distributed-database https://fauna.com/blog/acid-transactions-in-a-globally-distr... and http://cs.yale.edu/homes/thomson/publications/calvin-sigmod12.pdf http://cs.yale.edu/homes/thomson/publications/calvin-sigmod1...
- klodolph 8y agoFrom my reading of the article, it seems that this may fall under the "don't provide properties that make it easy to build systems on top of them" category. I wasn't even aware that this article was pitching Calvin as a good alternative, it seemed that the article presents Calvin as preliminary research rather than a proof that this is a good model for databases we can build systems on. You can't do dependent transactions in Calvin. Without experience building systems without dependent transactions, I'm worried that we may underestimate how much of an impact removing dependent transactions has on our ability to design working systems on time and under budget. This echoes earlier problems we had when everyone was building systems that didn't rely on database consistency... we often underestimated how difficult it was to build a working system without database consistency.
- abadid 8y agoTo clarify a couple of things: (1) Calvin does support dependent transactions (it uses OLLP to support them. (2) Calvin is just one example of a system that disallows arbitrary aborts. We've built a bunch of them in my lab. (3) I do not believe that disallowing arbitrary aborts results in fundamentally new limitations of the system. You can still use pessimistic or optimistic concurrency control. You can use deterministic or nondeterministic systems. You don't have to assume you know the read-write set in advance. And you can certainly support dependent transactions.
- klodolph 8y agoThanks for clarifying, this was definitely not clear from the article. Is this article part of a series? It's still not clear to me why OLLP works. When I think of optimistic concurrency my next concern is whether the system is guaranteed to make progress.
- abadid 8y agoI wasn't intending for it to be part of a series, but I agree that the OLLP technique may be of interest to a general audience and therefore a good subject for a future post.
- benesch 8y agoYou can always execute dependent transactions in a system that does not support them by running two non-dependent transactions. This is covered in section 3.2.1 of the Calvin paper [0] as the Optimistic Lock Location Prediction (OLLP) scheme. Basically, when you have a dependent transaction (i.e., a transaction where the write set depends on the results of earlier reads in the transaction), you split it into two separate transactions. The first transaction, the reconnaissance transaction, reads all the data necessary to determine the transaction's write set. Then the second transaction can declare the full read/write set based on the results of the recon transaction. While executing the second transaction, you verify that the data you're reading is the same as the data you read in the recon transaction; if it's not, you need to start the whole process over. This is certainly unfortunate, because you have to perform every read twice. But perhaps it's performant enough. Have you tried using FaunaDB/Calvin for your workload? It sounds like FaunaDB has support for dependent transactions using OLLP baked in, but I'm curious to know if there's a significant performant hit to using it. [0]: http://cs.yale.edu/homes/thomson/publications/calvin-sigmod12.pdf http://cs.yale.edu/homes/thomson/publications/calvin-sigmod1...
- jupiter90000 8y agoSo, if you really do want more information about the alternative proposed, look at the Calvin system mentioned in the article -- which also links to the paper for Calvin describing how it approaches the problem.
- toolslive 8y agoAn sketch of a solution in a blog post about streamy_db: https://domsj.info/2018/12/30/introducing-streamy-db.html https://domsj.info/2018/12/30/introducing-streamy-db.html