3 ms·
This is a good troll, and makes some good points, as Andy Pavlo always does. I'd diagnose the core problem differently, however. What application and system de
by mjb 3y ago
This is a good troll, and makes some good points, as Andy Pavlo always does.
I'd diagnose the core problem differently, however. What application and system developers care about in modern architectures is application level correctness from the perspective of the client (customer, tester, etc). What a lot of the research is interested in is isolation within the context of a single database. That's a useful building block, but doesn't compose into end-to-end correctness in ways that most application architects and developers can reason about as soon as the application spans multiple stores (because SoA, or because microservices, or because it interacts with a SaaS, or whatever).
So application builders are left with the problem of not being able to reason about how to build correct applications, and not having the tools to build those applications correctly, and so end up settling for some level of weird behavior under concurrency. Turning up the database isolation level may hurt performance, but does little to rein in the overall incorrectness, so there is little motivation to do it.
If the database research community wants application developers and operators to care about isolation more, I suspect that we need to care more about atomicity, isolation, and consistency "in the large". Think more broadly, across whole microservice or SoA architectures, and provide ways to give customers ACID properties end-to-end, and ways to compose databases, services, and systems that preserve these properties.
That doesn't mean that per-database or per-component isolation isn't important. It's practically important, and there's good research on the effect it has[1]. The problem is that its not sufficient, and so folks feel less motivated to pay the (perceived) performance costs.
For folks who'd like to understand isolation levels better, I'd recommend starting with "Seeing is Believing" (https://www.cs.cornell.edu/lorenzo/papers/Crooks17Seeing.pdf https://www.cs.cornell.edu/lorenzo/papers/Crooks17Seeing.pdf) and "A critique of ANSI SQL isolation levels" (https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-95-51.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...) both of which do a good job explaining the differences between levels in a way that isn't tied to implementation.
[1] One good paper there is "ACIDRain: Concurrency-Related Attacks on Database-Backed Web Applications" (http://www.bailis.org/papers/acidrain-sigmod2017.pdf http://www.bailis.org/papers/acidrain-sigmod2017.pdf).
- pdhborges 3y agoThere is a good article [1] discussing this state of affairs where application developers are straight jacked because they end up integrating too many systems and there is no distributed transaction covering them all. [1] https://queue.acm.org/detail.cfm?id=3025012 https://queue.acm.org/detail.cfm?id=3025012
- crabbone 3y ago> What application and system developers care about in modern architectures is application level correctness Meme template: You guys care about correctness? Also, please don't lump together application and system developers. In terms of software quality these are worlds apart. It's a lot more common in system world to care about correctness than it is in the application world. Of course, there are always exceptions, but a typical example of either camp goes about quality control very differently. In my experience, outside of a very small sector of application development, where correctness helps sales (eg. medicine, or other fields where software mistakes come with serious repercussions) the goal is to make the "good path" work in favorable conditions. It is also permissible to have "excusing circumstances" (a.k.a. documented defects) that render software inoperable. To contrast this, system developers typically have to deal with systems that are expected to work hell or high water. Well, again, outside of cases when failures are forgivable, like, management software / UI to the system rather than system itself. Same reason how you'd expect library code to be better tested and more robust in the face of edge cases than any individual application using the library. --- So, as you mentioned "as soon as the application spans multiple stores", application developers will give up all hope because there are too many layers, too many components, too much opaque third-party code. Whereas in systems, this is typically still all under your control. I.e. think about products like Ceph or Etcd -- they are designed to span multiple stores and still be correct. I had to go back and forth between application development and systems. And, for me, the world of application development is very unappealing specifically because of the attitude to quality. From my key-hole perspective, I wish people made fewer applications and concentrated on making systems more robust, because I feel like this would make computers more reliable, faster and cheaper... but there's a strong financial incentive to make applications, even though they are less reliable, slower and more expensive in the long term, because initial development is faster and cheaper.
- 3y ago