3 ms·
> Always pisses me off. University teaches the principles of ACID and how hard databases work to adhere to these principles, and then so called "NoSQL" comes al
by lmm 4y ago
> Always pisses me off. University teaches the principles of ACID and how hard databases work to adhere to these principles, and then so called "NoSQL" comes along and says "lol we have eventual consistency".
99% of the organisations that pride themselves on using "real databases" and ACID aren't actually using those guarantees or gaining anything out of them. Transactions are inherently useless in a web service, for example, because you can't open the transaction on the client, so the part of the system where the vast majority of consistency issues happen (the client <-> server communication) will always be outside the transaction boundary. ACID fanboys love to talk about how all the big name internet companies are built on RDBMSes, and fail to mention that most of them were built on MySQL 3 which never actually had working ACID in the first place.
If MongoDB had come first and SQL/ACID RDBMSes had come after, we'd recognise them for what they are: a grossly overengineered non-solution to the wrong problem.
- cmeacham98 4y ago> the part of the system where the vast majority of consistency issues happen This is survivorship bias: the reason the vast majority of consistency issues happen between the client and the server is _because_ the server can eliminate issues on its side with transactions and similar.
- lmm 4y agoNo, you're inherently a lot more likely to get inconsistency over the higher-latency and less-reliable public internet that between the server application and database that are probably sitting in the same rack, if not on the same machine.
- anyfoo 4y ago> Transactions are inherently useless in a web service, for example, because you can't open the transaction on the client, so the part of the system where the vast majority of consistency issues happen (the client <-> server communication) will always be outside the transaction boundary. I think I need more help understanding this. In terms of ACID, a consistency issue is something like, for example: "The client ordered an item, and the count of item availability was reduced by 1, but no actual order was saved in the database. An item is now 'lost' with no trace of where it went and how it disappeared." There is even another consistency hazard right in this very example: "The count of item availability was supposed to be reduced by 2 because 2 items were ordered by different clients at a same time, but a race condition (the two transactions both individually reading the same old value before writing back the new value) led to the item count only reduced by 1. There is now an extra item 'available' that does not exist." The canonical example is bank accounts and money transfers, but I figured I'd use something that was more likely in the domain of "99% of organizations" (though that also includes things like ERP, where a lot of other canonical examples apply). Care to elaborate how transactions are "inherently useless" in a web service here, and how the consistency issue happens "between client and server" communication? Presumably, in this example, the client/server communication triggering the transaction is the order of an item. There are not that many things that can realistically go wrong in that communication (with checksums on different layers, and nowadays even cryptography, guarding against most arbitrary corruption): The order was not received, or the order was received multiple times. In both those cases, the database remains internally consistent: Yes, the order happening 0 times or 2 times may have been wrong, but the database overall shows the state of what you would expect after 0 or 2 orders. No item has been going inexplicably "missing" or "created" until the next inventory, and the external issue can just be corrected by reissuing or canceling the orders, again with the prospect of an internally consistent database. And even a corrupted order would not change that.
- lmm 4y ago> In terms of ACID, a consistency issue is something like, for example: "The client ordered an item, and the count of item availability was reduced by 1, but no actual order was saved in the database. An item is now 'lost' with no trace of where it went and how it disappeared." > There is even another consistency hazard right in this very example: "The count of item availability was supposed to be reduced by 2 because 2 items were ordered by different clients at a same time, but a race condition (the two transactions both individually reading the same old value before writing back the new value) led to the item count only reduced by 1. There is now an extra item 'available' that does not exist." You don't need ACID to solve that kind of problem though; event sourcing and eventual consistency handles it fine. What you need ACID for is synchronous transactions; for example you want to check that both item A and item B are available, order both of them, and have it either both orders go through or neither of them, even though those are separate orders to separate tables (of course if you have a concept of an "order" as a first-class entity then you can trivially solve this - but again, in that case you don't need ACID at all). > Care to elaborate how transactions are "inherently useless" in a web service here, and how the consistency issue happens "between client and server" communication? So the example above - the whole point of ACID is to let you do two separate things at once, e.g. you want to open the page for item A and for item B, check that both are available, and then order them both on their respective pages. With an old-fashioned synchronous client-server system with a dedicated fat client you could do that, but on a web system obviously it's not possible (other than by creating some kind of "batched request" or "combined order" concept for doing both orders in the same request - but again, once you've done that you don't need ACID at all) because there's no way for the client to do two page loads in the same transaction.
- dragonwriter 4y ago> Transactions are inherently useless in a web service, for example, because you can’t open the transaction on the client (1) There is no reason you couldn’t, in a stateful, connected web app (or a stateful web service) open backend DB transactions controlled by activity on the client. You probably don’t want to, because, “ew, stateful”, and would prefer to use compensation strategies to deal with business transactions that span beyond a database transactions, but you could. (2) The conclusion “transactions are inherently useless in a web service” does not follow from the premise “you can’t open the transaction on the client”. They are just two completely unrelated things that you’ve glued together with “because”. I write web services. The backends very often need a consistent view of state and to be able to assure that a sequence of mutating operations against the DB are applied all or nothing; transactions do both of those things. The fact that transactions are opened from the backend doesn’t make them useless. > the part of the system where the vast majority of consistency issues happen (the client <-> server communication) will always be outside the transaction boundary. “Consistency issues” in the ACID sense do not (cannot, in fact, since “consistency” is a property of database state) happen anywhere other than inside the database. Client <-> server communications have all kinds of issues, but ACID;s C is not one of them. > ACID fanboys love to talk about how all the big name internet companies are built on RDBMSes Nah, actually, as an “ACID fanboy”, I’ll say that most big name internet companies were not built on ACID systems, and that if you are running a big internet company you have a much greater than usual chance of (1) having a case with a tradeoff that really does call for a non-ACID system, (2) having to invent revolutionary new technology if it turns out you actually do need an ACID system, because OTS RDBMS’s don’t scale the way you need. But for everyone else, you probably aren’t Google. > If MongoDB had come first and SQL/ACID RDBMSes had come after, While they weren’t MongoDB, specifically, non-relational, non-ACID, key-value stores where the “value” could be arbitrary data did exist before RDBMS’s. OTOH, users of MongoDB and other similar NoSQL system have often discovered that, oh yeah, they do want the things RDBMS’s provide, which is why the pendulum swung somewhat back in the RDBMS direction after peak NoSQL hype. NoSQL has its place, too, but the relational model and ACID guarantees actually do solve real problems.
- lmm 4y ago> (1) There is no reason you couldn’t, in a stateful, connected web app (or a stateful web service) open backend DB transactions controlled by activity on the client. You probably don’t want to, because, “ew, stateful”, and would prefer to use compensation strategies to deal with business transactions that span beyond a database transactions, but you could. You'd have to break the rules of how web systems are meant to work. HTTP requests are supposed to be independent (and may arrive out of order with each other etc.), holding a long-lived connection between server and client has limited support. > The backends very often need a consistent view of state and to be able to assure that a sequence of mutating operations against the DB are applied all or nothing; You never need that if these things are the result of a fixed client request; the set of things to be done can't change, so eventual consistency will serve the purpose. The case where you need ACID is where there's a complex back and forth to be had with the thing that's driving the change within the same transaction - and to the extent that "the thing that's driving the change" is solely on the backend, it's not really a web system. > “Consistency issues” in the ACID sense do not (cannot, in fact, since “consistency” is a property of database state) happen anywhere other than inside the database. Client <-> server communications have all kinds of issues, but ACID;s C is not one of them. Small-c consistency (actually I was thinking mainly in the CAP sense). Things like requesting to make a change based on stale data.