5 ms·

> and begin real investigation you will quickly find that many databases that are popular are totally fine losing data. MongoDB being the most famous example th

by anyfoo 4y ago

> and begin real investigation you will quickly find that many databases that are popular are totally fine losing data. MongoDB being the most famous example that I can think off of the top of my head. Always pisses me off. University teaches the principles of ACID and how hard databases work to adhere to these principles, and then so called "NoSQL" comes along and says "lol we have eventual consistency".

bombcar 4y ago
The heat death of the universe is eventually consistent.
- nickpeterson 4y ago
  Let’s hope our Kafka topics are caught up in time!
  - bombcar 4y ago
    Heat death of the universe has been postponed until Jira can be updated.
- candiddevmike 4y ago
  I don't think that is what the vendors mean when they discuss enforcing anti-entropy.
e40 4y ago
It is very frustrating that many of the NoSQL databases aren't ACID. Some even say they are, but they aren't. Frustrating to me because we have one that is, and it's a lot of work and harder to win benchmarks against the ones that fake it or aren't.
- whydoineedthis 4y ago
  which one is that?
  - e40 4y ago
    AllegroGraph. Written in Common Lisp.
- winrid 4y ago
  Which ones are lying?
  - e40 4y ago
    I don't want the liability for this, so I'm not going to say. Perhaps I should just delete the comment.
lmm 4y ago
> Always pisses me off. University teaches the principles of ACID and how hard databases work to adhere to these principles, and then so called "NoSQL" comes along and says "lol we have eventual consistency". 99% of the organisations that pride themselves on using "real databases" and ACID aren't actually using those guarantees or gaining anything out of them. Transactions are inherently useless in a web service, for example, because you can't open the transaction on the client, so the part of the system where the vast majority of consistency issues happen (the client <-> server communication) will always be outside the transaction boundary. ACID fanboys love to talk about how all the big name internet companies are built on RDBMSes, and fail to mention that most of them were built on MySQL 3 which never actually had working ACID in the first place. If MongoDB had come first and SQL/ACID RDBMSes had come after, we'd recognise them for what they are: a grossly overengineered non-solution to the wrong problem.
- cmeacham98 4y ago
  > the part of the system where the vast majority of consistency issues happen This is survivorship bias: the reason the vast majority of consistency issues happen between the client and the server is _because_ the server can eliminate issues on its side with transactions and similar.
  - lmm 4y ago
    No, you're inherently a lot more likely to get inconsistency over the higher-latency and less-reliable public internet that between the server application and database that are probably sitting in the same rack, if not on the same machine.
- anyfoo 4y ago
  > Transactions are inherently useless in a web service, for example, because you can't open the transaction on the client, so the part of the system where the vast majority of consistency issues happen (the client <-> server communication) will always be outside the transaction boundary. I think I need more help understanding this. In terms of ACID, a consistency issue is something like, for example: "The client ordered an item, and the count of item availability was reduced by 1, but no actual order was saved in the database. An item is now 'lost' with no trace of where it went and how it disappeared." There is even another consistency hazard right in this very example: "The count of item availability was supposed to be reduced by 2 because 2 items were ordered by different clients at a same time, but a race condition (the two transactions both individually reading the same old value before writing back the new value) led to the item count only reduced by 1. There is now an extra item 'available' that does not exist." The canonical example is bank accounts and money transfers, but I figured I'd use something that was more likely in the domain of "99% of organizations" (though that also includes things like ERP, where a lot of other canonical examples apply). Care to elaborate how transactions are "inherently useless" in a web service here, and how the consistency issue happens "between client and server" communication? Presumably, in this example, the client/server communication triggering the transaction is the order of an item. There are not that many things that can realistically go wrong in that communication (with checksums on different layers, and nowadays even cryptography, guarding against most arbitrary corruption): The order was not received, or the order was received multiple times. In both those cases, the database remains internally consistent: Yes, the order happening 0 times or 2 times may have been wrong, but the database overall shows the state of what you would expect after 0 or 2 orders. No item has been going inexplicably "missing" or "created" until the next inventory, and the external issue can just be corrected by reissuing or canceling the orders, again with the prospect of an internally consistent database. And even a corrupted order would not change that.

by anyfoo 4y ago

bombcar 4y ago

The heat death of the universe is eventually consistent.

nickpeterson 4y ago
Let’s hope our Kafka topics are caught up in time!
- bombcar 4y ago
  Heat death of the universe has been postponed until Jira can be updated.
candiddevmike 4y ago
I don't think that is what the vendors mean when they discuss enforcing anti-entropy.

e40 4y ago

It is very frustrating that many of the NoSQL databases aren't ACID. Some even say they are, but they aren't. Frustrating to me because we have one that is, and it's a lot of work and harder to win benchmarks against the ones that fake it or aren't.

whydoineedthis 4y ago
which one is that?
- e40 4y ago
  AllegroGraph. Written in Common Lisp.
winrid 4y ago
Which ones are lying?
- e40 4y ago
  I don't want the liability for this, so I'm not going to say. Perhaps I should just delete the comment.

lmm 4y ago

> Always pisses me off. University teaches the principles of ACID and how hard databases work to adhere to these principles, and then so called "NoSQL" comes along and says "lol we have eventual consistency". 99% of the organisations that pride themselves on using "real databases" and ACID aren't actually using those guarantees or gaining anything out of them. Transactions are inherently useless in a web service, for example, because you can't open the transaction on the client, so the part of the system where the vast majority of consistency issues happen (the client <-> server communication) will always be outside the transaction boundary. ACID fanboys love to talk about how all the big name internet companies are built on RDBMSes, and fail to mention that most of them were built on MySQL 3 which never actually had working ACID in the first place. If MongoDB had come first and SQL/ACID RDBMSes had come after, we'd recognise them for what they are: a grossly overengineered non-solution to the wrong problem.

cmeacham98 4y ago
> the part of the system where the vast majority of consistency issues happen This is survivorship bias: the reason the vast majority of consistency issues happen between the client and the server is _because_ the server can eliminate issues on its side with transactions and similar.
- lmm 4y ago
  No, you're inherently a lot more likely to get inconsistency over the higher-latency and less-reliable public internet that between the server application and database that are probably sitting in the same rack, if not on the same machine.
anyfoo 4y ago
> Transactions are inherently useless in a web service, for example, because you can't open the transaction on the client, so the part of the system where the vast majority of consistency issues happen (the client <-> server communication) will always be outside the transaction boundary. I think I need more help understanding this. In terms of ACID, a consistency issue is something like, for example: "The client ordered an item, and the count of item availability was reduced by 1, but no actual order was saved in the database. An item is now 'lost' with no trace of where it went and how it disappeared." There is even another consistency hazard right in this very example: "The count of item availability was supposed to be reduced by 2 because 2 items were ordered by different clients at a same time, but a race condition (the two transactions both individually reading the same old value before writing back the new value) led to the item count only reduced by 1. There is now an extra item 'available' that does not exist." The canonical example is bank accounts and money transfers, but I figured I'd use something that was more likely in the domain of "99% of organizations" (though that also includes things like ERP, where a lot of other canonical examples apply). Care to elaborate how transactions are "inherently useless" in a web service here, and how the consistency issue happens "between client and server" communication? Presumably, in this example, the client/server communication triggering the transaction is the order of an item. There are not that many things that can realistically go wrong in that communication (with checksums on different layers, and nowadays even cryptography, guarding against most arbitrary corruption): The order was not received, or the order was received multiple times. In both those cases, the database remains internally consistent: Yes, the order happening 0 times or 2 times may have been wrong, but the database overall shows the state of what you would expect after 0 or 2 orders. No item has been going inexplicably "missing" or "created" until the next inventory, and the external issue can just be corrected by reissuing or canceling the orders, again with the prospect of an internally consistent database. And even a corrupted order would not change that.