4 ms·

> Transactions are inherently useless in a web service, for example, because you can't open the transaction on the client, so the part of the system where the v

by anyfoo 4y ago

> Transactions are inherently useless in a web service, for example, because you can't open the transaction on the client, so the part of the system where the vast majority of consistency issues happen (the client <-> server communication) will always be outside the transaction boundary. I think I need more help understanding this. In terms of ACID, a consistency issue is something like, for example: "The client ordered an item, and the count of item availability was reduced by 1, but no actual order was saved in the database. An item is now 'lost' with no trace of where it went and how it disappeared." There is even another consistency hazard right in this very example: "The count of item availability was supposed to be reduced by 2 because 2 items were ordered by different clients at a same time, but a race condition (the two transactions both individually reading the same old value before writing back the new value) led to the item count only reduced by 1. There is now an extra item 'available' that does not exist." The canonical example is bank accounts and money transfers, but I figured I'd use something that was more likely in the domain of "99% of organizations" (though that also includes things like ERP, where a lot of other canonical examples apply). Care to elaborate how transactions are "inherently useless" in a web service here, and how the consistency issue happens "between client and server" communication? Presumably, in this example, the client/server communication triggering the transaction is the order of an item. There are not that many things that can realistically go wrong in that communication (with checksums on different layers, and nowadays even cryptography, guarding against most arbitrary corruption): The order was not received, or the order was received multiple times. In both those cases, the database remains internally consistent: Yes, the order happening 0 times or 2 times may have been wrong, but the database overall shows the state of what you would expect after 0 or 2 orders. No item has been going inexplicably "missing" or "created" until the next inventory, and the external issue can just be corrected by reissuing or canceling the orders, again with the prospect of an internally consistent database. And even a corrupted order would not change that.

lmm 4y ago
> In terms of ACID, a consistency issue is something like, for example: "The client ordered an item, and the count of item availability was reduced by 1, but no actual order was saved in the database. An item is now 'lost' with no trace of where it went and how it disappeared." > There is even another consistency hazard right in this very example: "The count of item availability was supposed to be reduced by 2 because 2 items were ordered by different clients at a same time, but a race condition (the two transactions both individually reading the same old value before writing back the new value) led to the item count only reduced by 1. There is now an extra item 'available' that does not exist." You don't need ACID to solve that kind of problem though; event sourcing and eventual consistency handles it fine. What you need ACID for is synchronous transactions; for example you want to check that both item A and item B are available, order both of them, and have it either both orders go through or neither of them, even though those are separate orders to separate tables (of course if you have a concept of an "order" as a first-class entity then you can trivially solve this - but again, in that case you don't need ACID at all). > Care to elaborate how transactions are "inherently useless" in a web service here, and how the consistency issue happens "between client and server" communication? So the example above - the whole point of ACID is to let you do two separate things at once, e.g. you want to open the page for item A and for item B, check that both are available, and then order them both on their respective pages. With an old-fashioned synchronous client-server system with a dedicated fat client you could do that, but on a web system obviously it's not possible (other than by creating some kind of "batched request" or "combined order" concept for doing both orders in the same request - but again, once you've done that you don't need ACID at all) because there's no way for the client to do two page loads in the same transaction.
- Mavvie 4y ago
  How can eventual consistency handle the inventory use case as well as an ACID system? You can't tell the user "your order is placed successfully" and then later tell them "actually no, it was out of stock".
  - lmm 4y ago
    That's a different problem from your previous examples, and it's the same problem whether it's ACID or not: there's always an opportunity for the user to see the item as being in stock, then request to order it, and discover it's out of stock. Your "recovery" from the case where there's nothing in stock doesn't need to happen in a transaction (and can't meaningfully do so); you just tell the user their order failed, and again you do that the same way whether it's ACID or not.
    - Mavvie 4y ago
      I see what you mean, but with ACID you can tell the user right away if the order was successfully placed. With eventual consistency you don't know how long it'll take so you either show them a loading spinner that takes an indeterminate amount of time, or what? Not saying that the spinner solution is wrong (as it definitely scales better), but the immediate feedback is something you lose with eventual consistency, by definition, right?
      - tsegratis 4y ago
        Im no expert but my recent conclusion from writing an ordering system is customers can instruct their banks to roll back many payments a month or even a year after the fact So actual ACID transactional consistency requires a year. So after basic checks just say you'll give them what they want and accept payment -- since you'll need failure recovery anyway for lack of delivery, refunds, acts of God etc So accept and plan for failure
        Mavvie 4y ago
        The idea I've seen repeated here a lot is to aim for strong consistency internal to your system, even though the real world/things external to your system will be eventually consistent. So the inventory <-> create order interactions are internal and can be strongly consistent, while the realities of the banking system are external and will be eventually consistent. Just because you can't make everything strongly consistent doesn't mean you shouldn't bother at all; it can still eliminate entire classes of errors (in this case, errors around inventory being over-consumed leading to cancelled orders are completely avoided with ACID) edit: I'm also no expert when it comes to distributed systems. Most of my experience is with transactional systems which I know quite well, and distributed is still a mystery to me. I am very open to new ideas around it and don't have a ton of confidence in my comments here.

by anyfoo 4y ago

lmm 4y ago

> In terms of ACID, a consistency issue is something like, for example: "The client ordered an item, and the count of item availability was reduced by 1, but no actual order was saved in the database. An item is now 'lost' with no trace of where it went and how it disappeared." > There is even another consistency hazard right in this very example: "The count of item availability was supposed to be reduced by 2 because 2 items were ordered by different clients at a same time, but a race condition (the two transactions both individually reading the same old value before writing back the new value) led to the item count only reduced by 1. There is now an extra item 'available' that does not exist." You don't need ACID to solve that kind of problem though; event sourcing and eventual consistency handles it fine. What you need ACID for is synchronous transactions; for example you want to check that both item A and item B are available, order both of them, and have it either both orders go through or neither of them, even though those are separate orders to separate tables (of course if you have a concept of an "order" as a first-class entity then you can trivially solve this - but again, in that case you don't need ACID at all). > Care to elaborate how transactions are "inherently useless" in a web service here, and how the consistency issue happens "between client and server" communication? So the example above - the whole point of ACID is to let you do two separate things at once, e.g. you want to open the page for item A and for item B, check that both are available, and then order them both on their respective pages. With an old-fashioned synchronous client-server system with a dedicated fat client you could do that, but on a web system obviously it's not possible (other than by creating some kind of "batched request" or "combined order" concept for doing both orders in the same request - but again, once you've done that you don't need ACID at all) because there's no way for the client to do two page loads in the same transaction.

Mavvie 4y ago
How can eventual consistency handle the inventory use case as well as an ACID system? You can't tell the user "your order is placed successfully" and then later tell them "actually no, it was out of stock".
- lmm 4y ago
  That's a different problem from your previous examples, and it's the same problem whether it's ACID or not: there's always an opportunity for the user to see the item as being in stock, then request to order it, and discover it's out of stock. Your "recovery" from the case where there's nothing in stock doesn't need to happen in a transaction (and can't meaningfully do so); you just tell the user their order failed, and again you do that the same way whether it's ACID or not.
  - Mavvie 4y ago
    I see what you mean, but with ACID you can tell the user right away if the order was successfully placed. With eventual consistency you don't know how long it'll take so you either show them a loading spinner that takes an indeterminate amount of time, or what? Not saying that the spinner solution is wrong (as it definitely scales better), but the immediate feedback is something you lose with eventual consistency, by definition, right?
    - tsegratis 4y ago
      Im no expert but my recent conclusion from writing an ordering system is customers can instruct their banks to roll back many payments a month or even a year after the fact So actual ACID transactional consistency requires a year. So after basic checks just say you'll give them what they want and accept payment -- since you'll need failure recovery anyway for lack of delivery, refunds, acts of God etc So accept and plan for failure
      - Mavvie 4y ago
        The idea I've seen repeated here a lot is to aim for strong consistency internal to your system, even though the real world/things external to your system will be eventually consistent. So the inventory <-> create order interactions are internal and can be strongly consistent, while the realities of the banking system are external and will be eventually consistent. Just because you can't make everything strongly consistent doesn't mean you shouldn't bother at all; it can still eliminate entire classes of errors (in this case, errors around inventory being over-consumed leading to cancelled orders are completely avoided with ACID) edit: I'm also no expert when it comes to distributed systems. Most of my experience is with transactional systems which I know quite well, and distributed is still a mystery to me. I am very open to new ideas around it and don't have a ton of confidence in my comments here.