4 ms·

> If you make a queueing system, it's impossible to guarantee anything other than delivery "at most once" (some loss occurs), or "at least once" (some duplicati

> If you make a queueing system, it's impossible to guarantee anything other than delivery "at most once" (some loss occurs), or "at least once" (some duplication occurs), but if you usually provide "exactly once" in practice, most of your customers will depend on this. That's only a condition at termination. For ongoing communication, you can guarantee exactly once delivery. When communication ceases, the final state between the ends is indeterminate. If you can keep talking, or resume after breaks, it's a solvable problem.

avidiax 1y ago
That's true, but it doesn't help the customers. In a large system, terminations (both planned and unplanned) happen all the time. For the unplanned ones, it is very difficult to ensure exactly once, at least from the perspective of the queueing service, who can't check that the processing of the message did or did not occur outside of the original connection and it's "acks".
- Animats 1y ago
  > That's true, but it doesn't help the customers. Sure it does. It's essential to making banking work. See ISO 20022 reconcilation.[1] This is the process by which errors in money transactions between banks are resolved. ISO 20022 is used by SWIFT, FedNow, FedWire, and systems in about 70 countries. Unique IDs make transactions idempotent, but that's just the start. Banking is an eventually consistent system. Messages go back and forth until both ends agree. If there's a loss of communication, some transactions will be resubmitted, but they won't be applied twice. Banks have been doing reconciliation for centuries. It's been sped up recently, but the concepts have not changed much. [1] https://www.finextra.com/blogposting/19808/how-exactly-will-iso-20022-fix-reconciliations https://www.finextra.com/blogposting/19808/how-exactly-will-...
  - tkluck 1y ago
    You're not contradicting the post you're replying to; you're describing common workarounds to approximate exactly-once semantics using at-least-once semantics on the message queue combined with external storage on the reader side.

avidiax 1y ago

That's true, but it doesn't help the customers. In a large system, terminations (both planned and unplanned) happen all the time. For the unplanned ones, it is very difficult to ensure exactly once, at least from the perspective of the queueing service, who can't check that the processing of the message did or did not occur outside of the original connection and it's "acks".

Animats 1y ago
> That's true, but it doesn't help the customers. Sure it does. It's essential to making banking work. See ISO 20022 reconcilation.[1] This is the process by which errors in money transactions between banks are resolved. ISO 20022 is used by SWIFT, FedNow, FedWire, and systems in about 70 countries. Unique IDs make transactions idempotent, but that's just the start. Banking is an eventually consistent system. Messages go back and forth until both ends agree. If there's a loss of communication, some transactions will be resubmitted, but they won't be applied twice. Banks have been doing reconciliation for centuries. It's been sped up recently, but the concepts have not changed much. [1] https://www.finextra.com/blogposting/19808/how-exactly-will-iso-20022-fix-reconciliations https://www.finextra.com/blogposting/19808/how-exactly-will-...
- tkluck 1y ago
  You're not contradicting the post you're replying to; you're describing common workarounds to approximate exactly-once semantics using at-least-once semantics on the message queue combined with external storage on the reader side.