9 ms·

Related: you can get at most once delivery or at least once delivery; you cannot get exactly once delivery. If I had a dollar for every junior who thought that

Related: you can get at most once delivery or at least once delivery; you cannot get exactly once delivery. If I had a dollar for every junior who thought that a lack of exactly once delivery guarantees was a bug...

stanac 2y ago
Yup, I try to explain it with shouting a message to someone in a crowded room. You can yell at your boss "I fixed the bug", they can confirm it or ignore you, which is delivery at most once if you don't repeat the message. If you try to repeat the message until they confirm it, it is at least once delivery. edit: Point is in confirming that message is received. If you don't receive the confirmation the message was delivered at most once.
bcoates 2y ago
This is a popular saying that is basically wrong. You have very limited guarantees around an arbitrarily bad partition, but this is also a detectable condition. Lots of defective systems exist, but in general non-defective systems generally guarantee "exactly once delivery or detected failure"
- rcxdude 2y ago
  That sounds like "at most once" to me
  - bcoates 2y ago
    If you unplug the network you can't send messages, correct. Other people upthread have already gone over how you can't separate delivery from message processing and how TCP's attempt to do so makes it defective (unless you layer a whole additional system on top of it rendering most of TCP's design irrelevant) If you were trying to make a new, non-broken system on top of TCP or otherwise, allowing multiple delivery doesn't add any correctness/robustness benefits -- it just makes messages cheaper to send and receive. There is no "at most once or at least once" choice except in the pickwickian sense that if you don't require delivery or delivery confirmation you can save the effort of even trying.
lisper 2y ago
If you can get at-least-once delivery, why can you not build exactly-once on top of that? [UPDATE] Apparently I need to be more explicit about this. My question is: if I can get at-least-once delivery, why can I not build an abstraction layer on the receiving node that provides the illusion of exactly-once delivery? It seems like it should be a simple matter of keeping a log of received messages, and discarding duplicates.
- to11mtm 2y ago
  Because 'exactly once' delivery is arguably a misnomer, you usually really want 'at least once delivery with acks and idempotent processing on the other side'. The difference is subtle but important in practice and specification.
  - lisper 2y ago
    > you usually really want 'at least once delivery with acks and idempotent processing on the other side'. Why? I'm pretty sure I really want (the illusion of) exactly-once delivery, and it seems to me that I can implement that pretty easily given at-least-once delivery. Why would I not want that? > The difference is subtle but important Why?
    - ZephyrBlu 2y ago
      > I'm pretty sure I really want (the illusion of) exactly-once delivery Do you know what idempotency is? This is exactly what he described. Idempotency is important to prevent unwanted behaviour for duplicate actions. If you have "exactly-once", and accidentally execute the action twice that could cause problems.
      - lisper 2y ago
        > Do you know what idempotency is? Yes. > This is exactly what he described. So? Idempotency and an exactly-once delivery abstraction are not the same thing.
pclmulqdq 2y ago
So fun fact, you actually can get exactly once delivery out of your network, but your network has to be not Ethernet/IP/TCP to do it. Every single one of those layers is mis-designed to allow you to get exactly-once delivery of messages (TCP doesn't even have a concept of messages). Your network won't have "exactly once" message transfer happening on it (it will internally be "at least once" for certain packets, but only small ones) and administering it will be very different than administering an Ethernet network, but network protocols absolutely can be designed to give exactly-once delivery to your software. The real reason most people outside of HPC don't do this is that exactly-once at the network layer is not that useful for most web stuff. You're going to have a higher layer that will drop stuff and retry anyway, so you might as well push the problem up the stack.
lowbloodsugar 2y ago
Let me get some popcorn before reading these comments.

stanac 2y ago

Yup, I try to explain it with shouting a message to someone in a crowded room. You can yell at your boss "I fixed the bug", they can confirm it or ignore you, which is delivery at most once if you don't repeat the message. If you try to repeat the message until they confirm it, it is at least once delivery. edit: Point is in confirming that message is received. If you don't receive the confirmation the message was delivered at most once.

bcoates 2y ago

This is a popular saying that is basically wrong. You have very limited guarantees around an arbitrarily bad partition, but this is also a detectable condition. Lots of defective systems exist, but in general non-defective systems generally guarantee "exactly once delivery or detected failure"

rcxdude 2y ago
That sounds like "at most once" to me
- bcoates 2y ago
  If you unplug the network you can't send messages, correct. Other people upthread have already gone over how you can't separate delivery from message processing and how TCP's attempt to do so makes it defective (unless you layer a whole additional system on top of it rendering most of TCP's design irrelevant) If you were trying to make a new, non-broken system on top of TCP or otherwise, allowing multiple delivery doesn't add any correctness/robustness benefits -- it just makes messages cheaper to send and receive. There is no "at most once or at least once" choice except in the pickwickian sense that if you don't require delivery or delivery confirmation you can save the effort of even trying.

lisper 2y ago

If you can get at-least-once delivery, why can you not build exactly-once on top of that? [UPDATE] Apparently I need to be more explicit about this. My question is: if I can get at-least-once delivery, why can I not build an abstraction layer on the receiving node that provides the illusion of exactly-once delivery? It seems like it should be a simple matter of keeping a log of received messages, and discarding duplicates.

to11mtm 2y ago
Because 'exactly once' delivery is arguably a misnomer, you usually really want 'at least once delivery with acks and idempotent processing on the other side'. The difference is subtle but important in practice and specification.
- lisper 2y ago
  > you usually really want 'at least once delivery with acks and idempotent processing on the other side'. Why? I'm pretty sure I really want (the illusion of) exactly-once delivery, and it seems to me that I can implement that pretty easily given at-least-once delivery. Why would I not want that? > The difference is subtle but important Why?
  - ZephyrBlu 2y ago
    > I'm pretty sure I really want (the illusion of) exactly-once delivery Do you know what idempotency is? This is exactly what he described. Idempotency is important to prevent unwanted behaviour for duplicate actions. If you have "exactly-once", and accidentally execute the action twice that could cause problems.
    - lisper 2y ago
      > Do you know what idempotency is? Yes. > This is exactly what he described. So? Idempotency and an exactly-once delivery abstraction are not the same thing.

pclmulqdq 2y ago

So fun fact, you actually can get exactly once delivery out of your network, but your network has to be not Ethernet/IP/TCP to do it. Every single one of those layers is mis-designed to allow you to get exactly-once delivery of messages (TCP doesn't even have a concept of messages). Your network won't have "exactly once" message transfer happening on it (it will internally be "at least once" for certain packets, but only small ones) and administering it will be very different than administering an Ethernet network, but network protocols absolutely can be designed to give exactly-once delivery to your software. The real reason most people outside of HPC don't do this is that exactly-once at the network layer is not that useful for most web stuff. You're going to have a higher layer that will drop stuff and retry anyway, so you might as well push the problem up the stack.

lowbloodsugar 2y ago

Let me get some popcorn before reading these comments.