4 ms·

Isn't a fundamental property of a queue that it's FIFO? UDP message delivery order is not guaranteed. Hell, UDP delivery itself is not guaranteed (although IME

Isn't a fundamental property of a queue that it's FIFO? UDP message delivery order is not guaranteed. Hell, UDP delivery itself is not guaranteed (although IME, messages don't usually get dropped unless they cross subnets).

bdcravens 1y ago
> UDP delivery itself is not guaranteed > I asked if at-most-one delivery was ok, he said yes. Use case satisfied.
- trelane 1y ago
  >> UDP delivery itself is not guaranteed >> I asked if at-most-one delivery was ok, he said yes. > Use case satisfied. No. https://wiki.wireshark.org/DuplicatePackets https://wiki.wireshark.org/DuplicatePackets "ConnectionlessProtocols such as UDP won't detect duplicate packets, because there's no information in, for example, the UDP header to identify a packet so that packets can be recognized as duplicates. The data from that packet will be indicated twice (or even more) to the application; it's the responsibility of the application to detect duplicates (perhaps by supplying enough information in its headers to do so) and process them appropriately, if necessary"
  - ranger_danger 1y ago
    protection from duplicate packets was never a requirement... you also assume there is no other identifying info inside the packets that might provide an inherent protection anyways
    - trelane 1y ago
      > protection from duplicate packets was never a requirement.. Nope. This is what "at-most-one" means. Either zero or one. Not two. Three is right out. > you also assume there is no other identifying info inside the packets that might provide an inherent protection anyways Yes, but then if we're positing a system on top of UDP, then it doesn't matter what the protocol is, TCP or UDP. We can also assume the underlying system we're relying on can also detect missing requests, or even ensure we have exactly one.
      - ranger_danger 1y ago
        > Nope. This is what "at-most-one" means. Either zero or one. Not two. But "is at-most-one ok?" doesn't necessarily mean that more is also unacceptable. All the valid and invalid conditions are not completely specified IMO.
        dragonwriter 1y ago
        > But "is at-most-one ok?" doesn't necessarily mean that more is also unacceptable. If you propose delivering an at-most-once solution, the customer agrees that that is an acceptable parameter, and then you deliver a solution that has no at-most-once guarantee, you haven't delivered what you proposed, and you certainly have no reason to believe you've met the customer's needs.
        ranger_danger 1y ago
        The way OP framed it IMO sounded like a very informal conversation, not a written contract with formal specifications... I think it's possible the person agreeing at-most-one was "ok" quite possibly didn't care either way, and might even view supporting duplicates as a positive extra feature... but I don't think this is something either of us can speak to authoritatively. > if we're positing a system on top of UDP, then it doesn't matter what the protocol is And this is still true, and neither of us know if they would have designed it this way or not, but they could.
adamcharnock 1y ago
> UDP message delivery order is not guaranteed My thinking was that ordering would be pretty unaffected when there is only a single hop. But yeah, we would have needed to test that under load.
tomjen3 1y ago
Even Azures native queue systems can guarantee that, because it gets hard to scale. You can only have one processor for any given queue, because otherwise one might take so long to process a message that others have finished processing others in the meantime. But given OPs need for at-most-once, this doesn't really seem to matter in the first place.

bdcravens 1y ago

> UDP delivery itself is not guaranteed > I asked if at-most-one delivery was ok, he said yes. Use case satisfied.

trelane 1y ago
>> UDP delivery itself is not guaranteed >> I asked if at-most-one delivery was ok, he said yes. > Use case satisfied. No. https://wiki.wireshark.org/DuplicatePackets https://wiki.wireshark.org/DuplicatePackets "ConnectionlessProtocols such as UDP won't detect duplicate packets, because there's no information in, for example, the UDP header to identify a packet so that packets can be recognized as duplicates. The data from that packet will be indicated twice (or even more) to the application; it's the responsibility of the application to detect duplicates (perhaps by supplying enough information in its headers to do so) and process them appropriately, if necessary"
- ranger_danger 1y ago
  protection from duplicate packets was never a requirement... you also assume there is no other identifying info inside the packets that might provide an inherent protection anyways
  - trelane 1y ago
    > protection from duplicate packets was never a requirement.. Nope. This is what "at-most-one" means. Either zero or one. Not two. Three is right out. > you also assume there is no other identifying info inside the packets that might provide an inherent protection anyways Yes, but then if we're positing a system on top of UDP, then it doesn't matter what the protocol is, TCP or UDP. We can also assume the underlying system we're relying on can also detect missing requests, or even ensure we have exactly one.
    - ranger_danger 1y ago
      > Nope. This is what "at-most-one" means. Either zero or one. Not two. But "is at-most-one ok?" doesn't necessarily mean that more is also unacceptable. All the valid and invalid conditions are not completely specified IMO.
      - dragonwriter 1y ago
        > But "is at-most-one ok?" doesn't necessarily mean that more is also unacceptable. If you propose delivering an at-most-once solution, the customer agrees that that is an acceptable parameter, and then you deliver a solution that has no at-most-once guarantee, you haven't delivered what you proposed, and you certainly have no reason to believe you've met the customer's needs.
        ranger_danger 1y ago
        The way OP framed it IMO sounded like a very informal conversation, not a written contract with formal specifications... I think it's possible the person agreeing at-most-one was "ok" quite possibly didn't care either way, and might even view supporting duplicates as a positive extra feature... but I don't think this is something either of us can speak to authoritatively. > if we're positing a system on top of UDP, then it doesn't matter what the protocol is And this is still true, and neither of us know if they would have designed it this way or not, but they could.

adamcharnock 1y ago

> UDP message delivery order is not guaranteed My thinking was that ordering would be pretty unaffected when there is only a single hop. But yeah, we would have needed to test that under load.

tomjen3 1y ago

Even Azures native queue systems can guarantee that, because it gets hard to scale. You can only have one processor for any given queue, because otherwise one might take so long to process a message that others have finished processing others in the meantime. But given OPs need for at-most-once, this doesn't really seem to matter in the first place.