6 ms·
UDP isn't unreliable, it's a convertible (2024)
- eptcyka 1y agoReliability is only achieved when you’ve spilled blood at least tens of times figuring out the next reliability hurdle. TCP does MTU detection, bandwidth detection and all this at relative real time, not once per connection.
- cestith 1y agoUDP doesn’t back off sending for congestion. How reliable is that? UDP over the network does not on its own guarantee packet retries or proper delivery order. How reliable is that? UDP on Linux to localhost or hair-pinned between two public IPs on the same host can result in reordered packets when the kernel is busy with a lot of traffic or the CPU is context switching enough. It happens when a UDP queue is handled by a certain core and gets swapped to another. I’ve had to set up two machines with a cable between them because the ordering is actually more stable than over localhost. A colleague is working on a kernel patch to bypass part of the UDP stack to restore proper ordering, which will probably need to be maintained in-house or at least hidden behind a kernel config knob upstream since this reordering is considered acceptable under the guarantees for UDP. How reliable is that? So, yeah, you can say UDP is reliable but compared to TCP or QUIC it actually really isn't.
- threatofrain 1y agoSome of these things turn out to be death penalty depending on the kind of app you're doing, and it can be quite surprising when it happens too. For example, redelivery with ordering with bad internet can lead to unbounded backup.
- Kiboneu 1y agoThe point of the article is that calling a protocol reliable doesn't make sense. You can (and should) use UDP if you don't care about (or you have a special way of handling) packet loss, and instead need to receive the latest packet as soon as possible. In the context of multiplayer FPS games and remote controlling drones -- it is actually more reliable.
- eptcyka 1y agoThe protocol is reliable in the sense that the protocol provides guarantees where either party can know that the other party received something. UDP provides no such guarantees. I'm a fan of using UDP where applicable, I loathe TCP-in-TCP, not at all a fan of head of line blocking, but I don't need to call UDP reliable just because it is a great protocol with it's own usecases.
- Kiboneu 1y agoTCP does not guarantee delivery, and there are cases where getting confirmation of receipt actually makes the /channel/ less reliable. Again, the point is not to call UDP reliable, it's that it doesn't make sense for the reasons I stated above. If you choose the wrong tool for a usecase then the communication channel will be unreliable in either case. There's a lot of confusion about this -- it is valid to call a communication channel unreliable for the system to respond accordingly, not the protocol itself.
- eptcyka 1y agoTCP does not guarantee delivery, but it guarantees that if you receive an ACK, then the acknowledged bytes were received. UDP provides no such guarantee. Yes, it is OK to call it unreliable, because you can't rely on it to reliably deliver packets - the sender will never know if the packets it sent were ever delivered without any external validation. Game servers and clients work around this by being fault-tolerant. QUIC runs on UDP but the way it achieves reliability is by implementing reliability of TCP in the application layer. You cannot expect to receive all UDP packets you send to yourself on localhost, I think that it is reasonable to call such a network protocol unreliable, in the sense that one cannot reliably know if sent packets have ever been delivered within the confines of the protocol itself.
- Kiboneu 1y ago(I don't know if you've seen my edit. I re-read your response to answer more relevantly to your point about ACK. But to be honest I still don't think it's correct.)
- toast0 1y ago> A colleague is working on a kernel patch to bypass part of the UDP stack to restore proper ordering, which will probably need to be maintained in-house or at least hidden behind a kernel config knob upstream since this reordering is considered acceptable under the guarantees for UDP. My understanding is that in Linux, TCP on localhost (including packets sent to a public address on a local interface) bypasses the stack; I don't see why it would be a problem for UDP to do the same. This is contrast with FreeBSD, where TCP to localhost is actually packetized and queued on the loopback interface, and you can experience congestion collapse on loopback if you do things right(wrong).
- cestith 1y agoTCP on localhost does bypass a certain part of the stack. That’s the same part he’s looking to bypass.
- MoltenMan 1y agoAm I going insane or is this an LLM post with all the em dashes replaced by double en dashes??
- deleted 1y ago[deleted]
- shiomiru 1y agoIt's deep in the uncanny valley. Even ignoring the writing style, the entire post is about the incredible insight of "you can build TCP on top of UDP" using a forced analogy. No further analysis, no real world example, just UDP and the convertible and the convertible and UDP...
- weepinbell 1y agohttps://www.pangram.com/history/f119a8ee-7e60-442a-8d0c-3fa0fe96d4b7/ https://www.pangram.com/history/f119a8ee-7e60-442a-8d0c-3fa0... I show this more of as a discussion point than as a definitive answer. There is more research on this tool than others: https://www.nature.com/articles/d41586-025-02936-6 https://www.nature.com/articles/d41586-025-02936-6 And I think it's interesting that it flags it as confidently AI generated. I also got a whiff of AI and I'm never sure how to take confirmation bias from AI detectors - though that said, I've gotten a whiff of AI before and had this detector say it's confidently human. Reading this I kept waiting for the... point? I feel like the whole thing was more like saying "UDP is like a convertible because you can strap a tarp on top when it rains". Like... sure? But that tarp is going to be crappy compared to a real roof. And the idea that that tiny layer is "the best of both worlds" is frankly ridiculous to me.
- mlhpdx 1y agoAuthor here. I've been called a "robot" in the past so perhaps my writing reflects that, but yes there was some autocomplete involved here.
- abound 1y agoI felt the same way, which is terrible news for me because I explicitly use double dashes in my (100% human) writing instead of em/en dashes -- explicitly because of the LLM association, but they're evolving
- diath 1y ago> It's so common it's almost an instinct. But that description doesn't quite hold up once you look closer. UDP isn't unreliable in the sense that it fails to do its job -- it's unreliable in the same way a convertible is "un-dry". Well, no, it's unreliable in the sense that if I ordered a package online, the courier may get lost on the way and never come with my package, and then I have to order it again if I want it delivered, and then hope that this time it actually does, in that case, it kind of does fail to do its job if you want things to be delivered.
- pklausler 1y agoIf you want your packages delivered in order, without duplication, whenever it is not impossible to get them to you -- don't use a fulfillment company that might send you multiple boxes, or get them out of order, or not deliver them at all, and is really no cheaper anyway.
- elevation 1y agoUDP is perfectly reliable when you control both sides of the link. With adjustments to ring buffers, IRQ/core affinity, and receive buffer size, you can receive a saturated 40GbE stream with no packet loss. It's only when you pass through a router that's specifically instructed "drop these first" that you run into drops.
- ekr____ 1y agoI would argue with "only". Full queues are only one reason you get packet drops. For example, packets can be damaged in transmission. This isn't common but that's not the same as saying it doesn't happen.
- klysm 1y agoOkay, but the important part of a protocol is what happens when things don't go perfectly to plan.
- jeroenhd 1y agoUDP has a checksum for a reason, bitflips happen and packets get mandled or lost, even if you control both sides of the link. Your chances are better but you can't assume packet loss or data errors can't happen just because the fiber is right in front of you.
- jandrese 1y agoThe UDP checksum is optional and weak. It's mostly a waste of time. It's far more likely if you get a bit flip that the layer 1 hardware will just drop the packet because it has a much better checksum that detected the error first. Even if you know every part of the hardware and software stack you have to plan for an occasional packet drop.
- jeroenhd 1y agoMy point wasn't that you should rely on the weak checksum, but rather that the possibility for failure is already baked in even for such a small set of fields, because errors of any kind can happen at any point even in the simplest setups. The statement With adjustments to ring buffers, IRQ/core affinity, and receive buffer size, you can receive a saturated 40GbE stream with no packet loss. _Probably_ works out _most_ of the time in physical setups (especially in server environments with strict ECC memory), but it's not something you can write a protocol around if you need all of that data to _actually_ arrive.
- wat10000 1y agoThis is a rather long-winded way of saying "unreliable means it doesn't retransmit for you, it doesn't mean it's shit." Which, duh? I don't know, maybe there are people out there who need to hear this, but I'd think that anyone who knows what UDP is at all will also understand the sense in which it's "unreliable."
- mlhpdx 1y agoFair. But there are legions of developers blindly doing REST over HTTP and creating bad experiences. I know it's fine for many, many cases -- but not all.
- igravious 1y agoSemantically something is unreliable if it's meant to be reliable I'd say. UDP is guaranteed to not be reliable.
- tptacek 1y agoThis is something that the classic TCP/IP books (both Comer and Stevens) got wrong, inexplicably, because David Reed was very clear about it. UDP isn't a transport protocol at all. It's an extension interface for building new transport protocols. It is the thing you would use (and weirdly didn't) to ship SCTP, and did use to build QUIC. It does not make sense to compare UDP to TCP; it's like saying TCP is worse at URL handling than HTTP. Talking about what UDP provides is a little bit like talking about what IP provides. It's almost a category error. Is "UDP" "reliable"? It's as reliable as whatever transport you build on it is!
- ekr____ 1y agoI agree with you about the category error. In all fairness, though, there are quite a few application protocols which are built directly on top of UDP with no explicit intermediate transport layer. DNS, RTP, and even sometimes SIP come immediately to mind.
- tptacek 1y agoI'd argue that DNS has an implicit transport protocol by dint of the QIDs. It's a very simple transport! But that's the point, right.
- ekr____ 1y agoI think we agree here: protocols built on top of UDP often build their own ad hoc reliability layer, as opposed to having an explicit one like TCP, QUIC, or SCTP.
- juliusdavies 1y agoSIP still has a reliability layer built on top: the humans will say “I didn’t catch that, can you repeat yourself?“ on the video call.
- jeroenhd 1y agoMore so than that, SIP itself has a ACK system to resend dropped packets or report failures. The RTP part will carry the "can you repeat that" part, unless you're sending DM's over SIP INFO messages.
- BinaryIgor 1y agoInteresting perspective, agreed 100%; having UDP available in the browser would open up a plethora of interesting new use cases - games, video/audio calls without WebRTC overhead - and techniques to use. Ohh wait - we will soon have it! https://developer.mozilla.org/en-US/docs/Web/API/WebTransport_API https://developer.mozilla.org/en-US/docs/Web/API/WebTranspor...
- SoftTalker 1y agoI don't like the convertible analogy. It doesn't have anything to do with reliability. I'd compare it to the mail. UDP is regular first-class mail. You send a letter, it probably gets to the address on the envelope. But it might get delayed, or even simply lost, and you'll never know. TCP is like certified mail. It is sent by the same physical systems, but it is tracked and you get an acknowledgement that the recipient received it.
- EvanAnderson 1y agoI always use the mail analogy when talking about unreliable packet delivery with non-technical people. People seem to grok it very quickly. (It makes spoofing and amplification attacks easy to communicate, too.)
- mlhpdx 1y agoAuthor here, and I don't really like it that much either. The problem is finding some new way to pose it so the only content out there isn't misleadingly trite.
- klysm 1y agoThis is such a bizarre read... it's clearly LLM written, at least in outline. It somehow manages to not do the most basic comparison of guarantees that UDP and TCP provide. Ordering is only mentioned in passing, and the important nuances are completely bypassed.
- jeroenhd 1y agoIt's written by someone trying to sell specialized cloud services targeted towards UDP based protocols. Talking up UDP like it's something special is part of the business strategy. And, to be fair, a lot of TCP-based or even HTTP-based application protocols could probably have been UDP without any trouble.
- klysm 1y agoUDP introduces trouble and the question is whether that’s worth it. It usually isn’t
- Maxatar 1y agoEven the title uses one of ChatGPT's signature lines "It's not X, it's Y."
- mlhpdx 1y agoSadly, that's almost an expected way of writing online content now. I'll do better next time!
- throw0101c 1y agoSadly the proliferation of middleboxes means that deploying a new Layer 4 is all but impossible: * https://en.wikipedia.org/wiki/Datagram_Congestion_Control_Protocol https://en.wikipedia.org/wiki/Datagram_Congestion_Control_Pr... * https://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol https://en.wikipedia.org/wiki/Stream_Control_Transmission_Pr... Curious to know if things can every move towards a point of allowing things. I know in many firewall 'languages' do have something like "allow tcp/443" or "allow udp/53" for nomenclature: perhaps something like "allow stream/443" and "allow dgram/53" to include (e.g.) TCP/SCTP and UDP/DCCP would allow more flexibility.
- tsimionescu 1y agoI never really understood this complaint. UDP adds such a tiny and useful header on top of IP that it hardly seems meaningful to reinvent the wheel instead of just using it.
- throw0101c 1y agoSee §Problem Space on what it was thought that UDP lacked that caused the creation of DCCP: * https://datatracker.ietf.org/doc/html/rfc4336#section-2 https://datatracker.ietf.org/doc/html/rfc4336#section-2
- tsimionescu 1y agoI'm not asking why you'd need a new transport protocol, that much is clear. The question is what is the disadvantage of building this new protocol on top of UDP, especially if you don't expect to work with middlebox developers to add special features for your new protocol. The only thing I get from that document that answers this question to some extent is the extra overhead of the checksum field in UDP headers. In other places, they generally don't seem to actually discuss the option of building a QUIC-like transport, that is a transport-layer protocol that still exists on top of the UDP skeleton.
- anticensor 1y ago
- blackcatsec 1y agoAs with any protocol and application, there's a big, massive, "it depends" on pretty much everything. There are pros, cons, tradeoffs to make that sometimes it's easier to just have the underlying platform make those trade-offs, or to surface those problems to the application layer for developers to decide how they want to handle those problems. Folks shouldn't necessarily argue "one is better" more so than they should consider all engineering aspects of why you should use one technology over another.
- chasing0entropy 1y agoI'm not a fan of the 'convertible' metaphor, UDP would be better equated to driving at 100mph on a 55mph TCP road. You might get there safely with no harm at twice the TCP speed... or you might lose some parts or you may never arrive. That said the old battle.net online and Quake/Unreal multiplayer system was a widespread implementation of UDP transport with reliability checks. Whenever your UDP stream desynced the game clock accelerated replaying the buffered moves back to back until you were back in sync.
- mlhpdx 1y agoThis video walkthrough of the Quake netcode shows some of the brilliance that can be applied over UDP, too. https://www.youtube.com/watch?v=b8J7fidxC8s https://www.youtube.com/watch?v=b8J7fidxC8s
- deleted 1y ago[deleted]