6 ms·

UDP is just about as reliable or unreliable as IP. It's a shim on top of IP to let unprivileged users send IP datagrams that can be multiplexed back to the righ

UDP is just about as reliable or unreliable as IP. It's a shim on top of IP to let unprivileged users send IP datagrams that can be multiplexed back to the right user. (Hence the name: "user" datagram protocol.) Lots of people talk about a "TCP or UDP" design decision, but that's usually not the relevant question for an application developer. Most UDP applications wouldn't just blat their data directly into UDP datagrams, just like almost nobody blats their data directly into IP datagrams. Typical applications generally want some sort of transport protocol on top of the datagrams (whether UDP-in-IP or just plain IP) that cares about reliability and often about fairness. That could be TCP-over-IP, but some (like MixApp, which wanted TCP but not the kernel's default implementation) use TCP-over-UDP-over-IP, BitTorrent uses LEDBAT-over-UDP, Mosh uses SSP-over-UDP, Chrome uses QUIC-over-UDP and falls back to SPDY-over-TCP and HTTP-over-TCP, etc. The idea that using UDP somewhere in the stack means "designing and implementing a reliable transport protocol from scratch" is silly. It doesn't mean that any more than using IP does. The question of SOCK_STREAM vs. SOCK_DGRAM is more typically, "Does the application want to use the one transport protocol that the kernel implements, namely a particular flavor of TCP, or does it want to use some other transport protocol implemented in a userspace library?"

huhtenberg 12y ago
> UDP is just about as reliable or unreliable as IP That's in theory. In practice ISPs routinely classify, prioritize and shape application traffic (based on either TCP/UDP ports or results of deep packet inspection), so running SSL over IP will yield different IP loss profile than when running SSL over UDP.
antirez 12y ago
Fun how this is more or less the contrary of what I think, so I'll add my point of view as a counter argument. IMHO if you need a transport layer, you are 99.9% of cases well served by TCP and should not consider UDP at all if not as a last resort to implement your transport protocol. Instead UDP is the way to go when you don't need a transport layer at all, and you can do with trivial ack / resend strategy (DNS is the obvious example), or just best-effort transfers where packet loss is not an issue at all (think a thermometer sending readings every second). In those cases the TCP three way handshake would be an overkill, and the benefits of TCP are minimal because of the use case.
- _gsin 12y ago
  > Instead UDP is the way to go when you don't need a transport layer at all, and you can do with trivial ack / resend strategy (DNS is the obvious example), or just best-effort transfers where packet loss is not an issue at all (think a thermometer sending readings every second) That depends I guess. As somebody who works mostly with cloud telephony, SIP (signaling) and RTP (audio) are almost always carried over UDP. Funny thing is that you'll see more occurrences of one-way audio or call drops because of people implementing the application level protocol like SIP wrong rather than the transport layer letting you down. That said, I would any day prefer a private MPLS network rather than the public internet. Though I can understand why somebody who writes a fast nosql database prefers TCP. Also, thanks for redis, we use it a lot and love it! :)
  - colanderman 12y ago
    That said, I would any day prefer a private MPLS network rather than the public internet. Thanks to your comment, a short Wikipedia trip later, I now know that "penultimate hop popping" is a thing and has a fantastic name.
  - MichaelGG 12y ago
    SIP signalling over UDP was a bad idea, especially with the mandatory TCP failover. SIP's actually a spectacular clusterfuck of a design overall anyways. It's actually impossible to implement SIP in an unambiguous manner on today's networks because even the parsing rules are so bad, that multiple, popular, stacks have incompatible ways of dealing with things. UDP for RTP makes sense because transmitting packets is pointless. In case of packetloss, the end user has to interpolate the result either via software or in their head. This private MPLS network... it wouldn't happen to run over the exact same equipment that'd be handling your IP traffic, would it?
    - bradleyland 12y ago
      Just a small correction for anyone else following along. This: > UDP for RTP makes sense because transmitting packets is pointless Should be this: > UDP for RTP makes sense because retransmitting packets is pointless TCP will retransmit if no ACK is received; UDP won't (there being no ACK and all).
paraboul 12y ago
Right. One usually want to reimplement a reliable protocol (e.g. similar to TCP) on top of UDP when dealing with peer-to-peer capabilities. Indeed, NAT traversal is easier to implement and more reliable when dealing with UDP techniques (e.g. UDP hole punching).
jacquesm 12y ago
> UDP is just about as reliable or unreliable as IP. Ok. > It's a shim on top of IP to let unprivileged users send IP datagrams that can be multiplexed back to the right user. (Hence the name: "user" datagram protocol.) It's mostly a way to send data in such a manner that you don't need to do the full 'call-setup-data-transmission-and-terminate' that would be required for other virtual circuit based protocols such as TCP when you don't need all that luxury. So for protocols that carry small amounts of data in a manner where retrying is not a problem and where loss of a packet is not an immediate disaster. It's also more suitable for real-time applications because of this than TCP (especially true for the first packet). Because of the fact that there is no virtual circuit a single listener can handle data from multiple senders. The 'USER' does not refer to unprivileged users but simply to users as opposed to system packets (such as for instance ICMP and other datagram like packets that are not usually sent out directly by applications). So it's not a privilege matter but a matter of user-space vs system modules elsewhere in the stack. > Lots of people talk about a "TCP or UDP" design decision, but that's usually not the relevant question for an application developer. It absolutely is. > Most UDP applications wouldn't just blat their data directly into UDP datagrams, They usually do exactly that. > just like almost nobody blats their data directly into IP datagrams. You're comparing apples with oranges, IP is one layer and TCP and UDP are on another. So you'd have to compare UDP with TCP and then you're back to that design decision again. > Typical applications generally want some sort of transport protocol on top of the datagrams (whether UDP-in-IP or just plain IP), that cares about reliability and often about fairness. Fairness is something that is usually not under control of the endpoints of a conversation but something that routers in between influence. They can decide to let a packet through or drop it (this goes for TCP as well as UDP), if a line is congested your UDP packets will usually (rules can be set to configure this) be dropped before your TCP packets will be in spite of the fact that TCP will re-try any lost packets. UDP packets can also be duplicated and routed in such a way that they arrive out-of-order. > That could be TCP-over-IP, but some (like MixApp, which wanted TCP but not the kernel's default implementation) use TCP-over-UDP-over-IP, BitTorrent uses LEDBAT-over-UDP, Mosh uses SSP-over-UDP, Chrome uses QUIC-over-UDP and falls back to SPDY-over-TCP and HTTP-over-TCP, etc. Running alternative protocols packaged inside other protocols is a time honored practice. See also: TCP over carrier pigeons and tunneling HTTP over DNS traffic (effectively using UDP). This is not in any way special, it's just a means to an end. > The idea that using UDP somewhere in the stack means "designing and implementing a reliable transport protocol from scratch" is silly. It doesn't mean that any more than using IP does. It actually comes down to exactly that. If you use UDP as your base and your application requires reliable transmission of data then you're going to have to deal with loss/duplication/sequencing at some other point in your application or put another (pre existing) protocol on top of it in order to mitigate these. If your application can tolerate those errors (or if they are not considered errors) then a naive implementation will do. > The question of SOCK_STREAM vs. SOCK_DGRAM is more typically, "Does the application want to use the one transport protocol that the kernel implements, namely a particular flavor of TCP, or does it want to use some other transport protocol implemented in a userspace library?" TCP is the default for anything requiring virtual circuits if you have demands that are not well describe by that model and/or need real time, low overhead and you're willing to do the work required to deal with UDPs inherent issues (if those are a problem) then you're totally free to do so. But the question is usually not 'do I need TCP', it usually is 'how do I avoid re-implementing TCP if I need its features'. It's a tough choice because at a minimum it means that you're going to have to write software for both endpoints. This is one of the reasons why we see HTTP over TCP in so many places where it wasn't originally intended: it is more or less guaranteed to be well tested and there are tons of tools available to use this protocol combination, especially browsers, fetchers and servers in all kinds of flavors. For UDP that situation is much less rosy and using UDP always translates into having to do a bunch of plumbing yourself.

huhtenberg 12y ago

> UDP is just about as reliable or unreliable as IP That's in theory. In practice ISPs routinely classify, prioritize and shape application traffic (based on either TCP/UDP ports or results of deep packet inspection), so running SSL over IP will yield different IP loss profile than when running SSL over UDP.

antirez 12y ago

Fun how this is more or less the contrary of what I think, so I'll add my point of view as a counter argument. IMHO if you need a transport layer, you are 99.9% of cases well served by TCP and should not consider UDP at all if not as a last resort to implement your transport protocol. Instead UDP is the way to go when you don't need a transport layer at all, and you can do with trivial ack / resend strategy (DNS is the obvious example), or just best-effort transfers where packet loss is not an issue at all (think a thermometer sending readings every second). In those cases the TCP three way handshake would be an overkill, and the benefits of TCP are minimal because of the use case.

_gsin 12y ago
> Instead UDP is the way to go when you don't need a transport layer at all, and you can do with trivial ack / resend strategy (DNS is the obvious example), or just best-effort transfers where packet loss is not an issue at all (think a thermometer sending readings every second) That depends I guess. As somebody who works mostly with cloud telephony, SIP (signaling) and RTP (audio) are almost always carried over UDP. Funny thing is that you'll see more occurrences of one-way audio or call drops because of people implementing the application level protocol like SIP wrong rather than the transport layer letting you down. That said, I would any day prefer a private MPLS network rather than the public internet. Though I can understand why somebody who writes a fast nosql database prefers TCP. Also, thanks for redis, we use it a lot and love it! :)
- colanderman 12y ago
  That said, I would any day prefer a private MPLS network rather than the public internet. Thanks to your comment, a short Wikipedia trip later, I now know that "penultimate hop popping" is a thing and has a fantastic name.
- MichaelGG 12y ago
  SIP signalling over UDP was a bad idea, especially with the mandatory TCP failover. SIP's actually a spectacular clusterfuck of a design overall anyways. It's actually impossible to implement SIP in an unambiguous manner on today's networks because even the parsing rules are so bad, that multiple, popular, stacks have incompatible ways of dealing with things. UDP for RTP makes sense because transmitting packets is pointless. In case of packetloss, the end user has to interpolate the result either via software or in their head. This private MPLS network... it wouldn't happen to run over the exact same equipment that'd be handling your IP traffic, would it?
  - bradleyland 12y ago
    Just a small correction for anyone else following along. This: > UDP for RTP makes sense because transmitting packets is pointless Should be this: > UDP for RTP makes sense because retransmitting packets is pointless TCP will retransmit if no ACK is received; UDP won't (there being no ACK and all).

paraboul 12y ago

Right. One usually want to reimplement a reliable protocol (e.g. similar to TCP) on top of UDP when dealing with peer-to-peer capabilities. Indeed, NAT traversal is easier to implement and more reliable when dealing with UDP techniques (e.g. UDP hole punching).

jacquesm 12y ago

> UDP is just about as reliable or unreliable as IP. Ok. > It's a shim on top of IP to let unprivileged users send IP datagrams that can be multiplexed back to the right user. (Hence the name: "user" datagram protocol.) It's mostly a way to send data in such a manner that you don't need to do the full 'call-setup-data-transmission-and-terminate' that would be required for other virtual circuit based protocols such as TCP when you don't need all that luxury. So for protocols that carry small amounts of data in a manner where retrying is not a problem and where loss of a packet is not an immediate disaster. It's also more suitable for real-time applications because of this than TCP (especially true for the first packet). Because of the fact that there is no virtual circuit a single listener can handle data from multiple senders. The 'USER' does not refer to unprivileged users but simply to users as opposed to system packets (such as for instance ICMP and other datagram like packets that are not usually sent out directly by applications). So it's not a privilege matter but a matter of user-space vs system modules elsewhere in the stack. > Lots of people talk about a "TCP or UDP" design decision, but that's usually not the relevant question for an application developer. It absolutely is. > Most UDP applications wouldn't just blat their data directly into UDP datagrams, They usually do exactly that. > just like almost nobody blats their data directly into IP datagrams. You're comparing apples with oranges, IP is one layer and TCP and UDP are on another. So you'd have to compare UDP with TCP and then you're back to that design decision again. > Typical applications generally want some sort of transport protocol on top of the datagrams (whether UDP-in-IP or just plain IP), that cares about reliability and often about fairness. Fairness is something that is usually not under control of the endpoints of a conversation but something that routers in between influence. They can decide to let a packet through or drop it (this goes for TCP as well as UDP), if a line is congested your UDP packets will usually (rules can be set to configure this) be dropped before your TCP packets will be in spite of the fact that TCP will re-try any lost packets. UDP packets can also be duplicated and routed in such a way that they arrive out-of-order. > That could be TCP-over-IP, but some (like MixApp, which wanted TCP but not the kernel's default implementation) use TCP-over-UDP-over-IP, BitTorrent uses LEDBAT-over-UDP, Mosh uses SSP-over-UDP, Chrome uses QUIC-over-UDP and falls back to SPDY-over-TCP and HTTP-over-TCP, etc. Running alternative protocols packaged inside other protocols is a time honored practice. See also: TCP over carrier pigeons and tunneling HTTP over DNS traffic (effectively using UDP). This is not in any way special, it's just a means to an end. > The idea that using UDP somewhere in the stack means "designing and implementing a reliable transport protocol from scratch" is silly. It doesn't mean that any more than using IP does. It actually comes down to exactly that. If you use UDP as your base and your application requires reliable transmission of data then you're going to have to deal with loss/duplication/sequencing at some other point in your application or put another (pre existing) protocol on top of it in order to mitigate these. If your application can tolerate those errors (or if they are not considered errors) then a naive implementation will do. > The question of SOCK_STREAM vs. SOCK_DGRAM is more typically, "Does the application want to use the one transport protocol that the kernel implements, namely a particular flavor of TCP, or does it want to use some other transport protocol implemented in a userspace library?" TCP is the default for anything requiring virtual circuits if you have demands that are not well describe by that model and/or need real time, low overhead and you're willing to do the work required to deal with UDPs inherent issues (if those are a problem) then you're totally free to do so. But the question is usually not 'do I need TCP', it usually is 'how do I avoid re-implementing TCP if I need its features'. It's a tough choice because at a minimum it means that you're going to have to write software for both endpoints. This is one of the reasons why we see HTTP over TCP in so many places where it wasn't originally intended: it is more or less guaranteed to be well tested and there are tons of tools available to use this protocol combination, especially browsers, fetchers and servers in all kinds of flavors. For UDP that situation is much less rosy and using UDP always translates into having to do a bunch of plumbing yourself.