4 ms·

I'm not sure if I'd go so far as to say that apps which rely on UDP fragment reassembly are "buggy", but I definitely agree that the article title is exaggerati

by kentonv 11y ago

I'm not sure if I'd go so far as to say that apps which rely on UDP fragment reassembly are "buggy", but I definitely agree that the article title is exaggerating by calling UDP on GCE "unusable". Many UDP services (including one operated by my company, on GCE) will work just fine. Interestingly Linux will actually, by default, throw EMSGSIZE any time you try to send() a UDP datagram that is larger than the detected network MTU to the destination. As I understand it, you have to explicitly turn this behavior off to get fragmentation. http://man7.org/linux/man-pages/man7/udp.7.html http://man7.org/linux/man-pages/man7/udp.7.html

ars 11y ago
> I'm not sure if I'd go so far as to say that apps which rely on UDP fragment reassembly are "buggy" Isn't fragment reassembly essentially implying all the packets (fragments) will arrive, and will be rearranged in order? i.e. exactly what TCP does? For example imagine sending a single 1MB packet via UDP, letting it fragment and be reassembled. What distinguishes that from TCP?
- jblow 11y ago
  No. If a fragment of a fragmented UDP packet is dropped, the packet is dropped. It is simple. If you understand why UDP exists you will get it.
  - jacquesm 11y ago
    After a suitable time-out. Hm. That makes me wonder if you couldn't use a scheme such as fragment-fragment-dropped-fragment repeatedly to eat up router or IP stack resources.
    - cbsmith 11y ago
      Let's just say it wouldn't chew up nearly the kind of resources as is achievable by selectively dropping IP packets for a TCP connection.
- cbsmith 11y ago
  > Isn't fragment reassembly essentially implying all the packets (fragments) will arrive, and will be rearranged in order? i.e. exactly what TCP does? No. It is implying that it does what UDP does. TCP is different because it constructs a continuous stream. UDP is inherently packet/message based. UDP ensures that all of the IP fragments for a given UDP packet get reassembled and rearranged and order. The UDP packets themselves are not arranged in order, and it is possible to see duplicates. > For example imagine sending a single 1MB packet via UDP Not possible. You can't send more than a 65,507 byte packet via UDP[1]. Sigh... unfortunately, there are a lot of misconceptions about the basic plumbing of TCP/IP. [1] UPDATE: as was pointed out in comments, my statement was only true for IPv4. Since IP packets in IPv6 can have jumbograms larger than 65,535 packets, IPv6 UDP does support larger packets. Of course, in that case, UDP doesn't handle the fragmentation/defragmentation protocol itself, and of course GCE won't handle anything IPv6.
  - jacquesm 11y ago
    There is this thing called 'IPv6 jumbograms'. https://tools.ietf.org/html/rfc2147 https://tools.ietf.org/html/rfc2147 (Good luck with that over the public internet, but it is possible to make udp packets larger than 64K)
    - cbsmith 11y ago
      Sorry. I was presuming we were talking about about UDP fragmentation with IPv4, as this was the bug report and of course GCE doesn't support IPv6. IPv6 jumbograms are a whole different kettle of fish, as they are really handled at the IP layer, and are supposed to only be for cases where you have an MTU > 65,575 bytes. As that link describes, the UDP protocol itself doesn't do any fragmentation or reassembly of those packets. RFC 2147 is really just an update to handle the fact that IPv6 can have a single packet that is larger than an IPv4 packet.
      - jacquesm 11y ago
        It is me that should say 'sorry', I found this on a bit of documentation about GCE: "Note: Compute Engine networks only support point-to-point IPv4 traffic. Broadcast and Multicast are not supported." https://cloud.google.com/compute/docs/networking?hl=en https://cloud.google.com/compute/docs/networking?hl=en And this: https://code.google.com/p/google-compute-engine/issues/detail?id=8 https://code.google.com/p/google-compute-engine/issues/detai... So: Sorry!
- JoeAltmaier 11y ago
  UDP is 'spray and pray'. If a packet is fragmented (goes out on the wire as more than one IP frame), several things can happen. Frames can jitter (arrive with different timing); they can be reordered (e.g. packets sent 1,2,3 but arrive 2,1,3 or 3,1,2 etc) and a frame can be dropped (e.g. arrive 1,3 or 2,3). UDP has no feedback from the receiver to the source. If a packet is dropped, the IP stack on the receiver will never complain and the IP stack on the sender will never resend. SO the receiver will silently drop the whole packet. Notice it can be hard to tell if a packet is dropped, or is reordered. You receive 2,3 - maybe 1 is still coming but reordered? So detecting dropped packets is a heuristic, depending on timing. Various algorithms will use a timer to say when to give up on a packet, or drop a packet if a frame from a subsequent packet is received, or tolerate one frame from a subsequent packet (maybe the last frame of the previous packet was reordered with the first of the next) and so on. But all the work is on the receiver. The sender is not involved. Unless there is some application-layer protocol that notices packets arriving with a gap, and sends something back (that might also not arrive) like "I got packet a,b,d but missed c". Etc. Notice the application cannot resend frames; only packets because it is unaware of fragmentation/reassembly which happens in the IP stack.
zurn 11y ago
Unfortunately this relies on PMTU discovery working, which is frequently broken by misconfigured firewalls by eg blocking all ICMP traffic. Now that GCE has a more exotic broken firewall that also breaks traffic with PMTU disabled, you have a lose-lose situation.

by kentonv 11y ago

ars 11y ago

> I'm not sure if I'd go so far as to say that apps which rely on UDP fragment reassembly are "buggy" Isn't fragment reassembly essentially implying all the packets (fragments) will arrive, and will be rearranged in order? i.e. exactly what TCP does? For example imagine sending a single 1MB packet via UDP, letting it fragment and be reassembled. What distinguishes that from TCP?

jblow 11y ago
No. If a fragment of a fragmented UDP packet is dropped, the packet is dropped. It is simple. If you understand why UDP exists you will get it.
- jacquesm 11y ago
  After a suitable time-out. Hm. That makes me wonder if you couldn't use a scheme such as fragment-fragment-dropped-fragment repeatedly to eat up router or IP stack resources.
  - cbsmith 11y ago
    Let's just say it wouldn't chew up nearly the kind of resources as is achievable by selectively dropping IP packets for a TCP connection.
cbsmith 11y ago
> Isn't fragment reassembly essentially implying all the packets (fragments) will arrive, and will be rearranged in order? i.e. exactly what TCP does? No. It is implying that it does what UDP does. TCP is different because it constructs a continuous stream. UDP is inherently packet/message based. UDP ensures that all of the IP fragments for a given UDP packet get reassembled and rearranged and order. The UDP packets themselves are not arranged in order, and it is possible to see duplicates. > For example imagine sending a single 1MB packet via UDP Not possible. You can't send more than a 65,507 byte packet via UDP[1]. Sigh... unfortunately, there are a lot of misconceptions about the basic plumbing of TCP/IP. [1] UPDATE: as was pointed out in comments, my statement was only true for IPv4. Since IP packets in IPv6 can have jumbograms larger than 65,535 packets, IPv6 UDP does support larger packets. Of course, in that case, UDP doesn't handle the fragmentation/defragmentation protocol itself, and of course GCE won't handle anything IPv6.
- jacquesm 11y ago
  There is this thing called 'IPv6 jumbograms'. https://tools.ietf.org/html/rfc2147 https://tools.ietf.org/html/rfc2147 (Good luck with that over the public internet, but it is possible to make udp packets larger than 64K)
  - cbsmith 11y ago
    Sorry. I was presuming we were talking about about UDP fragmentation with IPv4, as this was the bug report and of course GCE doesn't support IPv6. IPv6 jumbograms are a whole different kettle of fish, as they are really handled at the IP layer, and are supposed to only be for cases where you have an MTU > 65,575 bytes. As that link describes, the UDP protocol itself doesn't do any fragmentation or reassembly of those packets. RFC 2147 is really just an update to handle the fact that IPv6 can have a single packet that is larger than an IPv4 packet.
    - jacquesm 11y ago
      It is me that should say 'sorry', I found this on a bit of documentation about GCE: "Note: Compute Engine networks only support point-to-point IPv4 traffic. Broadcast and Multicast are not supported." https://cloud.google.com/compute/docs/networking?hl=en https://cloud.google.com/compute/docs/networking?hl=en And this: https://code.google.com/p/google-compute-engine/issues/detail?id=8 https://code.google.com/p/google-compute-engine/issues/detai... So: Sorry!
JoeAltmaier 11y ago
UDP is 'spray and pray'. If a packet is fragmented (goes out on the wire as more than one IP frame), several things can happen. Frames can jitter (arrive with different timing); they can be reordered (e.g. packets sent 1,2,3 but arrive 2,1,3 or 3,1,2 etc) and a frame can be dropped (e.g. arrive 1,3 or 2,3). UDP has no feedback from the receiver to the source. If a packet is dropped, the IP stack on the receiver will never complain and the IP stack on the sender will never resend. SO the receiver will silently drop the whole packet. Notice it can be hard to tell if a packet is dropped, or is reordered. You receive 2,3 - maybe 1 is still coming but reordered? So detecting dropped packets is a heuristic, depending on timing. Various algorithms will use a timer to say when to give up on a packet, or drop a packet if a frame from a subsequent packet is received, or tolerate one frame from a subsequent packet (maybe the last frame of the previous packet was reordered with the first of the next) and so on. But all the work is on the receiver. The sender is not involved. Unless there is some application-layer protocol that notices packets arriving with a gap, and sends something back (that might also not arrive) like "I got packet a,b,d but missed c". Etc. Notice the application cannot resend frames; only packets because it is unaware of fragmentation/reassembly which happens in the IP stack.

zurn 11y ago

Unfortunately this relies on PMTU discovery working, which is frequently broken by misconfigured firewalls by eg blocking all ICMP traffic. Now that GCE has a more exotic broken firewall that also breaks traffic with PMTU disabled, you have a lose-lose situation.