35 ms·
IPv6 Fragmentation Loss
- toast0 5y agoBased on how fragmentation is handled (generally poorly, often because there's little choice), I would have preferred truncation with an in-band signal. For TCP, truncation is a clear win; you get some of the packet, and can signal back to the other end that things are missing, and hopefully the other end adapts to stop sending packets that get truncated. (Of course, when a middle box uses large receive offload to combine the packets and then complains that they're too big to forward, it's hard to fix as an endpoint). For UDP, it's not so simple; IP fragmentation does allow for large data, all or nothing processing, without needing application level handling, but the cost of fragmentation is high. The out of band signalling when sending packets that are too large is too easy to break, and too many systems are still not setup to probe for path mtu blackholes (the biggest one for me is Android), and the workarounds are meh, too. Another option would be for IP fragments to have the protocol level header, so fragments could be grouped by the full 5-tuple (protocol, source ip, dest ip, source port, dest port) and kept if useful or dropped if not, without having to wait for all the fragments to appear.
- zamadatix 5y agoTruncation creates even more work actually. IP routers don't need to understand deeper protocols or do anything with them to fragment, simply split the IP packet into fragments and recalculate the IP checksum (and in v6 they don't even need to do that). To do this with truncation you have to know how to parse the inner protocol headers and modify things like length or checksums in those as well. You get the same issue putting the information on the fragments. Now there is no "IP layer" there is just "well we're using IP+UDP today and how that is right now should forever be baked into this hardware that will be here for 20 years" which is excatly the problem that led Google/IETF to push headers deeper with HTTP/3 to get out of that mess. You also can get an in-band signal that you're being fragmented in the middle without changing IP. E.g. TCP already negotiates an MSS, if IP fragments at the start of a group come in smaller than that you know there is something fragmenting in the middle.
- toast0 5y agoMinimum work for routers would be mark the IP packet as truncated, and adjust the IP checksum; upper level protocol can suck it. This is, of course, more work than dropping it on the floor and pretending you'll send an ICMP, but much less work than sending two (or ocassionally more) new packets. In the middle fragmentation is not really something that happens very often. IPv6 prohibited it, but in IPv4, nearly all packets are marked do not fragment, because IP routers weren't fragmenting much anyway; I think it's more likely to get an ICMP needs fragmentation packet on a too big packet with Don't Fragment, than to actually get fragmented delivery. Also, MSS is mostly not a negotiation; most stacks send what they think they can receive in the syn and the syn+ack. The only popular stack that sent min(received MSS, MSS from routing) was FreeBSD, but they changed to the common way in 12 IIRC; which in my opinion is a mistake, but I don't have enough data to show it... actually what seems best is to send back min (received MSS - X, MSS from routing), where X is 8 or 20, depending on if you more of your users are on misconfigured PPPoE or behind misconfigured IPIP tunnels.
- zamadatix 5y agoAt that point it pretty much amounts to "send a message in a raw IP header to the destination rather than a message in an IP+ICMP header to the source". The extra step of truncation of the original payload doesn't gain you anything except still doing more work on the router than v6 would have it do to gain a lot of pain for the upper layers/endpoint stacks all so the inner session doesn't have to resend the first part of the first packets of a conversation. The vast majority IPv4 traffic does not have the DNF bit set. Your logic of why they would doesn't even make sense as setting DNF only means it'll drop on routers that would fragment have fragmented not improve the situation with the ones that wouldn't have. MSS is definitely a negotiation but a negotiation just between the TCP aware nodes not along the whole IP path which is why I say endpoint stacks can use it to detect if the IP path is fragmenting by comparing incoming IP fragment sizes to the MSS.
- toast0 5y ago> The vast majority IPv4 traffic does not have the DNF bit set. I don't think that's correct. Windows, Linux (including Android), FreeBSD, macOs and (Apple) iOS all default to sending Don't Frag. Together, they form the overwhelming majority of IPv4 traffic. Nobody actually wants fragments, and very few fragments are seen in regular activity. When I ran high traffic servers, I would normally see a few fragments a minute per server, except for when we were under a chargen reflection DDoS attack; Microsoft Services for Unix sends back a hunormous UDP reply which is of course fragmented, and that caused some trouble with fragment reassembly buffers (dropping the buffer size to the minimum solved that, although during a DDoS the couple of people sending fragments would have a bad time; can't win them all). I think sometimes there's fragments in large UDP DNS replies; but it's generally best to avoid large UDP DNS replies because they often get dropped by poor decisions in network design and software, and there's probably a better way to do what you need to do. Stacks could use MSS and fragment size to do something cool, but they don't; in part, because nobody sends fragments anymore, and in part, because there's a lot of other cool stuff to do so path MTU gets left behind in a lot of places, like Android. :( Edit to add: > At that point it pretty much amounts to "send a message in a raw IP header to the destination rather than a message in an IP+ICMP header to the source". The thing is, we know that the router that's in a position to truncate the packet can probably get a packet to the destination (otherwise, it wouldn't have this packet). We don't have anything to indicate that it can get a packet to the source; and it turns out that often it can't. An in-band indication would be so much more useful than an out of band (which doesn't work), or fragmentation (which people don't want). Yes, the indication goes to the wrong party, the destination will get the information, but the source needs it; but most interactions on IPv4 are two way, so presumably the destination can tell the source that it needs to send smaller packets, again in-band with the communication it already has, that presumably works.
- anticristi 5y agoThe biggest single cause of network engineer hair loss must be MTU and fragmentation. We had some customers over a tunnel (IIRC, it was MPLS) that reduced the MTU from 1500 (default Ethernet) to 1496. This happened while our vendor changed some equipment and forgot to update either MTU to accommodate the extra MPLS header. Of course, there was a misconfigured firewall that wouldn't fragment and wouldn't send "ICMP won't fragment". The result? Most DNS queries went through. All chat applications worked. SSH generally worked, except when you started a full-screen terminal application. Smaller web pages load. Larger web pages loaded only partially. Imagine non-technical users explaining their issue. "The Internet is half-broken. Please help." God only knows how much hair I would have today if the world had figured MTU and fragmentation properly.
- iso1210 5y agoThe world did, then people came along and started blocking ICMP
- corty 5y agoYes. There is a special circle in hell for braindead firewall admins filtering ICMP. Unfortunately happens often enough that "MTU problem" is the first thing that comes to mind at the grandparents description. Next in line would be "broken DNS"...
- deleted 5y ago[deleted]
- temp667 5y agoThey are not braindead. ICMP used to be a bit broken and folks got sick of ping floods, sweeps, redirect attacks and tunneling hacks etc. Some pretty major places block ICMP as a result to keep their network reasoning and security analysis simpler.
- korethr 5y agoJust because they are major places doesn't mean they are right or such practice buys any improvement in security worth the operational cost it imposes.
- drewg123 5y agoPart of the problem with IPv6 extension headers is that there can be an unbounded number of them. No hardware designer wants to deal with that, so just dropping all of them is far easier.
- alerighi 5y agoOr we should stop thinking about processing packets with hardware. Does it still make differences these days? Designing hardware for a single purpose most of the times is not a good idea. That means that you are stuck with the implementation that was implemented in the hardware for a very long period. Also, an hardware implementation can't take into account (and handle) all particular cases, for example malformed packets. IP was designed to be flexible, you could have in theory used whatever L2 protocol you wanted, and whatever L4 protocol you wanted. Thanks to hardware implementation that considered only Ethernet as L2 protocol and TCP or UDP as possible L4 protocol anything was innovated. The reason that we are so slow to adopt IPv6, same reason. What would have take to just adopt it if we didn't have hardware implementation of IPv4? Software update and you are done. The problem is that in the world there are many not so good programmers that write inefficient code and thus people think that they need to implement things in specialized hardware to make them faster. You don't, at least in most cases.
- knorker 5y agoI could be wrong, but I would think high packet rates with large routing tables (default-free zone) would still need a TCAM.
- kazen44 5y agoAlso, hardware based offloading is a godsend for reliability aswell. You can upgrade the control plane of a router with traffic still flowing thanks to the data plane being seperate. You can even make another router do control plane for the data plane in another router. Not to mention the massive performance benefits.
- kevingadd 5y ago
- bombcar 5y agoIt seems will all "standards" there is the published standard as written, and the standard "as implemented" - and many of the unused corners quickly become "here be dragons".
- jandrese 5y agoThe article conclusion is basically that IPv6 extension headers (fragmentation is one of them) are useless on the Internet, which seems pretty reasonable to me. They're basically a research tool. The lack of on-path fragmentation in IPv6 is definitely on purpose. It was a mistake in IPv4 and would be silly to replicate in IPv6. The fragmentation header in IPv6 is effectively useless. It can only be done at the endpoints, and if that's the case the application should be doing it, not the stack. Instead IPv6 mandates path MTU discovery, which is the correct solution.
- majke 5y agoCouple of years ago I wrote a tool to check if end-hosts are complying: http://icmpcheckv6.popcount.org/ http://icmpcheckv6.popcount.org/ (v4 version http://icmpcheck.popcount.org/ http://icmpcheck.popcount.org/ ) it answers: - can fragments reach you - can PTB ICMP reach you hope it's useful. Prose: https://blog.cloudflare.com/ip-fragmentation-is-broken/ https://blog.cloudflare.com/ip-fragmentation-is-broken/ Notice: it's easy to run the tests headless with curl if you need to see if your server is configured fine. Fun fact is that it's very much not easy to accept/send fragmented packets from linux. I learned the hard way what `IP_NODEFRAG` is about.
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- 1vuio0pswjnm7 5y ago"IPv6 was intended to be "just like IPv4, but with 128-bit source and address fields". Just make IP addresses 96 bits longer and add no more magic. But it didn't turn out that way. " Where does that quoted portion come from. The mind of the OP author or someone else. If IPv6 was just a 128-bit version of IPv4, I would be an IPv6 user. As long as it continues to work, on the networks I control, I will prefer the relative simplicity of IPv6. Relative to IPv4, IPv6 is more complex.
- Dagger2 5y agov6 actually is pretty close to a wider v4. There are very few new things in it, the socket API is the same for both, and parts like addressing and routing work exactly the same. It could've been very different, but it's not. Running v6 is similar in complexity to v4, but a lot simpler than v4+NAT. Since NAT is more or less a necessity in v4 these days, v6 ends up being simpler in practice.
- xvilka 5y agoMeanwhile IPv6 adoption rate[1] slowed down, and apart from a few countries pushing the migration most simply don't care it seems. You would expect that global pandemic and going remote to speed up the change but for some reason it didn't. [1] https://www.google.com/intl/en/ipv6/statistics.html https://www.google.com/intl/en/ipv6/statistics.html
- zamadatix 5y agoThe initial rush was mostly due to mobile phone carriers, some even going as far as IPv6 only. Now that that's pretty much in place with pre 4g networks being turned off we're back to waiting on existing networks to actually convert to move the dial which has been a much slower process.
- usr1106 5y agoRegarding the half working internet it comes to my mind: We have remote Linux boxes accessible via ssh tunnels. Very rarely, but repeatedly the sudo password is not accepted. You type it extra carefully one by one, you copy-paste it, it just doesn't work. You angrily complain to the team has anybody changed the password in that box. Nobody has. Typically others say it works for them, in rare cases they can even confirm it at the same time. After a while, 10 or 20 minutes it just works again. It does not happen often, but in the meantime everyone in the team who initially claimed this cannot possibly happen has experienced it.
- unilynx 5y agoTCP doesn't lose traffic, it blocks and retransmits (until the connection itself breaks). it cannot lose just the password bytes entered to sudo and still transmit the rest so unless the connection didn't respond to anything at all after it failed to accept your password, your problem is not with the ssh tunnels or due to path/mtu issues themselves.
- usr1106 5y ago> TCP doesn't lose traffic, I know. And if it did you would see it in commands you type, too. Not just in invisible passwords. I just said this mystery came to my mind when reading the story of being able to communicate with som web servers but not others. Not trying to claim we have that same MTU problem. Actually now that I type it, I had the problem of half-broken internet myself in the past when roaming internationally with my phone. I needed to use a VPN to contact certain web servers. Need to keep this in mind when roaming becomes relevant again some day... (This happened in the UK and somewhere in the EU and affected e.g. my online banking and a Linux Foundation conference web site. I don't think censorship was the culprit.)
- danbmil99 5y agoReal-life advice: disable ipV6 whenever possible. It is nothing but a deadly cancer.
- neilalexander 5y agoWithout informed elaboration, it is incredibly difficult to take such a ridiculous comment seriously.
- danbmil99 5y agoFrom the perspective of a working programmer who wants to get things done, ipv4 does everything I need. ipv6 is nothing but a headache for no apparent advantage. It is poorly supported in the real world. I once had a situation where my internet provider appeared to support ipv6 but it was so buggy I could not successfully get apt to work. After similar problems with ssh, I routinely add "-4" to any ssh invocation. I just cannot be bothered with a protocol that adds nothing to my life but problems and headaches. I'll deal with ipv6 when I hear that ipv4 is being deprecated. As in, most likely never.
- neilalexander 5y agoFirst of all, it is not reasonable to blame the protocol for your ISP's inability to configure it properly. Secondly, I don't get this mentality that IPv6 is some totally inferior thing and I really don't get people advising others to disable IPv6 just because they don't understand it. Statements like "IPv4 does everything I need!" are, ultimately, totally missing the point. The fact is that we have hit the scale ceiling of the IPv4 address space and other cracks are showing and now the entire world needs something that will scale for the next few hundred billion devices. IPv6 is the protocol that will lift the scale ceiling higher, not just for you and your needs, but for everyone. It won't really change anything performance-wise, nor will it change how higher-layer protocols like TCP and UDP work, but that's intentional. You can hold out on principle if you like but it won't gain you anything. The world will just migrate around you eventually.
- thu2111 5y ago
- deleted 5y ago[deleted]