9 ms·

MTU discovery would be so much easier if the default behavior was truncate and forward when encountering a oversized packet. The endpoints can then just compare

• ikiris 1y ago
  And how do you tell the difference between cut off packets, and a mtu drop? What about crcs / frame checks? Do you regenerate the frames? Do you do this at routed interfaces? What if there's just layer 2 only involved?
  • LegionMammal978 1y ago
    > And how do you tell the difference between cut off packets, and a mtu drop? You don't, apart from enforcing a bare-minimum MTU for sanity's sake. If your jumbo-size packets are getting randomly cut off by a middlebox, then they probably aren't stable at that size anyway.
  • Veserv 1y ago
    Packets do not get “cut-off” normally. That is kind of the point. Some protocols allow transparent fragmentation, but the fragments need to encode enough information for reconstruction, so you can still detect “less data received than encoded on send”. You do not need bit error detection because you literally truncated the packet. The data is already lost. But in the process you learned it was due to MTU limits which is very useful. Protocols are already required to be robust to garbage that fails bit error detection anyways, so it is not “required” to always have valid integrity tags. You could transparently re-encode bit error detection on the truncated packet if you so desire to ensure data integrity of the “MTU resulted in truncation” packet that you are now forwarding, but again, not necessary. Any end-to-end protocol that encodes the intended data size in-band can use this technique across truncating transport layers. And any protocol which does so already requires implementations to not blindly trust the in-band value otherwise you get trivial buffer overflows. So, all non-grossly insecure client implementations should already be able to safely handle MTU truncation if they received it (they would just not be able to use that for MTU discovery until they are updated). The only thing you need is routers to truncate instead of drop and then you can slowly update client implementations to take advantage of the new feature since this middlebox change should not break any existing implementations unless they are inexcusably insecure.
    • ikiris 1y ago
      I don’t think you understand what normally looks like if you start forwarding damaged frames like this because you can’t tell the difference. That was the point.
      • Veserv 1y ago
        I literally have no idea what you are talking about. You can send garbage packets that conform to no known protocol on the internet. You can get more bit errors or perfect bit errors that make your bit error detection pass while still forwarding corrupt payloads. Transport protocols and channels must be and are robust to this. “Damaged” frames and frame integrity only matter if you need the contents of the entire packet to remain intact. Which you explicitly do not when truncating. The only new problem that arises is that maybe the in-band length information or headers get corrupted resulting in misinterpreting the truncation that actually occurred. And again, you already need to be robust to garbage. And you can just change my proposal to recompute the integrity tag on the truncated data if you think that really matters.
        • ikiris 1y ago
          I agree you have no idea what I’m talking about
• zamadatix 1y ago
  Truncation for a dedicated probe packet type: you lose the information it's a probe when you go through a tunnel of some sort (VPN, L2TP, IPsec, MPLS, VPLS, VXLAN, PBB, q-in-q, whatever). You're also dealing with different layers e.g. a client could send an L3 packet probe and now you're expecting a layer 2 PBB/q-in-q node to recognize IP packet types and treat them specially (layering violation). Truncation for all packet types: data in transit can occasionally get split for other reasons. Right now that's just made into loss, if we had built every protocol layer on the idea it should forward anyways then any instances of this type of loss also become MTU renegotiations, at best. At worst we're having to forward generally corrupted packets which can cause all sorts of other problems. It'd be another layering violation to require that e.g. an L2 switch must adjust the UDP checksum when it's intentionally truncating a packet, but that'd be the only way to avoid that. Tunnels (particularly secure) are also tricky here (you need to run multiple separate layers of this continuously to avoid truncation information not propagating to the right endpoints). It also doesn't allow for truly unidirectional protocols e.g. a UDP video stream as there is no allowance for out of session signaling to be possible. The above is for "if we have started networking day 1 with this plan in mind". There are of course additional problems given we didn't. I'm also not sure I follow how allowing any intermediate node to truncate a packet is any more authenticated. The (still ugly) beauty of using PMTUD-style approach over truncation or probe+notification is it doesn't try to make assumptions about how anything in the middle could ever work for the rest of time, and that makes it both simple (despite sounding like a lot of work) and reliable. You and your peer just exchange packets until you find the biggest size that fits (or that you care to check for) and you're off! MTU changes due to a path change? No problem, it's just part of your "I had a connection and the other side seems to have stopped responding. How do I attempt to continue" logic (be that retry a new session or attempt to be smart about it). It also plays nice with the ICMP too large messages - if they are there you can choose to listen, if they are not it still "just works". Or, like the article says, safe minimums can be more practical.
  • Veserv 1y ago
    You truncate for all packet types. Data in transit is almost never split for reasons other than fragmentation to avoid MTU problems. Any such split necessarily defines a fragmentation and reconstruction protocol so it still "preserves" the original send length information needed for truncation detection. If they have gone truly crazy and implemented a entire stream protocol transparently backing their flows then their transparent inner point-to-point layer would need to be aware of truncation in much the same way it would need to be aware of MTU limits anyways. Forwarding generally corrupted packets should not be a problem unless your middleboxes are aggressively engaging in layering violations. From the perspective of a middlebox that is not engaging in layering violations you just have headers with blobs of data. Truncating the blob of data is basically uninteresting; at most you recalculate your integrity tags at your appropriate layer. You do not and should not recompute anything at higher layers. Furthermore, your endpoints must already be robust to blobs of garbage that pass your integrity tag checking because it is trivial for malicious actors to send you blobs of garbage with correctly calculated integrity tags. And, even if you were fully isolated, you can still get correlated bit errors that result in a correct integrity tag despite payload bit errors. Every client implementation that is not grossly incompetent must already be robust to getting garbage. You only get problems when your middleboxes start mucking around and trying to be too smart and violating your point-point transport abstraction. You still get unidirectional protocols because you should manage truncation information out-of-band of any of your protocols. UDP or any other protocol should not communicate back to the sender that truncation happened. You do that some other way or even do not bother to do it at all. This is extra channel information that you can choose to communicate to let the other endpoint know about channel properties to make better data encoding decisions. You can transmit that in-band, out-of-band, on a different protocol, whatever. This is a higher level property of the communication channel between you and the other side. Truncation is better authenticated because the packet reaches the other, known, authenticated endpoint who is the entity who can inform you, over a authenticated channel, that the transport channel has problems. You do not get nonsense like ICMP too large messages which come from unknown, unauthenticated entities. Furthermore, truncated messages can still be authenticated as long as you authentication tag the base header which should never be in the truncated section (you still need to have a minimum MTU below which you should always reject, but that number is small and much smaller than existing MTUs).
• ay 1y ago
  With IPv4, clearing the DF bit in all egress packets and hacking on top of QUIC could give just enough of a wiggle room to make it possible to explore this between a pair of cooperating hosts even in today’s Internet. Anti-DDoS middle boxes will be almost certainly unhappy with lone fragments and UDP in general, so it’s a bit of a thorny path. The big question is what to do with IPv6, since the intermediary nodes will only drop. This bit unfortunately makes the whole exercise pretty theoretical, but it can be fun nonetheless to explore. Feel free to contact me at my github userid at gmail, if this is a topic of interest.
  • zamadatix 1y ago
    Most carrier/enterprise/hardware IPv4 routers, particular those on the internet, will not actually perform IPv4 fragmentation on behalf of the client traffic even though it's allowed by the IPv4 standard. Typically fragmentation is reserved for boxes which already have another reason to care about it (such as needing to NAT or inspect the packets) or the client endpoints themselves. I.e. the internet will (sparing security middleboxes) allow arbitrary IPv4 fragments through but it won't typically turn a 8000 byte packet into 6 fragments to fit through a 1500 byte MTU limitation on behalf of the clients. E.g. if you send a 1500 byte IPv4 ping without DF set to a cellular modem or someone with a DSL modem using PPPoE it'll almost always get dropped by the carrier rather than fragmented. Of course nothing is stopping you from labbing it up at home. Firewalls and software routers can usually be made to do refragmentation.
    • ay 1y ago
      Of course on the carrier boxes the fragmentation is done also not inline, so its behavior will depend on the aggressiveness of the CoPP configuration, and will be subject to the same pitfalls as the ICMP packet too big generation. Thanks for keeping me straight here! Based on the admittedly old study at [0] seems like some carriers just don’t bother to fragment, indeed - but by far not all of them. Firewalls might do virtual reassembly, so the trick with the initial fragment won’t fly there. This MTU subject is interesting for me because I have a little work in progress experiment: https://gerrit.fd.io/r/c/vpp/+/41914/1/src/plugins/pvti/pvti.rst https://gerrit.fd.io/r/c/vpp/+/41914/1/src/plugins/pvti/pvti... (the code itself is already in, but has a few crashy bugs still and I need to take make it not suck performance wise, but that is my attempt to revisit the issue of MTU for tunnel use case. The thesis is that keeping the 5-tuple will make “chunking”/“de-chunking” at tunnel endpoints much much simpler on the endpoints of the tunnel. The source of inspiration was a very practical setup at [1], which is, while looking horrible in theory (locally fragmented GRE over L2TP), actually gives a decent performance with 1500-byte end to end MTU over the tunnel. The open question is which inner MTU will be sane, taking into account the increased probability of loss with bigger inner MTU… intuitively seems like something like ~2.5K should just double the loss probability (because it’s 2x packets) and might be a workable compromise in 2025…. One could also do the same trick over QUIC, of course, but i wanted something tiny and easier to experiment with - and the ability to go over IPSec or wireguard as well as a secured underlay. [0] https://labs.ripe.net/author/emileaben/ripe-atlas-packet-size-matters/ https://labs.ripe.net/author/emileaben/ripe-atlas-packet-siz... [1] https://github.com/ayourtch/linode-ipv6-tunnel https://github.com/ayourtch/linode-ipv6-tunnel