4 ms·

Truncation for a dedicated probe packet type: you lose the information it's a probe when you go through a tunnel of some sort (VPN, L2TP, IPsec, MPLS, VPLS, VXL

Truncation for a dedicated probe packet type: you lose the information it's a probe when you go through a tunnel of some sort (VPN, L2TP, IPsec, MPLS, VPLS, VXLAN, PBB, q-in-q, whatever). You're also dealing with different layers e.g. a client could send an L3 packet probe and now you're expecting a layer 2 PBB/q-in-q node to recognize IP packet types and treat them specially (layering violation). Truncation for all packet types: data in transit can occasionally get split for other reasons. Right now that's just made into loss, if we had built every protocol layer on the idea it should forward anyways then any instances of this type of loss also become MTU renegotiations, at best. At worst we're having to forward generally corrupted packets which can cause all sorts of other problems. It'd be another layering violation to require that e.g. an L2 switch must adjust the UDP checksum when it's intentionally truncating a packet, but that'd be the only way to avoid that. Tunnels (particularly secure) are also tricky here (you need to run multiple separate layers of this continuously to avoid truncation information not propagating to the right endpoints). It also doesn't allow for truly unidirectional protocols e.g. a UDP video stream as there is no allowance for out of session signaling to be possible. The above is for "if we have started networking day 1 with this plan in mind". There are of course additional problems given we didn't. I'm also not sure I follow how allowing any intermediate node to truncate a packet is any more authenticated. The (still ugly) beauty of using PMTUD-style approach over truncation or probe+notification is it doesn't try to make assumptions about how anything in the middle could ever work for the rest of time, and that makes it both simple (despite sounding like a lot of work) and reliable. You and your peer just exchange packets until you find the biggest size that fits (or that you care to check for) and you're off! MTU changes due to a path change? No problem, it's just part of your "I had a connection and the other side seems to have stopped responding. How do I attempt to continue" logic (be that retry a new session or attempt to be smart about it). It also plays nice with the ICMP too large messages - if they are there you can choose to listen, if they are not it still "just works". Or, like the article says, safe minimums can be more practical.

Veserv 2y ago
You truncate for all packet types. Data in transit is almost never split for reasons other than fragmentation to avoid MTU problems. Any such split necessarily defines a fragmentation and reconstruction protocol so it still "preserves" the original send length information needed for truncation detection. If they have gone truly crazy and implemented a entire stream protocol transparently backing their flows then their transparent inner point-to-point layer would need to be aware of truncation in much the same way it would need to be aware of MTU limits anyways. Forwarding generally corrupted packets should not be a problem unless your middleboxes are aggressively engaging in layering violations. From the perspective of a middlebox that is not engaging in layering violations you just have headers with blobs of data. Truncating the blob of data is basically uninteresting; at most you recalculate your integrity tags at your appropriate layer. You do not and should not recompute anything at higher layers. Furthermore, your endpoints must already be robust to blobs of garbage that pass your integrity tag checking because it is trivial for malicious actors to send you blobs of garbage with correctly calculated integrity tags. And, even if you were fully isolated, you can still get correlated bit errors that result in a correct integrity tag despite payload bit errors. Every client implementation that is not grossly incompetent must already be robust to getting garbage. You only get problems when your middleboxes start mucking around and trying to be too smart and violating your point-point transport abstraction. You still get unidirectional protocols because you should manage truncation information out-of-band of any of your protocols. UDP or any other protocol should not communicate back to the sender that truncation happened. You do that some other way or even do not bother to do it at all. This is extra channel information that you can choose to communicate to let the other endpoint know about channel properties to make better data encoding decisions. You can transmit that in-band, out-of-band, on a different protocol, whatever. This is a higher level property of the communication channel between you and the other side. Truncation is better authenticated because the packet reaches the other, known, authenticated endpoint who is the entity who can inform you, over a authenticated channel, that the transport channel has problems. You do not get nonsense like ICMP too large messages which come from unknown, unauthenticated entities. Furthermore, truncated messages can still be authenticated as long as you authentication tag the base header which should never be in the truncated section (you still need to have a minimum MTU below which you should always reject, but that number is small and much smaller than existing MTUs).
- zamadatix 2y ago
  > Data in transit is almost never split for reasons other than fragmentation to avoid MTU problems Fragmentation is a specific (unrelated) term, it's not interchangeable with a split. You can have (depending on the protocols involved): - A runt due to a collision - A link drop during transmit - A problem during cut-through type transport You can do various things to combat some of these (such as fragment-free instead of cut-through in collision domains) but you can't guarantee every phy IP ends up riding over can or should avoid these constraints. > Forwarding generally corrupted packets should not be a problem unless your middleboxes are aggressively engaging in layering violations. From the perspective of a middlebox that is not engaging in layering violations you just have headers with blobs of data. If "delivery of something somewhere" is your only definition of a problem, perhaps :p. > Furthermore, your endpoints must already be robust to blobs of garbage that pass your integrity tag checking because it is trivial for malicious actors to send you blobs of garbage with correctly calculated integrity tags. Not only the endpoints to garbage in the data payloads but equally the gear to garbage in the network headers. Be it full authentication or just error detection, you don't want to just forward things with a corrupted network header and hope it doesn't cause an issue or security violation. Things like CRCs or HMACs are done per layer precisely for this kind of reason, going to truncation requires dropping that safe handling. > Every client implementation As a side note: the concerns have less to do with the clients, they have full context and control of their sessions in software land with little concerns from concerns in being the physical transport layer. Most all of these considerations need to be thought from the intermediate boxes doing the transport/truncation instead. > You still get unidirectional protocols because you should manage truncation information out-of-band of any of your protocols Unidirectional protocols cannot be expected to punt directionality to a separate session. In general, any time the answer to a network conundrum (such as the two generals) sounds as easy as "just move that to a separate channel which has the information" you have either duplicated the problem in that channel or added functionality which might not be physically available (or directionally available for security use case reasons, or scalably available for multicast, or something else for a use case that isn't 'inside out' from what might pop in mind as a 'standard' session). > Truncation is better authenticated because the packet reaches the other, known, authenticated endpoint who is the entity who can inform you, over a authenticated channel, that the transport channel has problems. I'm still not sure I follow - how is the message between endpoints still authenticated if middleboxes can modify the bytes, breaking an HMAC and/or CRC (if any), and it still gets delivered? Having authenticated an endpoint exists at an address you've sent a packet to before does not automatically authenticate any packet which arrives. You also skipped over any of the implications for network tunnels (secure/insecure) - is MTU discovery just not supposed to work in those use cases? I think you can absolutely make a domain specific protocol which is happy to use truncation for MTU discovery, I just don't think anything which is supposed to be as universally usable as IP can.
ay 2y ago
From the pragmatic standpoint: manually hard coding a safe minimum is the only approach which consistently works. PMTUD somehow missed that packet networks ditching the OOB mechanisms of circuit switched networks was a good thing. By adding an OOB mechanism of attempted MTU discovery. Unauthenticated. Yes, matching the 5-tuple from the original payload somewhat helps against the obvious security problem with this. (It was a fun 3-4 years while it was being added to systems across the ‘net while everyone was blocking the ICMP outright to avoid the exploitation. The burps of that one might still find in some security guidelines) But the number of the network admins who understand what do they have to configure in their ACLs and why, is scarily small compared to the overall pool size. Here’s another hurdle: for about two decades, to generate ICMP you have to punt the packet from hardware forwarding to the slow path. Which gets rate-limited. Which gives one a fantastic way to create extremely entertaining and hard to debug problems: a single misbehaving or malicious flow can disable the ICMP generation for everyone else. Make hardware that can do it in fast path ? Even if you don’t punt - you still have to rate-limit to prevent the unauthenticated amplification attack (28 bytes of added headers is not comparable with some of the DNS or NTP scenarios, but not great anyway) So - practically speaking, it can’t be relied on, other than a source for great stories. PLPMTUD is a little better, in a sense that it attempts to limit itself to inband probes, but then there is the delicate dance of loss customarily being used to signal the congestion. So this mechanism isn’t too reliable either, in very painful ways for the poor soul on call dealing with the outcomes. Ask me how I know.. ;-) Now, let’s add to this the extremely pragmatic and evil hack that is the TCP MSS clamping, coming back from the first PPPoE days; which makes just enough of the eyeball traffic work to make this a “small problem with unimportant traffic that no one cares for anyway”. So yes, safe minimums are a practical solution. Until one start to build the tunnels, that is. A wireguard tunnel inside IPSec tunnel. Because policy. Inside VXLAN tunnel inside another IPSec tunnel, because SD-WAN. Which traverses NAT64, because transition and address scarcity. At which point the previously safe minimums might not be safe anymore and we are back to square 1. I suspect when folks will start running QUIC over wireguard/ipsec/vxlan + IPv6 en masse we will learn that (surprise!) 1200 was not a safe value after all. So, with this in mind, I posit it’s nice to attempt to at least fantasize about the universe where MTU determination would be done entirely inline, even if hypothetical - if we had the benefit of today’s hindsight and could time travel - could we have made it better ? P.s. unidirectional protocols could be taken care of by fountain codes not unlike the I-, P- and B- frames in video world, with similar trade offs, moreover, I feel the unequal probability of loss depending on a place in the packet might allow for some interesting tricks.

Veserv 2y ago

You truncate for all packet types. Data in transit is almost never split for reasons other than fragmentation to avoid MTU problems. Any such split necessarily defines a fragmentation and reconstruction protocol so it still "preserves" the original send length information needed for truncation detection. If they have gone truly crazy and implemented a entire stream protocol transparently backing their flows then their transparent inner point-to-point layer would need to be aware of truncation in much the same way it would need to be aware of MTU limits anyways. Forwarding generally corrupted packets should not be a problem unless your middleboxes are aggressively engaging in layering violations. From the perspective of a middlebox that is not engaging in layering violations you just have headers with blobs of data. Truncating the blob of data is basically uninteresting; at most you recalculate your integrity tags at your appropriate layer. You do not and should not recompute anything at higher layers. Furthermore, your endpoints must already be robust to blobs of garbage that pass your integrity tag checking because it is trivial for malicious actors to send you blobs of garbage with correctly calculated integrity tags. And, even if you were fully isolated, you can still get correlated bit errors that result in a correct integrity tag despite payload bit errors. Every client implementation that is not grossly incompetent must already be robust to getting garbage. You only get problems when your middleboxes start mucking around and trying to be too smart and violating your point-point transport abstraction. You still get unidirectional protocols because you should manage truncation information out-of-band of any of your protocols. UDP or any other protocol should not communicate back to the sender that truncation happened. You do that some other way or even do not bother to do it at all. This is extra channel information that you can choose to communicate to let the other endpoint know about channel properties to make better data encoding decisions. You can transmit that in-band, out-of-band, on a different protocol, whatever. This is a higher level property of the communication channel between you and the other side. Truncation is better authenticated because the packet reaches the other, known, authenticated endpoint who is the entity who can inform you, over a authenticated channel, that the transport channel has problems. You do not get nonsense like ICMP too large messages which come from unknown, unauthenticated entities. Furthermore, truncated messages can still be authenticated as long as you authentication tag the base header which should never be in the truncated section (you still need to have a minimum MTU below which you should always reject, but that number is small and much smaller than existing MTUs).

zamadatix 2y ago
> Data in transit is almost never split for reasons other than fragmentation to avoid MTU problems Fragmentation is a specific (unrelated) term, it's not interchangeable with a split. You can have (depending on the protocols involved): - A runt due to a collision - A link drop during transmit - A problem during cut-through type transport You can do various things to combat some of these (such as fragment-free instead of cut-through in collision domains) but you can't guarantee every phy IP ends up riding over can or should avoid these constraints. > Forwarding generally corrupted packets should not be a problem unless your middleboxes are aggressively engaging in layering violations. From the perspective of a middlebox that is not engaging in layering violations you just have headers with blobs of data. If "delivery of something somewhere" is your only definition of a problem, perhaps :p. > Furthermore, your endpoints must already be robust to blobs of garbage that pass your integrity tag checking because it is trivial for malicious actors to send you blobs of garbage with correctly calculated integrity tags. Not only the endpoints to garbage in the data payloads but equally the gear to garbage in the network headers. Be it full authentication or just error detection, you don't want to just forward things with a corrupted network header and hope it doesn't cause an issue or security violation. Things like CRCs or HMACs are done per layer precisely for this kind of reason, going to truncation requires dropping that safe handling. > Every client implementation As a side note: the concerns have less to do with the clients, they have full context and control of their sessions in software land with little concerns from concerns in being the physical transport layer. Most all of these considerations need to be thought from the intermediate boxes doing the transport/truncation instead. > You still get unidirectional protocols because you should manage truncation information out-of-band of any of your protocols Unidirectional protocols cannot be expected to punt directionality to a separate session. In general, any time the answer to a network conundrum (such as the two generals) sounds as easy as "just move that to a separate channel which has the information" you have either duplicated the problem in that channel or added functionality which might not be physically available (or directionally available for security use case reasons, or scalably available for multicast, or something else for a use case that isn't 'inside out' from what might pop in mind as a 'standard' session). > Truncation is better authenticated because the packet reaches the other, known, authenticated endpoint who is the entity who can inform you, over a authenticated channel, that the transport channel has problems. I'm still not sure I follow - how is the message between endpoints still authenticated if middleboxes can modify the bytes, breaking an HMAC and/or CRC (if any), and it still gets delivered? Having authenticated an endpoint exists at an address you've sent a packet to before does not automatically authenticate any packet which arrives. You also skipped over any of the implications for network tunnels (secure/insecure) - is MTU discovery just not supposed to work in those use cases? I think you can absolutely make a domain specific protocol which is happy to use truncation for MTU discovery, I just don't think anything which is supposed to be as universally usable as IP can.

ay 2y ago

From the pragmatic standpoint: manually hard coding a safe minimum is the only approach which consistently works. PMTUD somehow missed that packet networks ditching the OOB mechanisms of circuit switched networks was a good thing. By adding an OOB mechanism of attempted MTU discovery. Unauthenticated. Yes, matching the 5-tuple from the original payload somewhat helps against the obvious security problem with this. (It was a fun 3-4 years while it was being added to systems across the ‘net while everyone was blocking the ICMP outright to avoid the exploitation. The burps of that one might still find in some security guidelines) But the number of the network admins who understand what do they have to configure in their ACLs and why, is scarily small compared to the overall pool size. Here’s another hurdle: for about two decades, to generate ICMP you have to punt the packet from hardware forwarding to the slow path. Which gets rate-limited. Which gives one a fantastic way to create extremely entertaining and hard to debug problems: a single misbehaving or malicious flow can disable the ICMP generation for everyone else. Make hardware that can do it in fast path ? Even if you don’t punt - you still have to rate-limit to prevent the unauthenticated amplification attack (28 bytes of added headers is not comparable with some of the DNS or NTP scenarios, but not great anyway) So - practically speaking, it can’t be relied on, other than a source for great stories. PLPMTUD is a little better, in a sense that it attempts to limit itself to inband probes, but then there is the delicate dance of loss customarily being used to signal the congestion. So this mechanism isn’t too reliable either, in very painful ways for the poor soul on call dealing with the outcomes. Ask me how I know.. ;-) Now, let’s add to this the extremely pragmatic and evil hack that is the TCP MSS clamping, coming back from the first PPPoE days; which makes just enough of the eyeball traffic work to make this a “small problem with unimportant traffic that no one cares for anyway”. So yes, safe minimums are a practical solution. Until one start to build the tunnels, that is. A wireguard tunnel inside IPSec tunnel. Because policy. Inside VXLAN tunnel inside another IPSec tunnel, because SD-WAN. Which traverses NAT64, because transition and address scarcity. At which point the previously safe minimums might not be safe anymore and we are back to square 1. I suspect when folks will start running QUIC over wireguard/ipsec/vxlan + IPv6 en masse we will learn that (surprise!) 1200 was not a safe value after all. So, with this in mind, I posit it’s nice to attempt to at least fantasize about the universe where MTU determination would be done entirely inline, even if hypothetical - if we had the benefit of today’s hindsight and could time travel - could we have made it better ? P.s. unidirectional protocols could be taken care of by fountain codes not unlike the I-, P- and B- frames in video world, with similar trade offs, moreover, I feel the unequal probability of loss depending on a place in the packet might allow for some interesting tricks.