3 ms·

Thanks. Why does the source host sets the Don't Fragment flag?

contingencies 11y ago
According to the original IPv4 RFC @ http://tools.ietf.org/html/rfc791 http://tools.ietf.org/html/rfc791 page #25... If the Don't Fragment flag (DF) bit is set, then internet fragmentation of this datagram is NOT permitted, although it may be discarded. This can be used to prohibit fragmentation in cases where the receiving host does not have sufficient resources to reassemble internet fragments. One example of use of the Don't Fragment feature is to down line load a small host. A small host could have a boot strap program that accepts a datagram stores it in memory and then executes it. What it seems to mean is things like PXE[1] or BOOTP[2]. Basically, DF was built to allow the sender to optionally force zero fragmentation by intermediate hosts en-route that are connected to networks with a smaller MTU than the packet size originally emitted by the sender. This was originally intended to be of use because the sender was somehow made aware of limitations in the recipient's network stack. Probably 20 or more years ago it was used for awhile as a latency-related hack for certain applications (VOIP, video, low-latency finance, certain scientific experiments generating vast amounts of data, etc.) mostly on UDP[3], though we have better methods for those now that operate through other IP headers (QoS). Theoretically, intermediate nodes could also use it as part of path selection during routing, though I have no idea if this has been done or is encouraged - eg. a packet from node A reaches node B en-route to node C. Node B has two routes to node C. The lower-cost route is available with a smaller MTU than the packet size, and a higher-cost route is available with a large enough MTU to accommodate the packet size. The DF flag could be used by routing logic at node B to automatically shuffle the packet across that higher-cost route. I believe the Path MTU discovery[4] feature of modern Linux kernels also probably uses this mechanism combined with short TTLs (ie. maximum hop counts, which also cause ICMP errors to be returned on their failure, and are the basis of traceroute[5], itself an unintended hack based on unrecognized capabilities of the IP protocol's specification) to optimize long-lived traffic flows. There must be other edge-case or optimization-related wishy-washy reasons to set it, too. The main thing to remember is: none of these were really intended by the authors of IPv4. This whole historic pile of what-if edge-case hackyness has been thrown out in favor of a better system in IPv6[6]. [1] http://en.wikipedia.org/wiki/Preboot_Execution_Environment http://en.wikipedia.org/wiki/Preboot_Execution_Environment [2] http://en.wikipedia.org/wiki/Bootstrap_Protocol http://en.wikipedia.org/wiki/Bootstrap_Protocol [3] http://en.wikipedia.org/wiki/User_Datagram_Protocol http://en.wikipedia.org/wiki/User_Datagram_Protocol [4] http://en.wikipedia.org/wiki/Path_MTU_Discovery http://en.wikipedia.org/wiki/Path_MTU_Discovery [5] http://en.wikipedia.org/wiki/Traceroute http://en.wikipedia.org/wiki/Traceroute [6] http://en.wikipedia.org/wiki/IPv6#Simplified_processing_by_routers http://en.wikipedia.org/wiki/IPv6#Simplified_processing_by_r...
- avodonosov 11y ago
  What is strange for me - the guy in the article uses web browser, ssh and other general programs. Why whould his network stack set the DF flag.
  - contingencies 11y ago
    Maybe his kernel or an intermediate router has path MTU discovery enabled?
    - avodonosov 11y ago
      I found the answer - he uses IPv6, where routers never fragment, they just drop the packet and send ICMP response.
- beneater 11y ago
  The reason is because you never want routers to have to fragment your packets ever. Fragmentation is really inefficient. So any modern stack will always set DF and listen for ICMP unreachables. In other words, PMTU discovery. None of the other reasons you mention are really relevant.
  - avodonosov 11y ago
    So, the sender uses DonfFragment as a tool to detect the optimal packet size. Alternative idea: when router needs to fragment a packet, router passes the fragmented packet through, but sends an ICMP to the source - "fragmentation happened". (Or the party which re-assembles the fragmented packet, maybe receiver, sends this "fragmentation happened" message). So this IP packet does not need to be resent, the source can optimize packet size for future, and the network doesn't break if ICMP is disabled somewhere. We don't want to constantly send these "fragmentation happened" ICMP messages (if they don't reach the source and it keeps sending large packets), so the router sends ICMP not always, but only for first 3 fragmented packets of that source in each 10 minutes. [I am just thinking, it's not a real proposal.]
caf 11y ago
It is nearly universally the case that TCP packets have the DF flag set. This is because it's most efficient to have TCP segments sized to fit into a single packet each - if part of a segment doesn't arrive, the entire segment will be resent anyway. So the DF flag is set, and the sending TCP reduces its estimate of the path-MTU whever it sees an ICMP "Fragmentation Needed" response.

According to the original IPv4 RFC @ http://tools.ietf.org/html/rfc791 http://tools.ietf.org/html/rfc791 page #25... If the Don't Fragment flag (DF) bit is set, then internet fragmentation of this datagram is NOT permitted, although it may be discarded. This can be used to prohibit fragmentation in cases where the receiving host does not have sufficient resources to reassemble internet fragments. One example of use of the Don't Fragment feature is to down line load a small host. A small host could have a boot strap program that accepts a datagram stores it in memory and then executes it. What it seems to mean is things like PXE[1] or BOOTP[2]. Basically, DF was built to allow the sender to optionally force zero fragmentation by intermediate hosts en-route that are connected to networks with a smaller MTU than the packet size originally emitted by the sender. This was originally intended to be of use because the sender was somehow made aware of limitations in the recipient's network stack. Probably 20 or more years ago it was used for awhile as a latency-related hack for certain applications (VOIP, video, low-latency finance, certain scientific experiments generating vast amounts of data, etc.) mostly on UDP[3], though we have better methods for those now that operate through other IP headers (QoS). Theoretically, intermediate nodes could also use it as part of path selection during routing, though I have no idea if this has been done or is encouraged - eg. a packet from node A reaches node B en-route to node C. Node B has two routes to node C. The lower-cost route is available with a smaller MTU than the packet size, and a higher-cost route is available with a large enough MTU to accommodate the packet size. The DF flag could be used by routing logic at node B to automatically shuffle the packet across that higher-cost route. I believe the Path MTU discovery[4] feature of modern Linux kernels also probably uses this mechanism combined with short TTLs (ie. maximum hop counts, which also cause ICMP errors to be returned on their failure, and are the basis of traceroute[5], itself an unintended hack based on unrecognized capabilities of the IP protocol's specification) to optimize long-lived traffic flows. There must be other edge-case or optimization-related wishy-washy reasons to set it, too. The main thing to remember is: none of these were really intended by the authors of IPv4. This whole historic pile of what-if edge-case hackyness has been thrown out in favor of a better system in IPv6[6]. [1] http://en.wikipedia.org/wiki/Preboot_Execution_Environment http://en.wikipedia.org/wiki/Preboot_Execution_Environment [2] http://en.wikipedia.org/wiki/Bootstrap_Protocol http://en.wikipedia.org/wiki/Bootstrap_Protocol [3] http://en.wikipedia.org/wiki/User_Datagram_Protocol http://en.wikipedia.org/wiki/User_Datagram_Protocol [4] http://en.wikipedia.org/wiki/Path_MTU_Discovery http://en.wikipedia.org/wiki/Path_MTU_Discovery [5] http://en.wikipedia.org/wiki/Traceroute http://en.wikipedia.org/wiki/Traceroute [6] http://en.wikipedia.org/wiki/IPv6#Simplified_processing_by_routers http://en.wikipedia.org/wiki/IPv6#Simplified_processing_by_r...

avodonosov 11y ago
What is strange for me - the guy in the article uses web browser, ssh and other general programs. Why whould his network stack set the DF flag.
- contingencies 11y ago
  Maybe his kernel or an intermediate router has path MTU discovery enabled?
  - avodonosov 11y ago
    I found the answer - he uses IPv6, where routers never fragment, they just drop the packet and send ICMP response.
beneater 11y ago
The reason is because you never want routers to have to fragment your packets ever. Fragmentation is really inefficient. So any modern stack will always set DF and listen for ICMP unreachables. In other words, PMTU discovery. None of the other reasons you mention are really relevant.
- avodonosov 11y ago
  So, the sender uses DonfFragment as a tool to detect the optimal packet size. Alternative idea: when router needs to fragment a packet, router passes the fragmented packet through, but sends an ICMP to the source - "fragmentation happened". (Or the party which re-assembles the fragmented packet, maybe receiver, sends this "fragmentation happened" message). So this IP packet does not need to be resent, the source can optimize packet size for future, and the network doesn't break if ICMP is disabled somewhere. We don't want to constantly send these "fragmentation happened" ICMP messages (if they don't reach the source and it keeps sending large packets), so the router sends ICMP not always, but only for first 3 fragmented packets of that source in each 10 minutes. [I am just thinking, it's not a real proposal.]

caf 11y ago

It is nearly universally the case that TCP packets have the DF flag set. This is because it's most efficient to have TCP segments sized to fit into a single packet each - if part of a segment doesn't arrive, the entire segment will be resent anyway. So the DF flag is set, and the sending TCP reduces its estimate of the path-MTU whever it sees an ICMP "Fragmentation Needed" response.