5 ms·

My question after reading the article is how would we go about addressing this? It seems that IPv4 (and v6 to some extent) allow for MTUs of up to 64KiB. Is th

My question after reading the article is how would we go about addressing this? It seems that IPv4 (and v6 to some extent) allow for MTUs of up to 64KiB. Is the only thing limiting the selection of higher Path MTUs the fact that most Ethernet implementations have a MTU of 1500 bytes? Do network hardware companies sell devices that tolerate higher MTUs? Is this a hardware limitation, or it it just unwillingness to go past the 1500 bytes because that's the IEEE standard?

majke 9y ago
This is a subject for another blog post. Or a book! This problem has been excessively discussed in early IP days. The alternative to current model is to have a fixed packet size. This inherently means that for some media the packet size will be not optimal. The current model in IPv4 is reasonable - that is: dynamic packet size, with either routers doing fragmentation or signaling mechanism to push back the PMTU onto the originating host. The problem is that there is very little visibility if these mechanisms work. The linked online tests are an attempt to fix this - by easily showing everyone if their CPE / ISP / Path is reasonable and behaves correctly. Testing fragmentation / ICMP PMTUD delivery historically was pretty hard. We need more visibility.
- bogomipz 9y ago
  >"The alternative to current model is to have a fixed packet size" This was already done with ATM for L2 and the 53 byte fixed cell size. You don't hear about ATM any more.
noselasd 9y ago
Most (at least most non consumer devices) supports ethernet frames of up to about 9000 bytes - aka Jumbo Frames.
bogomipz 9y ago
Sure, lost of hardware supports jumbo frames which are 9K bytes in length. Ethernet however uses a 32 bit CRC to detect bit inversion and this CRC is only reliable up to around 12K bytes.
- ajb 9y ago
  That doesn't sound right. Do you have a citation for the ethernet CRC becoming unreliable over 12K?
  - cesarb 9y ago
    I did a quick search, and found one. The page at https://web.archive.org/web/20060918065212/www.aarnet.edu.au/engineering/networkdesign/mtu/size.html https://web.archive.org/web/20060918065212/www.aarnet.edu.au... says: "The weakening of this frame check which greater frame sizes is explored in R. Jain's "Error Characteristics of Fiber Distributed Data Interface (FDDI)", which appeared in IEEE Transactions on Communications, August 1990. [...] Firstly, the power of ethernet's Frame Check Sequence is the major limitation on increasing the ethernet MTU beyond 11444 bytes. Secondly, frame sizes under 11445 bytes are as well protected by ethernet's Frame Check Sequence as frame sizes under 1518 bytes." Another page at https://web.archive.org/web/20070426062813/http://sd.wareonearth.com:80/~phil/jumbo.html https://web.archive.org/web/20070426062813/http://sd.wareone... explains why 9000 bytes is commonly chosen: "Why 9000? First because ethernet uses a 32 bit CRC that loses its effectiveness above about 12000 bytes. And secondly, 9000 was large enough to carry an 8 KB application datagram (e.g. NFS) plus packet header overhead." (The page doesn't mention it, but I guess only multiples of 1500 were considered, and 9000 is the smallest multiple of 9000 greater than 8192.) Other links: "Raising the Internet MTU": https://web.archive.org/web/20070427184111/www.psc.edu/~mathis/MTU/ https://web.archive.org/web/20070427184111/www.psc.edu/~math... "Thoughts on increasing MTUs on the internet" (NANOG thread from 2007): https://www.nanog.org/mailinglist/mailarchives/old_archive/2007-04/msg00335.html https://www.nanog.org/mailinglist/mailarchives/old_archive/2... Edit: found the source of the "9000 bytes" de facto standard: https://noc.net.internet2.edu/i2network/jumbo-frames/rrsum-almes-mtu.html https://noc.net.internet2.edu/i2network/jumbo-frames/rrsum-a...
  - bogomipz 9y ago
    Sure, see the following https://tools.ietf.org/id/draft-ietf-isis-ext-eth-01.txt https://tools.ietf.org/id/draft-ietf-isis-ext-eth-01.txt: 11. Appendix 2. Comments from the draft's authors: FDDI and Ethernet use the same error checking mechanism; CRC-32. The probability of undetected errors remains constant for frame sizes between 3007 and 91639 bits (approximately 376 to 11455 bytes). Setting the maximum size of jumbo frames to 9018 bytes falls well within this range. There is no increase in undetected errors when using jumbo frames and the existing CRC-32 as the error detection mechanism. This is also worth a read: https://tools.ietf.org/html/draft-anumita-tcpm-stronger-checksum-00 https://tools.ietf.org/html/draft-anumita-tcpm-stronger-chec... >The strength of the Ethernet CRC checksum and the 16 bitTransport checksum has been found to reduce for data segments that are largerthan the standard Ethernet MTU. Koopman et. al. [Koopman] have explored a number of CRC polynomials as well as the polynomial used in the Ethernet CRC calculation. And the following on page 32: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.4.8352&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.4.8... >"Due to the nature of the CRC algorithm, the probability of undetected errors is the same for frame sizes between 376 and 11,455 bytes. Thus to maintain the same bit error rate accuracy as standard Ethernet, frames should ideally not exceed 11455 bytes."

majke 9y ago

This is a subject for another blog post. Or a book! This problem has been excessively discussed in early IP days. The alternative to current model is to have a fixed packet size. This inherently means that for some media the packet size will be not optimal. The current model in IPv4 is reasonable - that is: dynamic packet size, with either routers doing fragmentation or signaling mechanism to push back the PMTU onto the originating host. The problem is that there is very little visibility if these mechanisms work. The linked online tests are an attempt to fix this - by easily showing everyone if their CPE / ISP / Path is reasonable and behaves correctly. Testing fragmentation / ICMP PMTUD delivery historically was pretty hard. We need more visibility.

bogomipz 9y ago
>"The alternative to current model is to have a fixed packet size" This was already done with ATM for L2 and the 53 byte fixed cell size. You don't hear about ATM any more.

noselasd 9y ago

Most (at least most non consumer devices) supports ethernet frames of up to about 9000 bytes - aka Jumbo Frames.

bogomipz 9y ago

Sure, lost of hardware supports jumbo frames which are 9K bytes in length. Ethernet however uses a 32 bit CRC to detect bit inversion and this CRC is only reliable up to around 12K bytes.

ajb 9y ago
That doesn't sound right. Do you have a citation for the ethernet CRC becoming unreliable over 12K?
- cesarb 9y ago
  I did a quick search, and found one. The page at https://web.archive.org/web/20060918065212/www.aarnet.edu.au/engineering/networkdesign/mtu/size.html https://web.archive.org/web/20060918065212/www.aarnet.edu.au... says: "The weakening of this frame check which greater frame sizes is explored in R. Jain's "Error Characteristics of Fiber Distributed Data Interface (FDDI)", which appeared in IEEE Transactions on Communications, August 1990. [...] Firstly, the power of ethernet's Frame Check Sequence is the major limitation on increasing the ethernet MTU beyond 11444 bytes. Secondly, frame sizes under 11445 bytes are as well protected by ethernet's Frame Check Sequence as frame sizes under 1518 bytes." Another page at https://web.archive.org/web/20070426062813/http://sd.wareonearth.com:80/~phil/jumbo.html https://web.archive.org/web/20070426062813/http://sd.wareone... explains why 9000 bytes is commonly chosen: "Why 9000? First because ethernet uses a 32 bit CRC that loses its effectiveness above about 12000 bytes. And secondly, 9000 was large enough to carry an 8 KB application datagram (e.g. NFS) plus packet header overhead." (The page doesn't mention it, but I guess only multiples of 1500 were considered, and 9000 is the smallest multiple of 9000 greater than 8192.) Other links: "Raising the Internet MTU": https://web.archive.org/web/20070427184111/www.psc.edu/~mathis/MTU/ https://web.archive.org/web/20070427184111/www.psc.edu/~math... "Thoughts on increasing MTUs on the internet" (NANOG thread from 2007): https://www.nanog.org/mailinglist/mailarchives/old_archive/2007-04/msg00335.html https://www.nanog.org/mailinglist/mailarchives/old_archive/2... Edit: found the source of the "9000 bytes" de facto standard: https://noc.net.internet2.edu/i2network/jumbo-frames/rrsum-almes-mtu.html https://noc.net.internet2.edu/i2network/jumbo-frames/rrsum-a...
- bogomipz 9y ago
  Sure, see the following https://tools.ietf.org/id/draft-ietf-isis-ext-eth-01.txt https://tools.ietf.org/id/draft-ietf-isis-ext-eth-01.txt: 11. Appendix 2. Comments from the draft's authors: FDDI and Ethernet use the same error checking mechanism; CRC-32. The probability of undetected errors remains constant for frame sizes between 3007 and 91639 bits (approximately 376 to 11455 bytes). Setting the maximum size of jumbo frames to 9018 bytes falls well within this range. There is no increase in undetected errors when using jumbo frames and the existing CRC-32 as the error detection mechanism. This is also worth a read: https://tools.ietf.org/html/draft-anumita-tcpm-stronger-checksum-00 https://tools.ietf.org/html/draft-anumita-tcpm-stronger-chec... >The strength of the Ethernet CRC checksum and the 16 bitTransport checksum has been found to reduce for data segments that are largerthan the standard Ethernet MTU. Koopman et. al. [Koopman] have explored a number of CRC polynomials as well as the polynomial used in the Ethernet CRC calculation. And the following on page 32: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.4.8352&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.4.8... >"Due to the nature of the CRC algorithm, the probability of undetected errors is the same for frame sizes between 376 and 11,455 bytes. Thus to maintain the same bit error rate accuracy as standard Ethernet, frames should ideally not exceed 11455 bytes."