5 ms·

It's only a pity that things fossilized on 1500 byte packets. Yeah, we can compensate for that with hardware, but it's ridiculous to do 100G or even 10G in 150

It's only a pity that things fossilized on 1500 byte packets. Yeah, we can compensate for that with hardware, but it's ridiculous to do 100G or even 10G in 1500 byte chunks.

NovemberWhiskey 3y ago
Jumbo frames are a thing.
- eqvinox 3y ago
  Jumbo frames are not a thing on the internet. The internet has fossilized at 1500 byte MTU.
  - NovemberWhiskey 3y ago
    Jumbo frames could be a thing on the internet if there was a meaningful value proposition. Jumbo frames get plenty of adoption in LAN environments.
    - eqvinox 3y ago
      > Jumbo frames could be a thing on the internet It being a thing on "the internet" would require it being a thing on a sizable majority of the internet. You'd need to get large network operators, peering points, user-facing ISPs, and even users themselves on board to change their setups. And Path MTU discovery is still sufficiently unreliable as to make it incredibly painful to have partial large-MTU networks. And if you do any of this with standard home customers, a hellscape torrent of user complaints is going to rain down on your support contacts. Which costs money. More money than is lost by the higher cost of routing smaller packets. So, no, jumbo frames could not be a thing on the internet. There's a reason it's called fossilization. There is no technical reason precluding changing this, it's just frozen into way too many places to be changed.
      - NovemberWhiskey 3y ago
        This the same argument why IPv6 can't be a thing on the internet; I agree that the book isn't closed entirely on that yet, but very significant progress has been made.
        eqvinox 3y ago
        Except disruptions (i.e. worse service than IPv4 only) from rolling out IPv6 are the exception while disruptions from rolling out jumbo frames are absolutely the norm.
        quickthrower2 3y ago
        They seemed to have made it work side by side. I smiled when I pinged a cloud resource and got an IPv6 the other day.
  - KaiserPro 3y ago
    > The internet has fossilized at 1500 byte MTU. the internet has a whole bunch of non ethernet stuff, a lot of which has different frame sizes. Its totally possible that backhaul is running jumbo frames, or something like it, but you'd never really know that. Conversely ADSL has odd frame sizes(inherited from ATM; 48 bytes if I remember correctly), but you don't see that because its hidden from you. Cable has a frame sizes ranging from ~500 up to 2000 bytes. Again, hidden from you. One of the joys of TCP/IP is that different frame sizes are handled for you. Sure it might be beneficial to have a frame size that marries up with packet size, it might not. you don't really know, because the internets.
    - AnimalMuppet 3y ago
      ATM used 53-byte "cells".
    - tsimionescu 3y ago
      The bottom line is that if you ever send an IP frame larger than 1500 bytes outside of your own network, it's most likely that it will never reach its destination. Especially for IPv6.
    - skullone 3y ago
      I wouldn't call them "frame sizes" for ATM or DSL. It's a bit of a transparent fragmentation into "cells" onto the "transport" layer. It could carry arbitrary "frame" sizes on that
- aswanson 3y ago
  Ill defined, but yes, definitely a thing.
  - NovemberWhiskey 3y ago
    I thought frame size negotiation was an optional part of LLDP now. I suspect it fits into the broad category of things that could be well-defined if there was broad adoption of a poorly adopted standard.
- kalleboo 3y ago
  I've been terrified to enable jumbo frames because every guide I've read has always had "you have to make sure that every single machine you ever communicate with has to also have jumbo frames on or you will cause a black hole that sucks the whole earth into it!!" kind of disclaimers when I just want to lower the overhead in copying files from my NAS that has some pathetic Atom CPU in it.
  - theblazehen 3y ago
    If you're not blocking ICMP, then PMTUD will take care of any issues where other hosts don't support jumbo frames
    - kalleboo 3y ago
      I'll try it. If that's the case it's a shame we didn't take the chance with IPv6 to push to higher default MTUs since IPv6 already relies to a much higher degree on PMTUD since it lacks fragmentation.
      - NovemberWhiskey 3y ago
        Well, the minimum MTU increased at least - it's 1280 bytes in IPv6 vs something ridiculous like 68 bytes in IPv4.
- citrin_ru 3y ago
  Never seen used in the practice though. One of problems AFAIK - most switch default to dropping jumbo frames. You can enable but if you'll left one unconfigured for some reason you will get hard to diagnose problem. The same with end hosts - they (inside a given L2 domain) should have the same MTU and if you'll left a box with the default MTU you got a problem. To make jumbo frames easier to use switches should forward them by default and hosts should accept frames large than MTA (but send MTU sized ones).
  - averageRoyalty 3y ago
    Jumbo frames are used constantly in practice, just not often on the internet. They're common in all sorts of networks, especially enterprise and storage. If your switch is configured on defaults and not running a for purpose config, you probably don't need jumbo frames.
globular-toast 3y ago
Yeah, what's with that? I looked into enabling jumbo frames on my home network (just for fun, I don't really need it) and it seemed horrendous as those packets could end up on the internet and probably wouldn't work.
- WanderPanda 3y ago
  Shouldn‘t the NAT take care of the outgoing frames as it is operating on the tcp/udp level?
  - dale_glass 3y ago
    No. NAT does Network Address Translation. It just changes addresses in packets. The packets are still whatever size they are. There's fragmentation, but that's separate from NAT and often broken.
- darkr 3y ago
  Your local router, if correctly configured should do fragmentation. But in 95% of use cases, jumbo frames aren’t worth the hassle.
  - tsimionescu 3y ago
    Not for IPv6. And for IPv4, I believe few routers actually handle IP fragmentation properly or at all (which is why this was removed from IPv6).
- NovemberWhiskey 3y ago
  This is what Path MTU Discovery is for, in principle, or what MSS clamping at your router will do.
austin-cheney 3y ago
Just to be picky the correct term is data gram, which describes layer 2 segmentation. Packets describe segmentation between switches, which is layer 3.
- cduzz 3y ago
  I'm pretty sure a switch is just an L2 bridge but still uses the ethernet address management mechanism (mac addresses) not the L3 IP (or whatever) address / routing mechanisms. Modern ethernet, on wires anyhow, is very different from the original one with a shared broadcast domain. Ironically, wireless networks are still very much like the original "you've got a piece of wire and everyone yells into it after listening for a short period of time" mechanism.
  - mcmcmc 3y ago
    L3 switches are absolutely a thing, they allow communication between different VLANs on the same hardware without needing to go through a router.
    - wolfendin 3y ago
      A layer 3 switch is just a switch and a router in the same chassis. I have layer 3 switches doing eBGP.
      - jandrese 3y ago
        We used to joke that Cisco would sell you either a switch that could route or a router that switches also.
      - cduzz 3y ago
        Well, the "OSI" model where there's a little disassembly and assembly line in your box, where each layer gets taken off by one robot and the payload of that inside package gets handed off to another concern -- all lies. The chips that do this (if they've got the features, anyhow) do L2, L3, L4, L5, etc all at the same time. So it's not an L2 switch and L3 router -- it's both at the same time, looking at the whole packet at once. But -- there is no such thing as an ethernet "router" -- it's just a bridge with a forwarding table populated (usually) by listening for MAC addresses and updating forwarding tables. "flood and learn" There are even less ethernetty things out there that use the ethernet signaling but mechanically populate the forwarding tables of switches. But a "thing that forwards between vlans" usually means "a thing that forwards packets from one L3 subnet to another." You could probably make some insane custom switch that has routing rules for forwarding mac address packets from one vlan to another based on a bunch of zany rules, but such a cursed object would be hated universally by all who come after you.

It's only a pity that things fossilized on 1500 byte packets. Yeah, we can compensate for that with hardware, but it's ridiculous to do 100G or even 10G in 1500 byte chunks.

NovemberWhiskey 3y ago

Jumbo frames are a thing.

eqvinox 3y ago
Jumbo frames are not a thing on the internet. The internet has fossilized at 1500 byte MTU.
- NovemberWhiskey 3y ago
  Jumbo frames could be a thing on the internet if there was a meaningful value proposition. Jumbo frames get plenty of adoption in LAN environments.
  - eqvinox 3y ago
    > Jumbo frames could be a thing on the internet It being a thing on "the internet" would require it being a thing on a sizable majority of the internet. You'd need to get large network operators, peering points, user-facing ISPs, and even users themselves on board to change their setups. And Path MTU discovery is still sufficiently unreliable as to make it incredibly painful to have partial large-MTU networks. And if you do any of this with standard home customers, a hellscape torrent of user complaints is going to rain down on your support contacts. Which costs money. More money than is lost by the higher cost of routing smaller packets. So, no, jumbo frames could not be a thing on the internet. There's a reason it's called fossilization. There is no technical reason precluding changing this, it's just frozen into way too many places to be changed.
    - NovemberWhiskey 3y ago
      This the same argument why IPv6 can't be a thing on the internet; I agree that the book isn't closed entirely on that yet, but very significant progress has been made.
      - eqvinox 3y ago
        Except disruptions (i.e. worse service than IPv4 only) from rolling out IPv6 are the exception while disruptions from rolling out jumbo frames are absolutely the norm.
      - quickthrower2 3y ago
        They seemed to have made it work side by side. I smiled when I pinged a cloud resource and got an IPv6 the other day.
- KaiserPro 3y ago
  > The internet has fossilized at 1500 byte MTU. the internet has a whole bunch of non ethernet stuff, a lot of which has different frame sizes. Its totally possible that backhaul is running jumbo frames, or something like it, but you'd never really know that. Conversely ADSL has odd frame sizes(inherited from ATM; 48 bytes if I remember correctly), but you don't see that because its hidden from you. Cable has a frame sizes ranging from ~500 up to 2000 bytes. Again, hidden from you. One of the joys of TCP/IP is that different frame sizes are handled for you. Sure it might be beneficial to have a frame size that marries up with packet size, it might not. you don't really know, because the internets.
  - AnimalMuppet 3y ago
    ATM used 53-byte "cells".
  - tsimionescu 3y ago
    The bottom line is that if you ever send an IP frame larger than 1500 bytes outside of your own network, it's most likely that it will never reach its destination. Especially for IPv6.
  - skullone 3y ago
    I wouldn't call them "frame sizes" for ATM or DSL. It's a bit of a transparent fragmentation into "cells" onto the "transport" layer. It could carry arbitrary "frame" sizes on that
aswanson 3y ago
Ill defined, but yes, definitely a thing.
- NovemberWhiskey 3y ago
  I thought frame size negotiation was an optional part of LLDP now. I suspect it fits into the broad category of things that could be well-defined if there was broad adoption of a poorly adopted standard.
kalleboo 3y ago
I've been terrified to enable jumbo frames because every guide I've read has always had "you have to make sure that every single machine you ever communicate with has to also have jumbo frames on or you will cause a black hole that sucks the whole earth into it!!" kind of disclaimers when I just want to lower the overhead in copying files from my NAS that has some pathetic Atom CPU in it.
- theblazehen 3y ago
  If you're not blocking ICMP, then PMTUD will take care of any issues where other hosts don't support jumbo frames
  - kalleboo 3y ago
    I'll try it. If that's the case it's a shame we didn't take the chance with IPv6 to push to higher default MTUs since IPv6 already relies to a much higher degree on PMTUD since it lacks fragmentation.
    - NovemberWhiskey 3y ago
      Well, the minimum MTU increased at least - it's 1280 bytes in IPv6 vs something ridiculous like 68 bytes in IPv4.
citrin_ru 3y ago
Never seen used in the practice though. One of problems AFAIK - most switch default to dropping jumbo frames. You can enable but if you'll left one unconfigured for some reason you will get hard to diagnose problem. The same with end hosts - they (inside a given L2 domain) should have the same MTU and if you'll left a box with the default MTU you got a problem. To make jumbo frames easier to use switches should forward them by default and hosts should accept frames large than MTA (but send MTU sized ones).
- averageRoyalty 3y ago
  Jumbo frames are used constantly in practice, just not often on the internet. They're common in all sorts of networks, especially enterprise and storage. If your switch is configured on defaults and not running a for purpose config, you probably don't need jumbo frames.

globular-toast 3y ago

Yeah, what's with that? I looked into enabling jumbo frames on my home network (just for fun, I don't really need it) and it seemed horrendous as those packets could end up on the internet and probably wouldn't work.

WanderPanda 3y ago
Shouldn‘t the NAT take care of the outgoing frames as it is operating on the tcp/udp level?
- dale_glass 3y ago
  No. NAT does Network Address Translation. It just changes addresses in packets. The packets are still whatever size they are. There's fragmentation, but that's separate from NAT and often broken.
darkr 3y ago
Your local router, if correctly configured should do fragmentation. But in 95% of use cases, jumbo frames aren’t worth the hassle.
- tsimionescu 3y ago
  Not for IPv6. And for IPv4, I believe few routers actually handle IP fragmentation properly or at all (which is why this was removed from IPv6).
NovemberWhiskey 3y ago
This is what Path MTU Discovery is for, in principle, or what MSS clamping at your router will do.

austin-cheney 3y ago

Just to be picky the correct term is data gram, which describes layer 2 segmentation. Packets describe segmentation between switches, which is layer 3.

cduzz 3y ago
I'm pretty sure a switch is just an L2 bridge but still uses the ethernet address management mechanism (mac addresses) not the L3 IP (or whatever) address / routing mechanisms. Modern ethernet, on wires anyhow, is very different from the original one with a shared broadcast domain. Ironically, wireless networks are still very much like the original "you've got a piece of wire and everyone yells into it after listening for a short period of time" mechanism.
- mcmcmc 3y ago
  L3 switches are absolutely a thing, they allow communication between different VLANs on the same hardware without needing to go through a router.
  - wolfendin 3y ago
    A layer 3 switch is just a switch and a router in the same chassis. I have layer 3 switches doing eBGP.
    - jandrese 3y ago
      We used to joke that Cisco would sell you either a switch that could route or a router that switches also.
    - cduzz 3y ago
      Well, the "OSI" model where there's a little disassembly and assembly line in your box, where each layer gets taken off by one robot and the payload of that inside package gets handed off to another concern -- all lies. The chips that do this (if they've got the features, anyhow) do L2, L3, L4, L5, etc all at the same time. So it's not an L2 switch and L3 router -- it's both at the same time, looking at the whole packet at once. But -- there is no such thing as an ethernet "router" -- it's just a bridge with a forwarding table populated (usually) by listening for MAC addresses and updating forwarding tables. "flood and learn" There are even less ethernetty things out there that use the ethernet signaling but mechanically populate the forwarding tables of switches. But a "thing that forwards between vlans" usually means "a thing that forwards packets from one L3 subnet to another." You could probably make some insane custom switch that has routing rules for forwarding mac address packets from one vlan to another based on a bunch of zany rules, but such a cursed object would be hated universally by all who come after you.