9 ms·

Ethernet History Deepdive – Why Do We Have Different Frame Types?

175 points by un_ess 2y ago

userbinator 2y ago
Ironically, this version of the header published in 1980 is what we still use to this day. IMHO Ethernet is one of the of great examples of backwards compatibility in the computing world. Even the wireless standards present frames to the upper layers like they're Ethernet. It's also a counterexample to the bureaucracy of standards bodies --- the standard that actually became widely used was the one that got released first. The other example that comes to mind is OSI vs DoD(TCP/IP).
- betaby 2y ago
  OSI is very widely used, most of the large ISPs use it. End consumers are just unaware of that fact. See https://en.wikipedia.org/wiki/IS-IS https://en.wikipedia.org/wiki/IS-IS
  - tialaramex 2y ago
    I think this is a stretch, like saying the X.500 Directory System is widely used based on the fact that PKIX is technically adapting X.509 and thus your TLS certificates depend on the X.500 directory system. End users aren't just "unaware" that it's actually the X.500 system, it functionally isn't the X.500 system, PKIX mandates an "alternate" scheme for the Internet and the directory called for by X.500 has never actually existed. Likewise then, IS-IS is the protocol that OSI standardized, but we're not using as part of an OSI system.
    - layer8 2y ago
      X.500 is widely used in the form of LDAP and Active Directory, however.
      - tristor 2y ago
        Active Directory is not based on X.500 and LDAP was directly created as an alternative to the DAP standard that is part of X.500. While X.500 is a precursor to both of these things, and influenced both of these things, and both of these things interoperated with X.500, they are not X.500. X.500 is for all intents and purposes pretty much dead in 2024, although I did deploy an X.500 based directory service in 2012 and it's probably still alive and running.
        p_l 2y ago
        LDAP is just "lightweight" protocol for accessing an X.500 directory service. The semantics are the same for the accessed store, IIRC. The protocol definition explicitly talks about being used to access X.500 data stores as complementary interface to DAP (which requires lower layers of OSI stack vs LDAP's raw stream of bytes)
        tristor 2y ago
        Yes, that is why it was invented, as I alluded to. LDAP was an alternative to DAP. DAP is part of the X.500 standard, LDAP is not. LDAP when it was first invented was built to access an X.500 directory. That is no longer a base requirement and most directories in the wild are not built on X.500. Standards mean things, just because LDAP was originally built to access X.500 doesn't mean it's part of X.500, and X.500 is a very detailed and specific standard which most directory services no longer follow.
        GauntletWizard 2y ago
        To reiterate in a slightly different form: Ldap is an inheritor of, and successor to, DAP/X.500, but not backwards compatible and only superficially resembling it at this point.
        lukeh 2y ago
        Ironically there's probably more X.500 remnants in Active Directory than in the UMich-derived LDAP servers, thanks to its Exchange 5.5 legacy.
  - Hikikomori 2y ago
    Would call it stretch to say it was widely used by ISPs. Some old ones may still be using integrated IS-IS as their IGP (early OSPF had scaling issues and complicated solutions for that), but that's nothing like widely using the ISO stack. But they might have used IS-IS it to route NSAP in their network at some point in time to manage ATM-era equipment, the ISP I worked still had some ctunnels for that purpose, I doubt they still have them.
    - p_l 2y ago
      It was still widely used for IPv4 and v6 in 2000s. ex telco engineer told us that two major reasons was that IS-IS was simply more efficient, and that they didn't require IP communication between routers - meaning troubleshooting and management was easier than providing p2p routes for IP-based protocols.
      - Hikikomori 2y ago
        It's still widely used, among old ISPs because why switch when it's just as good or better than the alternative and your engineers know it best. But it's not the whole stack as implied in the first reply. It was popular primarily because OSPF didn't scale well with hundreds to thousands of routers with the minimal CPU power even large core routers had.
        p_l 2y ago
        Scalability was one part. Later the fact that you didn't need separate routing setup for v6 also was part of it. But for a telco, the fact that you didn't have to setup IP connectivity in order for routers to see each other, was also a crucial ability.
        Hikikomori 2y ago
        OSPF has supported unnumbered links for a long time, don't need IP on the link with point to point and multicast.
- rjsw 2y ago
  OSI was usable as a wide area network before TCP/IP was.
  - fanf2 2y ago
    [citation needed]
    - rjsw 2y ago
      Personal experience. Was developing OSI applications in 1989, you could order an X.25 circuit from your PTT and run OSI over that using the ISODE [1] toolkit. The earliest ISP in my country didn't start until 1992. [1] https://en.wikipedia.org/wiki/ISO_Development_Environment https://en.wikipedia.org/wiki/ISO_Development_Environment
      - zik 2y ago
        TCP/IP was developed in the 1970s and adopted as the protocol standard for ARPANET (the predecessor to the Internet) in 1983.
        yusyusyus 2y ago
        and commercialization wasnt done until the like 1994. OSI as a successor was still proposed when the first BGP4 rfc came out. before commercialization happened, IP was mostly the realm of government and education.
        zik 2y ago
        I feel like your experience was quite different to mine. I used TCP/IP in the late 80s at university and doing commercial contract work. I remember OSI existing but at the places I worked it was treated as less common.
- aylons 2y ago
  > It's also a counterexample to the bureaucracy of standards bodies --- the standard that actually became widely used was the one that got released first. Sounds like a cautionary tale: whatever gets released first will stick. If you make a blunder, generations will have to live with it (like IPv4).
dale_glass 2y ago
I wish we could have another and bump the packet size. We're at the point where we can have millions of packets per second going through a network interface, and it starts to get very silly. It's at the point where even a 10G connection requires some thought to actually perform properly. I've managed to get bottlenecked on high end hardware requiring a whole detour into SR-IOV just to get back to decent speeds.
- supahfly_remix 2y ago
  > I wish we could have another and bump the packet size. The clock precision (100s of ppm) of the NIC oscillators on either side of a network connection gives a physical upper limit on the Ethernet packet size. The space between the packets lets the slower side "catch up". See https://en.wikipedia.org/wiki/Interpacket_gap https://en.wikipedia.org/wiki/Interpacket_gap for more info. We could use more precise oscillators to have longer packets but at a more expensive cost.
  - monocasa 2y ago
    You don't need that as much on modern protocols. The point of 8b/10b or 64b/66b is that it guarantees enough edges to allow receivers to be self clocking from the incoming bits being more or less thrown directly into a PLL.
    - namibj 2y ago
      That's a separate concern. The previously mentioned issue is that to never buffer packets in a reclocking repeater on a link, you _need_ the incoming packet rate to never be higher than the rate at which you can send them back out, or else you'd fill up/buffer. If your repeaters are actually switches, this manifests as whether you occasionally drop packets on a full link with uncongested switching fabric. Think two switches with a 10G port and 8 1G ports each used to compress 8 long cables into one (say, via vlan-tagging based on which of the 8 ports).
      - deleted 2y ago
        [deleted]
mannyv 2y ago
One thing that isn't mentioned is that the physical layer at the time was 'flat' ie: a network had a shared wire. That means bus arbitration (to prevent collisions) was a big deal. Token ring solved that by passing tokens, which presumably guarantees latency. I believe Ethernet just raised a line high, and it was up to everyone to respect that. Of course that changed when switches came out. I have a 10/100 hub in a closet somewhere for debugging, since it's nice to not have to remember how to get into a switch and set the monitor port. Token ring equivalents are still used in lots of places. From what I remember cable modem data is basically token ring off of channel 0 (though that may not be accurate anymore).
- akira2501 2y ago
  > I believe Ethernet just raised a line high, and it was up to everyone to respect that. It's actually much simpler than that. When you transmit you also listen. If what you hear is not what you sent, there is a collision, and you backoff.
  - FuriouslyAdrift 2y ago
    Exactly. CSMA/CD (carrier sense multiple access with collision detection)
  - jhayward 2y ago
    To be specific, it's even more basic than that. For 10Base-5 and all the coax ethernets, it was "if there's more energy on the wire than you are transmitting, a collision is present".
londons_explore 2y ago
I wish layer 2 and layer 3 were 'refactored' to force all links to be point to point, which they effectively are in the modern world. When was the last time you saw ethernet frame collisions because you used a hub not a switch? We'd get rid of the idea of a broadcast domain. We'd get rid of Mac address and ARP. Switches and routers would become the same device. We'd just use ip addresses for routing, and the 'next hop' would always be the opposite end of the link you sent a packet over. The world would be a simpler place, and no functionality would have been lost.
- FuriouslyAdrift 2y ago
  All wifi is a giant collision domain. Also, each segment of a wired network is a collision domain. What you are describing is more in line with MPLS or Infiniband. I agree with you frustration. I prefer to design networks that start routing right at the access port or even using an agent, virtual network port, or VPN endpoint at the client or application (like QUIC), but that is very expensive from a resource standpoint. IPv6 is also another way to get closer to what you are describing. In my perfect world, we'd move to something like a mashup of MPLS and HIP (https://en.wikipedia.org/wiki/Host_Identity_Protocol https://en.wikipedia.org/wiki/Host_Identity_Protocol) If want to study something more "routed" and more point to point, look at private mobile networks (5G). What we don't want is more layers of abstraction... that's making every slow, brittle and impossible to troubleshoot.
  - fsckboy 2y ago
    >Also, each segment of a wired network is a collision domain huh? where "segment" means where you are using a hub not a switch? cuz that was a long time ago
    - kbolino 2y ago
      If the links are operating at only 10Mbps or 100Mbps, it's possible for them to operate in half-duplex mode (e.g., if the cables don't have all 8 pins wired properly), even with a switch. In this mode, there's a collision domain between the host device and the switch.
miohtama 2y ago
The story is a bit like the Xkcd classic https://xkcd.com/927/ https://xkcd.com/927/ Looks like it took some years for one standard to prevail. Also TCP/IP was not clear winner in the early days.

175 points by un_ess 2y ago

Ironically, this version of the header published in 1980 is what we still use to this day. IMHO Ethernet is one of the of great examples of backwards compatibility in the computing world. Even the wireless standards present frames to the upper layers like they're Ethernet. It's also a counterexample to the bureaucracy of standards bodies --- the standard that actually became widely used was the one that got released first. The other example that comes to mind is OSI vs DoD(TCP/IP).

betaby 2y ago
OSI is very widely used, most of the large ISPs use it. End consumers are just unaware of that fact. See https://en.wikipedia.org/wiki/IS-IS https://en.wikipedia.org/wiki/IS-IS
- tialaramex 2y ago
  I think this is a stretch, like saying the X.500 Directory System is widely used based on the fact that PKIX is technically adapting X.509 and thus your TLS certificates depend on the X.500 directory system. End users aren't just "unaware" that it's actually the X.500 system, it functionally isn't the X.500 system, PKIX mandates an "alternate" scheme for the Internet and the directory called for by X.500 has never actually existed. Likewise then, IS-IS is the protocol that OSI standardized, but we're not using as part of an OSI system.
  - layer8 2y ago
    X.500 is widely used in the form of LDAP and Active Directory, however.
    - tristor 2y ago
      Active Directory is not based on X.500 and LDAP was directly created as an alternative to the DAP standard that is part of X.500. While X.500 is a precursor to both of these things, and influenced both of these things, and both of these things interoperated with X.500, they are not X.500. X.500 is for all intents and purposes pretty much dead in 2024, although I did deploy an X.500 based directory service in 2012 and it's probably still alive and running.
      - p_l 2y ago
        LDAP is just "lightweight" protocol for accessing an X.500 directory service. The semantics are the same for the accessed store, IIRC. The protocol definition explicitly talks about being used to access X.500 data stores as complementary interface to DAP (which requires lower layers of OSI stack vs LDAP's raw stream of bytes)
        tristor 2y ago
        Yes, that is why it was invented, as I alluded to. LDAP was an alternative to DAP. DAP is part of the X.500 standard, LDAP is not. LDAP when it was first invented was built to access an X.500 directory. That is no longer a base requirement and most directories in the wild are not built on X.500. Standards mean things, just because LDAP was originally built to access X.500 doesn't mean it's part of X.500, and X.500 is a very detailed and specific standard which most directory services no longer follow.
        GauntletWizard 2y ago
        To reiterate in a slightly different form: Ldap is an inheritor of, and successor to, DAP/X.500, but not backwards compatible and only superficially resembling it at this point.
      - lukeh 2y ago
        Ironically there's probably more X.500 remnants in Active Directory than in the UMich-derived LDAP servers, thanks to its Exchange 5.5 legacy.
- Hikikomori 2y ago
  Would call it stretch to say it was widely used by ISPs. Some old ones may still be using integrated IS-IS as their IGP (early OSPF had scaling issues and complicated solutions for that), but that's nothing like widely using the ISO stack. But they might have used IS-IS it to route NSAP in their network at some point in time to manage ATM-era equipment, the ISP I worked still had some ctunnels for that purpose, I doubt they still have them.
  - p_l 2y ago
    It was still widely used for IPv4 and v6 in 2000s. ex telco engineer told us that two major reasons was that IS-IS was simply more efficient, and that they didn't require IP communication between routers - meaning troubleshooting and management was easier than providing p2p routes for IP-based protocols.
    - Hikikomori 2y ago
      It's still widely used, among old ISPs because why switch when it's just as good or better than the alternative and your engineers know it best. But it's not the whole stack as implied in the first reply. It was popular primarily because OSPF didn't scale well with hundreds to thousands of routers with the minimal CPU power even large core routers had.
      - p_l 2y ago
        Scalability was one part. Later the fact that you didn't need separate routing setup for v6 also was part of it. But for a telco, the fact that you didn't have to setup IP connectivity in order for routers to see each other, was also a crucial ability.
        Hikikomori 2y ago
        OSPF has supported unnumbered links for a long time, don't need IP on the link with point to point and multicast.
rjsw 2y ago
OSI was usable as a wide area network before TCP/IP was.
- fanf2 2y ago
  [citation needed]
  - rjsw 2y ago
    Personal experience. Was developing OSI applications in 1989, you could order an X.25 circuit from your PTT and run OSI over that using the ISODE [1] toolkit. The earliest ISP in my country didn't start until 1992. [1] https://en.wikipedia.org/wiki/ISO_Development_Environment https://en.wikipedia.org/wiki/ISO_Development_Environment
    - zik 2y ago
      TCP/IP was developed in the 1970s and adopted as the protocol standard for ARPANET (the predecessor to the Internet) in 1983.
      - yusyusyus 2y ago
        and commercialization wasnt done until the like 1994. OSI as a successor was still proposed when the first BGP4 rfc came out. before commercialization happened, IP was mostly the realm of government and education.
        zik 2y ago
        I feel like your experience was quite different to mine. I used TCP/IP in the late 80s at university and doing commercial contract work. I remember OSI existing but at the places I worked it was treated as less common.
aylons 2y ago
> It's also a counterexample to the bureaucracy of standards bodies --- the standard that actually became widely used was the one that got released first. Sounds like a cautionary tale: whatever gets released first will stick. If you make a blunder, generations will have to live with it (like IPv4).

dale_glass 2y ago

I wish we could have another and bump the packet size. We're at the point where we can have millions of packets per second going through a network interface, and it starts to get very silly. It's at the point where even a 10G connection requires some thought to actually perform properly. I've managed to get bottlenecked on high end hardware requiring a whole detour into SR-IOV just to get back to decent speeds.

supahfly_remix 2y ago
> I wish we could have another and bump the packet size. The clock precision (100s of ppm) of the NIC oscillators on either side of a network connection gives a physical upper limit on the Ethernet packet size. The space between the packets lets the slower side "catch up". See https://en.wikipedia.org/wiki/Interpacket_gap https://en.wikipedia.org/wiki/Interpacket_gap for more info. We could use more precise oscillators to have longer packets but at a more expensive cost.
- monocasa 2y ago
  You don't need that as much on modern protocols. The point of 8b/10b or 64b/66b is that it guarantees enough edges to allow receivers to be self clocking from the incoming bits being more or less thrown directly into a PLL.
  - namibj 2y ago
    That's a separate concern. The previously mentioned issue is that to never buffer packets in a reclocking repeater on a link, you _need_ the incoming packet rate to never be higher than the rate at which you can send them back out, or else you'd fill up/buffer. If your repeaters are actually switches, this manifests as whether you occasionally drop packets on a full link with uncongested switching fabric. Think two switches with a 10G port and 8 1G ports each used to compress 8 long cables into one (say, via vlan-tagging based on which of the 8 ports).
    - deleted 2y ago
      [deleted]

mannyv 2y ago

One thing that isn't mentioned is that the physical layer at the time was 'flat' ie: a network had a shared wire. That means bus arbitration (to prevent collisions) was a big deal. Token ring solved that by passing tokens, which presumably guarantees latency. I believe Ethernet just raised a line high, and it was up to everyone to respect that. Of course that changed when switches came out. I have a 10/100 hub in a closet somewhere for debugging, since it's nice to not have to remember how to get into a switch and set the monitor port. Token ring equivalents are still used in lots of places. From what I remember cable modem data is basically token ring off of channel 0 (though that may not be accurate anymore).

akira2501 2y ago
> I believe Ethernet just raised a line high, and it was up to everyone to respect that. It's actually much simpler than that. When you transmit you also listen. If what you hear is not what you sent, there is a collision, and you backoff.
- FuriouslyAdrift 2y ago
  Exactly. CSMA/CD (carrier sense multiple access with collision detection)
- jhayward 2y ago
  To be specific, it's even more basic than that. For 10Base-5 and all the coax ethernets, it was "if there's more energy on the wire than you are transmitting, a collision is present".

londons_explore 2y ago

I wish layer 2 and layer 3 were 'refactored' to force all links to be point to point, which they effectively are in the modern world. When was the last time you saw ethernet frame collisions because you used a hub not a switch? We'd get rid of the idea of a broadcast domain. We'd get rid of Mac address and ARP. Switches and routers would become the same device. We'd just use ip addresses for routing, and the 'next hop' would always be the opposite end of the link you sent a packet over. The world would be a simpler place, and no functionality would have been lost.

FuriouslyAdrift 2y ago
All wifi is a giant collision domain. Also, each segment of a wired network is a collision domain. What you are describing is more in line with MPLS or Infiniband. I agree with you frustration. I prefer to design networks that start routing right at the access port or even using an agent, virtual network port, or VPN endpoint at the client or application (like QUIC), but that is very expensive from a resource standpoint. IPv6 is also another way to get closer to what you are describing. In my perfect world, we'd move to something like a mashup of MPLS and HIP (https://en.wikipedia.org/wiki/Host_Identity_Protocol https://en.wikipedia.org/wiki/Host_Identity_Protocol) If want to study something more "routed" and more point to point, look at private mobile networks (5G). What we don't want is more layers of abstraction... that's making every slow, brittle and impossible to troubleshoot.
- fsckboy 2y ago
  >Also, each segment of a wired network is a collision domain huh? where "segment" means where you are using a hub not a switch? cuz that was a long time ago
  - kbolino 2y ago
    If the links are operating at only 10Mbps or 100Mbps, it's possible for them to operate in half-duplex mode (e.g., if the cables don't have all 8 pins wired properly), even with a switch. In this mode, there's a collision domain between the host device and the switch.

miohtama 2y ago

The story is a bit like the Xkcd classic https://xkcd.com/927/ https://xkcd.com/927/ Looks like it took some years for one standard to prevail. Also TCP/IP was not clear winner in the early days.