4 ms·

To build on your comment, although it's been years since I studied Ethernet in depth... - (On the bus thread) Ethernet started from an assumption of bad behavi

by ethbr0 4y ago

To build on your comment, although it's been years since I studied Ethernet in depth... - (On the bus thread) Ethernet started from an assumption of bad behavior (out of spec cabling, misbehaving clients, etc.) and tighten requirements just enough to construct a useful network. Much better balance between de facto ruggedness vs performance than its peers. - From the beginning, Ethernet reasoned that it was cheaper to put logic in purpose-built networking hardware than endpoints (i.e. PC network adapters). This was a better scaling tradeoff. 1x $$$ network device + 100x $ client adapter vs 1x $$ networking device + 100x $$ client adapter. - Because of the above, you started to get really cost- and data-efficient networks when the cost of Ethernet switches plummeted. (Remember, in early Ethernet days, networks were hub/broadcast-only!)

jacquesm 4y ago
I remember paying about $1000 per port for 100 megabit switches.
- ethbr0 4y ago
  Ha! But they delivered that much value (or more), so the market supported the price until supply flooded. We could do worse for a transformative technology ranking metric than "How overpriced was this when first released?" (significant look at Nvidia cards)
  - jacquesm 4y ago
    I had a bunch of workloads that quite literally got cut down to about 15% or so of the original runtime (a cluster compressing a whole archive of CDs for a large broadcaster) so I happily paid up. But still... $1000 / port!! And here I have sitting next to me a 48 port gigabit switch that cost 15% of what that 100 megabit switch cost in 1996 or so. Iirc it was one of the first D-link products I bought, it definitely wasn't great (it ran pretty hot) but it worked quite well enough. Amazing progress.
    - bombcar 4y ago
      And you can get switches for less than $25 per 10gb port now. Of course the jump from 10mb hub to 100mb switch was much larger than any of the later jumps, just because of the reduced noise.
- drewg123 4y ago
  Ethernet switches are actually pretty complex things, when you think about it. They have to learn what MAC addresses are behind each port, and build a complex forwarding table and do table lookups in real time. The larger the switch, the more complex it is. Its hard to make it scale. Around the same era, Myrinet switches with higher bandwidth (1.2Gb/s if I remember correctly) and higher density at a fraction of the port cost of slower ethernet switches. This was possible because the Myrinet switches were dumb.. The Myrinet network elected a "mapper" that distributed routes to all NICs. The NICs then pre-pended routing flits to the front of each packet. So to forward a packet to its destination, all a Myrinet switch had to do was strip off and then read the first flit, see that it said "exit this hop on port 7", and then forward it on to the next switch. Higher densities were achieved with multiple chips inside the cabinet. In the mid 2000s we even built one of what was, at the time, one of the worlds largest ethernet switches using (newer, faster Myrinet) internally, and encapsulating the ethernet traffic inside Myrinet. That product died due to pressure from folks that were our partners, but felt threatened by our incredibly inexpensive high density switches. https://www.networkworld.com/article/2323306/myricom-rolls-out-switch-for-large-clusters.html https://www.networkworld.com/article/2323306/myricom-rolls-o... EDIT: fixed routing flit description, added link to PR
  - jacquesm 4y ago
    Very interesting!
  - jabl 4y ago
    Sounds similar to Infiniband where each subnet has a subnet manager which calculates routing tables for the entire subnet, and assigns 16-bit local identifiers (LID) so you stations don't need to use the full 16 byte GUID's. Also Infiniband packets are power of two sized, making fast switching easier.
    - ethbr0 4y ago
      Neat! (Re: you and parent) At their core, most hardware evolutions seem like optimizing compute:memory:storage:functionality vs the (changing) current state of the art/economy. When Ethernet was first released, compute was expensive. Made sense to centralize compute (in routers) and make everything else dumb-tolerant. Now, compute is cheap and plentiful at network-calculating scales and throughout expectations are very high, so it makes sense to burn compute (including in clients) to simplify routing hardware.

by ethbr0 4y ago

jacquesm 4y ago

I remember paying about $1000 per port for 100 megabit switches.

ethbr0 4y ago
Ha! But they delivered that much value (or more), so the market supported the price until supply flooded. We could do worse for a transformative technology ranking metric than "How overpriced was this when first released?" (significant look at Nvidia cards)
- jacquesm 4y ago
  I had a bunch of workloads that quite literally got cut down to about 15% or so of the original runtime (a cluster compressing a whole archive of CDs for a large broadcaster) so I happily paid up. But still... $1000 / port!! And here I have sitting next to me a 48 port gigabit switch that cost 15% of what that 100 megabit switch cost in 1996 or so. Iirc it was one of the first D-link products I bought, it definitely wasn't great (it ran pretty hot) but it worked quite well enough. Amazing progress.
  - bombcar 4y ago
    And you can get switches for less than $25 per 10gb port now. Of course the jump from 10mb hub to 100mb switch was much larger than any of the later jumps, just because of the reduced noise.
drewg123 4y ago
Ethernet switches are actually pretty complex things, when you think about it. They have to learn what MAC addresses are behind each port, and build a complex forwarding table and do table lookups in real time. The larger the switch, the more complex it is. Its hard to make it scale. Around the same era, Myrinet switches with higher bandwidth (1.2Gb/s if I remember correctly) and higher density at a fraction of the port cost of slower ethernet switches. This was possible because the Myrinet switches were dumb.. The Myrinet network elected a "mapper" that distributed routes to all NICs. The NICs then pre-pended routing flits to the front of each packet. So to forward a packet to its destination, all a Myrinet switch had to do was strip off and then read the first flit, see that it said "exit this hop on port 7", and then forward it on to the next switch. Higher densities were achieved with multiple chips inside the cabinet. In the mid 2000s we even built one of what was, at the time, one of the worlds largest ethernet switches using (newer, faster Myrinet) internally, and encapsulating the ethernet traffic inside Myrinet. That product died due to pressure from folks that were our partners, but felt threatened by our incredibly inexpensive high density switches. https://www.networkworld.com/article/2323306/myricom-rolls-out-switch-for-large-clusters.html https://www.networkworld.com/article/2323306/myricom-rolls-o... EDIT: fixed routing flit description, added link to PR
- jacquesm 4y ago
  Very interesting!
- jabl 4y ago
  Sounds similar to Infiniband where each subnet has a subnet manager which calculates routing tables for the entire subnet, and assigns 16-bit local identifiers (LID) so you stations don't need to use the full 16 byte GUID's. Also Infiniband packets are power of two sized, making fast switching easier.
  - ethbr0 4y ago
    Neat! (Re: you and parent) At their core, most hardware evolutions seem like optimizing compute:memory:storage:functionality vs the (changing) current state of the art/economy. When Ethernet was first released, compute was expensive. Made sense to centralize compute (in routers) and make everything else dumb-tolerant. Now, compute is cheap and plentiful at network-calculating scales and throughout expectations are very high, so it makes sense to burn compute (including in clients) to simplify routing hardware.