Ethernet switches are actually pretty complex things, when you think about it. They have to learn what MAC addresses are behind each port, and build a complex

Ethernet switches are actually pretty complex things, when you think about it. They have to learn what MAC addresses are behind each port, and build a complex forwarding table and do table lookups in real time. The larger the switch, the more complex it is. Its hard to make it scale. Around the same era, Myrinet switches with higher bandwidth (1.2Gb/s if I remember correctly) and higher density at a fraction of the port cost of slower ethernet switches. This was possible because the Myrinet switches were dumb.. The Myrinet network elected a "mapper" that distributed routes to all NICs. The NICs then pre-pended routing flits to the front of each packet. So to forward a packet to its destination, all a Myrinet switch had to do was strip off and then read the first flit, see that it said "exit this hop on port 7", and then forward it on to the next switch. Higher densities were achieved with multiple chips inside the cabinet. In the mid 2000s we even built one of what was, at the time, one of the worlds largest ethernet switches using (newer, faster Myrinet) internally, and encapsulating the ethernet traffic inside Myrinet. That product died due to pressure from folks that were our partners, but felt threatened by our incredibly inexpensive high density switches. https://www.networkworld.com/article/2323306/myricom-rolls-out-switch-for-large-clusters.html https://www.networkworld.com/article/2323306/myricom-rolls-o... EDIT: fixed routing flit description, added link to PR