5 ms·

I wish layer 2 and layer 3 were 'refactored' to force all links to be point to point, which they effectively are in the modern world. When was the last time

I wish layer 2 and layer 3 were 'refactored' to force all links to be point to point, which they effectively are in the modern world. When was the last time you saw ethernet frame collisions because you used a hub not a switch? We'd get rid of the idea of a broadcast domain. We'd get rid of Mac address and ARP. Switches and routers would become the same device. We'd just use ip addresses for routing, and the 'next hop' would always be the opposite end of the link you sent a packet over. The world would be a simpler place, and no functionality would have been lost.

FuriouslyAdrift 2y ago
All wifi is a giant collision domain. Also, each segment of a wired network is a collision domain. What you are describing is more in line with MPLS or Infiniband. I agree with you frustration. I prefer to design networks that start routing right at the access port or even using an agent, virtual network port, or VPN endpoint at the client or application (like QUIC), but that is very expensive from a resource standpoint. IPv6 is also another way to get closer to what you are describing. In my perfect world, we'd move to something like a mashup of MPLS and HIP (https://en.wikipedia.org/wiki/Host_Identity_Protocol https://en.wikipedia.org/wiki/Host_Identity_Protocol) If want to study something more "routed" and more point to point, look at private mobile networks (5G). What we don't want is more layers of abstraction... that's making every slow, brittle and impossible to troubleshoot.
- fsckboy 2y ago
  >Also, each segment of a wired network is a collision domain huh? where "segment" means where you are using a hub not a switch? cuz that was a long time ago
  - kbolino 2y ago
    If the links are operating at only 10Mbps or 100Mbps, it's possible for them to operate in half-duplex mode (e.g., if the cables don't have all 8 pins wired properly), even with a switch. In this mode, there's a collision domain between the host device and the switch.
- adrian_b 2y ago
  WiFi uses a different protocol than classic Ethernet, with "Collision Avoidance" instead of "Collision Detection". The reason is that one WiFi station cannot know what sources of radio interference exist at the other stations of a network, because it may hear only a part of them at its location. So all remnants of the original Ethernet could be removed from wired Ethernet, which does not need layer 2 protocols, while keeping adequate layer 2 protocols for wireless communications. Besides WiFi, there are also long-range point-to-point wireless links, where directive antennas are used at both ends. For these, there is no difference from wired links, so they do not need layer 2 protocols.
addaon 2y ago
> When was the last time you saw ethernet frame collisions because you used a hub not a switch? 10base-T1S is just beginning to ramp up in the automotive industry, which modifies the super-successful 100base-T1 to be cheaper by (a) allowing cheaper PHYs; (b) allowing cheaper endpoints due to the lower data rate to handle; (c) allowing lower-spec single twisted pair wiring; and ... (d) allowing multi-drop. This is intended to allow Ethernet to push down into the space that CAN-FD is currently occupying, and looks likely to succeed, at least in some niches.
- londons_explore 2y ago
  > 10base-T1S I think that standard is a huge mistake... 10Mbits isn't enough for a modern vehicle (no cameras, radars, screens etc). Many sensors alone can push megabits, and in the modern world engineers want to send their data json formatted not with bitfields. Instead they should have used an cdma-like design with the physical being a 2 cent microcontroller for things like bulbs and micro switches. Then, for things like cameras which require more megabits use a 30 cent microcontroller with a higher chip rate, all transmitting in the same bus and using code division to avoid needing to worry about scheduling.
  - addaon 2y ago
    The 100base-T1 and 1000base-T1 standards already exist, and are already widely adopted. 10base-T1S is intended to replace CAN, which is sub-10 Mb/s, and often ~2 Mb/s; this is a niche where cost comes first. For non-multidrop links, intermixing 1000, 100, and 10 Mbit/s ethernet links on the same switch is trivial, so each link can be independently chosen and cost-minimized; and for sensors and high bandwidth items with safety impact, multidrop is generally not the preferred approach anyway. Basically -- 10base-T1S fits a new niche (for Ethernet), where the niches you mention are already well addressed.
  - throw0101d 2y ago
    > I think that standard is a huge mistake... You have to start somewhere. They're going for 25 Gbps: > In addition to the more computer-oriented two and four-pair variants, the 10BASE-T1,[20] 100BASE-T1[21] and 1000BASE-T1[22] single-pair Ethernet (SPE) physical layers are intended for industrial and automotive applications[23] or as optional data channels in other interconnect applications.[24] The distances that single pair operates at full duplex depends on the speed: 1000m (1km) with 802.3cg-2019 10BASE-T1L; 15 m or 49 ft with 100BASE-T1 (link segment type A); up to 40 m or 130 ft using 1000BASE-T1 link segment type B with up to four in-line connectors. Both physical layers require a balanced twisted pair with an impedance of 100 Ω. The cable must be capable of transmitting 600 MHz for 1000BASE-T1 and 66 MHz for 100BASE-T1. 2.5 Gb/s, 5 Gb/s, and 10 Gb/s over a 15 m single pair is standardized in 802.3ch-2020.[25] In June 2023, 802.3cy added 25 Gb/s speeds at lengths up to 11 m.[26] * https://en.wikipedia.org/wiki/Ethernet_over_twisted_pair#Single-pair https://en.wikipedia.org/wiki/Ethernet_over_twisted_pair#Sin...
ninkendo 2y ago
A really interesting article covering this: “The world in which IPv6 was a good design” https://apenwarr.ca/log/20170810 https://apenwarr.ca/log/20170810 It talks about how when IPv6 was being designed, they wanted to do exactly that: drop most of the layer 2 stuff, abandon the idea of a bus network, make everything point-to-point, all switches would be L3 routers, etc. Search for “What if instead the world worked like this?” for the relevant part. My question though, is how would IP assignment work for each of the intermediary devices between me and (say) my ISP’s gateway? My computer is plugged into a switch right now, which is plugged into another switch, which is plugged into my router, which has a point-to-point link to the ISP gateway. Would my router get a /64, then delegate a /68 to the next “router” (ie. The physical thing I currently call a switch), which would delegate a /72 to the next one, etc? How would it determine the optimal IP allocation? What if there’s a cycle? Aren’t we sorta reinventing spanning tree at this point? (I’m genuinely curious about this, because I don’t really grok all of the implications of an “everything is L3” world like this.)
- p_l 2y ago
  For v6-specific world, scoped addresses and scoped multicast are explicitly for that purpose. You do not need to hierarchically subnet each following router, you just need to be able to express "next hop" for the subnets you need to route towards. You use link-local autoconfiguration, and use appropriately-scoped multicast addresses to ask "all-nodes" or "all-routers", making autoconfiguration a breeze compared to v4 world. In v4 world a similar setup is also possible, though specific details of the setup differ, and you have to setup addresses manually for each p2p link.
  - ninkendo 2y ago
    I mean sure, you’d definitely use a scoped address to talk to the next hop, but it still doesn’t solve how the router/switch knows which port to send the packet to for its next destination. Say I have a global unicast address on my desktop, 2 hops from my router, and I want to allow traffic to it. My router gets a packet sent to it over its link-local address, with a destination header of my desktop’s IP. Say it has 4 ports (each going to another router/switch, each with its own link-local address.) How would it know which port to use as the next hop? It would need a routing table, and that would need to be configured automatically if we want to work as well as switches do today. What would be the protocol for this auto configuration? BGP or something like it? How do the routers know the available address space? Or are we just stipulating that we’d invent a protocol for this, if it had ever happened? In ethernet we have the Spanning Tree Protocol for this, to discover the topology of an Ethernet network and know which links to use for which MAC addresses (including the ability to detect cycles.) I feel like something like spanning tree would still need to exist in an all-unicast, no-ethernet, L3-only world. Does such a thing exist already, or would we need to invent it in this counterfactual universe?
toast0 2y ago
If you force all links to be conceptually point to point, you probably make it harder to do some things. Already 1G and higher force full duplex, and 100base-TX full duplex is very common. I've still got a couple 10baseT half duplex devices though. I have redundant internet/nat routers at home (overkill!), and they communicate amongst each other to decide which is active and which isn't, but either way, the active one ARPs for the router address with 00-00-5E-00-01-01 as the mac address. The rest of the network just sends off-network packets to 00-00-5E-00-01-01, and failover happens because switches figure out what port is currently using that address. I share a different mac address for the upstream connection, which is PPPoE (sadly), but same deal --- when failover happens, the new computer starts using the address and everything figures it out, because stations are allowed to move to different ports by design.
- zamadatix 2y ago
  You can pretty much 1:1 what you describe in the redundancy case with IP, just replace the "relearn which port the MAC address associated with that IP is on" with "relearn which port the next hop for that IP is on". Things tend to get a little messier than people expect in figuring out the "what values do I use for the point to point links and how do they get assigned" step of things, though there are some clever answers there too.
db48x 2y ago
Actually, I know of one large system that heavily relies on having racks and racks of servers all located in the same broadcast domain. It makes the networking a bit more complicated, but in turn the software is a lot less complicated. It’s a decent trade–off.
MisterTea 2y ago
That sounds a lot like ATM where you called a machine and received a point to point pipe. Though you had to call first unlike Ethernet where you fling packets into the ether at will. ATM over SONET is used heavily in teleco but is on its way out in favor of OTN and Ethernet.
myrandomcomment 2y ago
Too expensive to do in an ASIC. There is a reason the MAC table is bigger than the routing table on a chip, because it is cheaper. Think of an ASIC as a box that can be divided up in to smaller boxes that are an index. The total number of boxes is limited by the size of the ASIC. The bigger chip, the more box and greater the cost. To do MAC forwarding it takes 2 boxes. To do routing it takes 5. To do an ACL match it takes 14. This is the reason OpenFlow never really worked on switches at scale. What you are asking for is someone simpler to MPLS and that is an expensive feature die size wise. I have highly oversimplified this post, but it is mostly correct at a 1000ft level.

FuriouslyAdrift 2y ago

All wifi is a giant collision domain. Also, each segment of a wired network is a collision domain. What you are describing is more in line with MPLS or Infiniband. I agree with you frustration. I prefer to design networks that start routing right at the access port or even using an agent, virtual network port, or VPN endpoint at the client or application (like QUIC), but that is very expensive from a resource standpoint. IPv6 is also another way to get closer to what you are describing. In my perfect world, we'd move to something like a mashup of MPLS and HIP (https://en.wikipedia.org/wiki/Host_Identity_Protocol https://en.wikipedia.org/wiki/Host_Identity_Protocol) If want to study something more "routed" and more point to point, look at private mobile networks (5G). What we don't want is more layers of abstraction... that's making every slow, brittle and impossible to troubleshoot.

fsckboy 2y ago
>Also, each segment of a wired network is a collision domain huh? where "segment" means where you are using a hub not a switch? cuz that was a long time ago
- kbolino 2y ago
  If the links are operating at only 10Mbps or 100Mbps, it's possible for them to operate in half-duplex mode (e.g., if the cables don't have all 8 pins wired properly), even with a switch. In this mode, there's a collision domain between the host device and the switch.
adrian_b 2y ago
WiFi uses a different protocol than classic Ethernet, with "Collision Avoidance" instead of "Collision Detection". The reason is that one WiFi station cannot know what sources of radio interference exist at the other stations of a network, because it may hear only a part of them at its location. So all remnants of the original Ethernet could be removed from wired Ethernet, which does not need layer 2 protocols, while keeping adequate layer 2 protocols for wireless communications. Besides WiFi, there are also long-range point-to-point wireless links, where directive antennas are used at both ends. For these, there is no difference from wired links, so they do not need layer 2 protocols.

addaon 2y ago

> When was the last time you saw ethernet frame collisions because you used a hub not a switch? 10base-T1S is just beginning to ramp up in the automotive industry, which modifies the super-successful 100base-T1 to be cheaper by (a) allowing cheaper PHYs; (b) allowing cheaper endpoints due to the lower data rate to handle; (c) allowing lower-spec single twisted pair wiring; and ... (d) allowing multi-drop. This is intended to allow Ethernet to push down into the space that CAN-FD is currently occupying, and looks likely to succeed, at least in some niches.

londons_explore 2y ago
> 10base-T1S I think that standard is a huge mistake... 10Mbits isn't enough for a modern vehicle (no cameras, radars, screens etc). Many sensors alone can push megabits, and in the modern world engineers want to send their data json formatted not with bitfields. Instead they should have used an cdma-like design with the physical being a 2 cent microcontroller for things like bulbs and micro switches. Then, for things like cameras which require more megabits use a 30 cent microcontroller with a higher chip rate, all transmitting in the same bus and using code division to avoid needing to worry about scheduling.
- addaon 2y ago
  The 100base-T1 and 1000base-T1 standards already exist, and are already widely adopted. 10base-T1S is intended to replace CAN, which is sub-10 Mb/s, and often ~2 Mb/s; this is a niche where cost comes first. For non-multidrop links, intermixing 1000, 100, and 10 Mbit/s ethernet links on the same switch is trivial, so each link can be independently chosen and cost-minimized; and for sensors and high bandwidth items with safety impact, multidrop is generally not the preferred approach anyway. Basically -- 10base-T1S fits a new niche (for Ethernet), where the niches you mention are already well addressed.
- throw0101d 2y ago
  > I think that standard is a huge mistake... You have to start somewhere. They're going for 25 Gbps: > In addition to the more computer-oriented two and four-pair variants, the 10BASE-T1,[20] 100BASE-T1[21] and 1000BASE-T1[22] single-pair Ethernet (SPE) physical layers are intended for industrial and automotive applications[23] or as optional data channels in other interconnect applications.[24] The distances that single pair operates at full duplex depends on the speed: 1000m (1km) with 802.3cg-2019 10BASE-T1L; 15 m or 49 ft with 100BASE-T1 (link segment type A); up to 40 m or 130 ft using 1000BASE-T1 link segment type B with up to four in-line connectors. Both physical layers require a balanced twisted pair with an impedance of 100 Ω. The cable must be capable of transmitting 600 MHz for 1000BASE-T1 and 66 MHz for 100BASE-T1. 2.5 Gb/s, 5 Gb/s, and 10 Gb/s over a 15 m single pair is standardized in 802.3ch-2020.[25] In June 2023, 802.3cy added 25 Gb/s speeds at lengths up to 11 m.[26] * https://en.wikipedia.org/wiki/Ethernet_over_twisted_pair#Single-pair https://en.wikipedia.org/wiki/Ethernet_over_twisted_pair#Sin...

ninkendo 2y ago

A really interesting article covering this: “The world in which IPv6 was a good design” https://apenwarr.ca/log/20170810 https://apenwarr.ca/log/20170810 It talks about how when IPv6 was being designed, they wanted to do exactly that: drop most of the layer 2 stuff, abandon the idea of a bus network, make everything point-to-point, all switches would be L3 routers, etc. Search for “What if instead the world worked like this?” for the relevant part. My question though, is how would IP assignment work for each of the intermediary devices between me and (say) my ISP’s gateway? My computer is plugged into a switch right now, which is plugged into another switch, which is plugged into my router, which has a point-to-point link to the ISP gateway. Would my router get a /64, then delegate a /68 to the next “router” (ie. The physical thing I currently call a switch), which would delegate a /72 to the next one, etc? How would it determine the optimal IP allocation? What if there’s a cycle? Aren’t we sorta reinventing spanning tree at this point? (I’m genuinely curious about this, because I don’t really grok all of the implications of an “everything is L3” world like this.)

p_l 2y ago
For v6-specific world, scoped addresses and scoped multicast are explicitly for that purpose. You do not need to hierarchically subnet each following router, you just need to be able to express "next hop" for the subnets you need to route towards. You use link-local autoconfiguration, and use appropriately-scoped multicast addresses to ask "all-nodes" or "all-routers", making autoconfiguration a breeze compared to v4 world. In v4 world a similar setup is also possible, though specific details of the setup differ, and you have to setup addresses manually for each p2p link.
- ninkendo 2y ago
  I mean sure, you’d definitely use a scoped address to talk to the next hop, but it still doesn’t solve how the router/switch knows which port to send the packet to for its next destination. Say I have a global unicast address on my desktop, 2 hops from my router, and I want to allow traffic to it. My router gets a packet sent to it over its link-local address, with a destination header of my desktop’s IP. Say it has 4 ports (each going to another router/switch, each with its own link-local address.) How would it know which port to use as the next hop? It would need a routing table, and that would need to be configured automatically if we want to work as well as switches do today. What would be the protocol for this auto configuration? BGP or something like it? How do the routers know the available address space? Or are we just stipulating that we’d invent a protocol for this, if it had ever happened? In ethernet we have the Spanning Tree Protocol for this, to discover the topology of an Ethernet network and know which links to use for which MAC addresses (including the ability to detect cycles.) I feel like something like spanning tree would still need to exist in an all-unicast, no-ethernet, L3-only world. Does such a thing exist already, or would we need to invent it in this counterfactual universe?

toast0 2y ago

If you force all links to be conceptually point to point, you probably make it harder to do some things. Already 1G and higher force full duplex, and 100base-TX full duplex is very common. I've still got a couple 10baseT half duplex devices though. I have redundant internet/nat routers at home (overkill!), and they communicate amongst each other to decide which is active and which isn't, but either way, the active one ARPs for the router address with 00-00-5E-00-01-01 as the mac address. The rest of the network just sends off-network packets to 00-00-5E-00-01-01, and failover happens because switches figure out what port is currently using that address. I share a different mac address for the upstream connection, which is PPPoE (sadly), but same deal --- when failover happens, the new computer starts using the address and everything figures it out, because stations are allowed to move to different ports by design.

zamadatix 2y ago
You can pretty much 1:1 what you describe in the redundancy case with IP, just replace the "relearn which port the MAC address associated with that IP is on" with "relearn which port the next hop for that IP is on". Things tend to get a little messier than people expect in figuring out the "what values do I use for the point to point links and how do they get assigned" step of things, though there are some clever answers there too.

db48x 2y ago

Actually, I know of one large system that heavily relies on having racks and racks of servers all located in the same broadcast domain. It makes the networking a bit more complicated, but in turn the software is a lot less complicated. It’s a decent trade–off.

MisterTea 2y ago

That sounds a lot like ATM where you called a machine and received a point to point pipe. Though you had to call first unlike Ethernet where you fling packets into the ether at will. ATM over SONET is used heavily in teleco but is on its way out in favor of OTN and Ethernet.

myrandomcomment 2y ago

Too expensive to do in an ASIC. There is a reason the MAC table is bigger than the routing table on a chip, because it is cheaper. Think of an ASIC as a box that can be divided up in to smaller boxes that are an index. The total number of boxes is limited by the size of the ASIC. The bigger chip, the more box and greater the cost. To do MAC forwarding it takes 2 boxes. To do routing it takes 5. To do an ACL match it takes 14. This is the reason OpenFlow never really worked on switches at scale. What you are asking for is someone simpler to MPLS and that is an expensive feature die size wise. I have highly oversimplified this post, but it is mostly correct at a 1000ft level.