3 ms·

It sounds like he was insinuating that a BGP based Solution scales better than a VLAN/Spanning Tree Solution when your infrastructure reaches an extremely large

It sounds like he was insinuating that a BGP based Solution scales better than a VLAN/Spanning Tree Solution when your infrastructure reaches an extremely large capacity. I'm not sure exactly what he's referring to, but I would guess that it has something to do with either the 12-bit VLAN ID limitation (802.1q maximum theoretical limit of 4096 VLAN ID's), or more likely a limitation relating to the MAC Address Table and/or a Solution that relies on MAC Addresses (for example, 802.1d spanning tree). I believe there are new approaches to overcome these limitations (for example, 802.1ab double tagging or VMWare's VXLAN), but I guess they just decided to skip Layer 2 entirely. I've never worked in an environment where BGP is used extensively internally, but I guess from a conceptual standpoint it could be done with a good ibgp/ebgp strategy and use of techniques like Route Reflectors and/or Confederations. Going with the Layer 3 approach also seems to match well with their vision of the Open Compute Network, where the Networking Appliances are viewed more like Virtual Appliances. Very cool stuff. Wish I worked there, way more fun that the stuff I'm working with.

jimB0 13y ago
Here are some things: One scale problem with layer 2 networks is you have to go to layer 3 at some point and you have an aggregation node(s) that have to do a considerable amount of ARP request/reply traffic. The request/reply traffic is handled by the CPU of the machine processing. This can run the CPU hotter than you really want and if the aggregation nodes are control plane policing ARP (most sw vendors do), they will intermittently drop arp requests. Once the request/reply is processed, the node pushes the ARP entry into hardware. ARP hardware tables are limited resources as well. If you exceed them, traffic will either be black-holed or forwarded by the CPU of the node. Both of these are undesirable. Additionally, it becomes difficult to scale forwarding capacity of layer 2 spanning tree (or loop free) based networks to greater than 2 aggregation nodes since you will either block ports or create loops.

jimB0 13y ago

Here are some things: One scale problem with layer 2 networks is you have to go to layer 3 at some point and you have an aggregation node(s) that have to do a considerable amount of ARP request/reply traffic. The request/reply traffic is handled by the CPU of the machine processing. This can run the CPU hotter than you really want and if the aggregation nodes are control plane policing ARP (most sw vendors do), they will intermittently drop arp requests. Once the request/reply is processed, the node pushes the ARP entry into hardware. ARP hardware tables are limited resources as well. If you exceed them, traffic will either be black-holed or forwarded by the CPU of the node. Both of these are undesirable. Additionally, it becomes difficult to scale forwarding capacity of layer 2 spanning tree (or loop free) based networks to greater than 2 aggregation nodes since you will either block ports or create loops.