3 ms·

> Scale: Running an all layer2 network requires devices that have large enough CAM tables to support all connected devices ... in the broadcast domain. Maybe y

> Scale: Running an all layer2 network requires devices that have large enough CAM tables to support all connected devices ... in the broadcast domain. Maybe you save on CAM, but now youre buying more TCAM/SRAM for your route table. You're going to get what, a couple thousand prefixes on your trident boxes? And burn a bunch on all the /32s and ECMP'd routes you have? I'm quite curious why more routing looked better than label based switching in a meshed network.

jimB0 13y ago
To run BGP, you would not have to run /32s everywhere. You could have multiple tiers of aggregation that keep your routing tables manageable. It would look something like this. hosts (/32) -> rack-sw -> (/26) -> aggregation-sw -> (/17) If you need some specifics to be advertised, scale would be dependent on the number of specifics that you desire to be advertised out of rack in the case above, so it would look something like this. hosts (/32) -> rack-sw -> (/26 + N * /32_external) -> aggregation-sw -> (/17) On the note about label switching versus ip forwarding, I would not think of it as either/or but rather both could be used since you need to eventually serve pages to users.
- donavanm 13y ago
  Yeah, i was specifically thinking of the prefixes jn the tor and agg layer. I read it as having all the vips in the rack as next hop /32s. Add a bunch of intra rack ecmp groups and its not insignificant. Im not sure i buy the feasability of /27 per rack or a simple /17 supernet at the agg layer long term. Rack address counts will not be uniform, id expect everything from /29s to /23s depending on host types. Assuming uniform allocation at the pod/dc level also seems like a stretch. Renumbering and super/subnetting is going to fragment the ip locality over time. Ecmping through the core eats yet more prefixes, and (Id guess) forces hot ones to be deagg'd. All that said, you do get a couple thousand prefixes on your $10,000 trident box. And im obviously not an NE. So maybe it would work inside a small cluster.

jimB0 13y ago

To run BGP, you would not have to run /32s everywhere. You could have multiple tiers of aggregation that keep your routing tables manageable. It would look something like this. hosts (/32) -> rack-sw -> (/26) -> aggregation-sw -> (/17) If you need some specifics to be advertised, scale would be dependent on the number of specifics that you desire to be advertised out of rack in the case above, so it would look something like this. hosts (/32) -> rack-sw -> (/26 + N * /32_external) -> aggregation-sw -> (/17) On the note about label switching versus ip forwarding, I would not think of it as either/or but rather both could be used since you need to eventually serve pages to users.

donavanm 13y ago
Yeah, i was specifically thinking of the prefixes jn the tor and agg layer. I read it as having all the vips in the rack as next hop /32s. Add a bunch of intra rack ecmp groups and its not insignificant. Im not sure i buy the feasability of /27 per rack or a simple /17 supernet at the agg layer long term. Rack address counts will not be uniform, id expect everything from /29s to /23s depending on host types. Assuming uniform allocation at the pod/dc level also seems like a stretch. Renumbering and super/subnetting is going to fragment the ip locality over time. Ecmping through the core eats yet more prefixes, and (Id guess) forces hot ones to be deagg'd. All that said, you do get a couple thousand prefixes on your $10,000 trident box. And im obviously not an NE. So maybe it would work inside a small cluster.