6 ms·

"I am proposing to fundamentally rethink the router, adopt modern software architecture and paradigms, and urge the industry to catch up after 10 years of stagn

by hueving 10y ago

"I am proposing to fundamentally rethink the router, adopt modern software architecture and paradigms, and urge the industry to catch up after 10 years of stagnation." I think the author may have been living in a hole. This is not a new idea. Datacenter routing in many of the big companies is already being done with SDN ala OpenFlow or some other custom protocol. Right now you can buy a whitebox 'switch' with 40gbps interfaces and load various operating systems that enable different management styles (e.g. OpenFlow control like Google http://opennetsummit.org/archives/apr12/hoelzle-tue-openflow.pdf http://opennetsummit.org/archives/apr12/hoelzle-tue-openflow...). The router has already been 're-thought', it's currently just hiding under the term 'whitebox datacenter switch'.

hannesgredler 10y ago
where theory hits reality is when those whiteboxes and their routing-stacks encounter the 600K routes coming from 40 different exits. still no viable solution out there today- quagga, bird, any ?
- hueving 10y ago
  >where theory hits reality is when those whiteboxes and their routing-stacks encounter the 600K routes coming from 40 different exits. still no viable solution out there today- quagga, bird, any ? Well handling the routes themselves can be offloaded to regular servers. The issue is indeed managing the forwarding tables. "Something something aggregation, non contiguous bitmasks, NDA" runs wildly for the door. :)
- bogomipz 10y ago
  The IPv4 table today is about 610K routes. There shouldn't be a problem fitting the RIB, associated BGP communities and AS Path info in 32 Gigs of DDR 4 RAM. There are plenty of boards that will hold 750Mbs of RAM from the usual folks - DELL, HP and Supermicro. So why is there no viable option? Also this is a hardware concern. How does choice of open source solution - Vyatta, Qugga or Bird matter? What is the issue?
  - wmf 10y ago
    The issue is that Tomahawk has far less than 610K entries of TCAM and a dozen different teams are exploring various types of RIB caching to accommodate that.
    - bogomipz 10y ago
      Sorry, what is Tomahawk?
      - hannesgredler 10y ago
        Its a 3TBits/s chipset from Broadcom https://www.broadcom.com/products/ethernet-communication-and-switching/switching/bcm56960-series https://www.broadcom.com/products/ethernet-communication-and...
        bogomipz 10y ago
        Thanks for the link, but how does this relate to my comment asking comment asking why you can't build your own router with commodity hardware? Isn't the Tommahawk and Jericho intended for ODMs who are building traditional switches?
        wmf 10y ago
        It's pointless to use anything other than these ASICs because their price/performance is so far ahead. Software routing is especially pointless.
    - scurvy 10y ago
      Tomahawk is old and busted. Jericho is the new hotness. Full tables in FIB. https://www.broadcom.com/press/release.php?id=s902223 https://www.broadcom.com/press/release.php?id=s902223
      - wmf 10y ago
        Shh! You're ruining it.
- scurvy 10y ago
  A new whitebox based on Jericho is your solution.
CodingInfra 10y ago
This is not mainstream. I forget which OFC this was first presented at, but the Google presenter said they had ~120 employees dedicated to this project. Most companies don't have Google's talent or resources to implement something equivalent. Everyone is quick to reference Google, but can you tell me 2 other commercial customers (non-research) who have deployed OpenFlow at scale - probably not. ;-) Operators who are doing this today are the exception. Going back to Linux, fact is using Linux on a bare metal switch today is like using Slackware in 1995. The need for a more turn-key Linux distro, with better apps (better routing, telemetry, ...etc) is needed to compliment white box hardware today. For a long time the HW has been the limiting factor since most merchant PFE platforms supported <50k FIB entries. Now with 1M+ (on chip) around the corner, SW quality and scale will need to improve accordingly.
- hueving 10y ago
  >who have deployed OpenFlow at scale I didn't say it was OpenFlow everywhere. I meant custom software in some way (i.e. not an off the shelf Juniper/Cisco/whatever). OpenFlow has pretty narrow use cases once you dump reactive flows so it's definitely rare in the wild. For other customers using whitebox switches with different OS's booted, look at whoever is paying companies like Cumulus/Big Switch.
- pas 10y ago
  Probably every OpenStack hosting company uses something like OpenFlow to provide the virtual/tenant/overlay network, so Rackspace et al. Also see Project Calico for containers, they use the linux kernel as FIB, but obviously they could just as well use OpenFlow to program switches for bare metal machines (which then might run containers).
  - pquerna 10y ago
    As deployed, most OpenStack SDNs will use OVS or similar software on top of a (xen,kvm) Hypervisor, connected to some kind of central controller, which just maybe would look kinda like OpenFlow. But all of this is running as an overlay on top of a much more traditional network. This is very different than a TOR switch or core-router running OpenFlow. Only a few places are running more than a couple thousand VMs in a single SDN, and its nothing like what is being discussed above.
hackney 10y ago
Personally, I think what needs to be rethought is the fact they are a separate device from the actual pc. Imo, they should exist inside the computer, cable modem included. As far as switches are concerned, obviously those will take more time to absorb into the general infrastructure of things. Simply put from a user perspective, it would be nice to have the 'router' inside the box and not separated from it.
- forgottenpass 10y ago
  they should exist inside the computer, cable modem included Not _that_ kind of router. The type of router connected to you cable modem is a box that performs NAT, not IP routing.
  - hackney 10y ago
    Yes, that kind. I have a cable modem router that has no buttons or controls on it whatsoever, separate from the pc, obviously. Then, I have the actual wireless router attached to that. I could have bought a single cable modem/wireless router. Regardless, depending upon the application there are essentially all one and the same in intent.
    - icebraining 10y ago
      There are many such devices, e.g. the Netgear N600. In any case, you're not talking about the kind of routers being discussed in this thread.
      - hackney 10y ago
        If the 'thread' applies to rethinking routers then I have every right to have posted what I have. Seems you are the one that wants to make an argument while I was making my own personal suggestion. Instead of assuming that you are smart while I am not, I suggest you let it go.
        deleted 10y ago
walrus01 10y ago
> Right now you can buy a whitebox 'switch' with 40gbps interfaces and load various operating systems that enable different management styles (e.g. OpenFlow control like Google http://opennetsummit.org/archives/apr12/hoelzle-tue-openflow... http://opennetsummit.org/archives/apr12/hoelzle-tue-openflow...). This is all very useful for in-building traffic until you get the traffic to the edge of your datacenter/colo/hosting environment network and need to exchange traffic with other ISPs... At which point you need a serious chassis based router with redundancy and full layer 3 capabilities (example: Juniper MX960, Cisco ASR9006 or 9010). whitebox datacenter switches are a great way to get a lot of 10, 40 and 100 Gbps layer 2 ethernet switch capacity within a datacenter cheaply, they're NOT routers and should not be mistaken with them. They are things that you connect your hypervisor hardware platforms to (example: a whole shitload of 1U servers or facebook/OCP type servers each with dual socket, 16-core xeons in them), is a switch.
- wmf 10y ago
  It's only a matter of time until whitebox routers exist; Accton recently submitted one to OCP. Then it will take a few years for the software stack to be built.

by hueving 10y ago

hannesgredler 10y ago

where theory hits reality is when those whiteboxes and their routing-stacks encounter the 600K routes coming from 40 different exits. still no viable solution out there today- quagga, bird, any ?

hueving 10y ago
>where theory hits reality is when those whiteboxes and their routing-stacks encounter the 600K routes coming from 40 different exits. still no viable solution out there today- quagga, bird, any ? Well handling the routes themselves can be offloaded to regular servers. The issue is indeed managing the forwarding tables. "Something something aggregation, non contiguous bitmasks, NDA" runs wildly for the door. :)
bogomipz 10y ago
The IPv4 table today is about 610K routes. There shouldn't be a problem fitting the RIB, associated BGP communities and AS Path info in 32 Gigs of DDR 4 RAM. There are plenty of boards that will hold 750Mbs of RAM from the usual folks - DELL, HP and Supermicro. So why is there no viable option? Also this is a hardware concern. How does choice of open source solution - Vyatta, Qugga or Bird matter? What is the issue?
- wmf 10y ago
  The issue is that Tomahawk has far less than 610K entries of TCAM and a dozen different teams are exploring various types of RIB caching to accommodate that.
  - bogomipz 10y ago
    Sorry, what is Tomahawk?
    - hannesgredler 10y ago
      Its a 3TBits/s chipset from Broadcom https://www.broadcom.com/products/ethernet-communication-and-switching/switching/bcm56960-series https://www.broadcom.com/products/ethernet-communication-and...
      - bogomipz 10y ago
        Thanks for the link, but how does this relate to my comment asking comment asking why you can't build your own router with commodity hardware? Isn't the Tommahawk and Jericho intended for ODMs who are building traditional switches?
        wmf 10y ago
        It's pointless to use anything other than these ASICs because their price/performance is so far ahead. Software routing is especially pointless.
  - scurvy 10y ago
    Tomahawk is old and busted. Jericho is the new hotness. Full tables in FIB. https://www.broadcom.com/press/release.php?id=s902223 https://www.broadcom.com/press/release.php?id=s902223
    - wmf 10y ago
      Shh! You're ruining it.
scurvy 10y ago
A new whitebox based on Jericho is your solution.

CodingInfra 10y ago

This is not mainstream. I forget which OFC this was first presented at, but the Google presenter said they had ~120 employees dedicated to this project. Most companies don't have Google's talent or resources to implement something equivalent. Everyone is quick to reference Google, but can you tell me 2 other commercial customers (non-research) who have deployed OpenFlow at scale - probably not. ;-) Operators who are doing this today are the exception. Going back to Linux, fact is using Linux on a bare metal switch today is like using Slackware in 1995. The need for a more turn-key Linux distro, with better apps (better routing, telemetry, ...etc) is needed to compliment white box hardware today. For a long time the HW has been the limiting factor since most merchant PFE platforms supported <50k FIB entries. Now with 1M+ (on chip) around the corner, SW quality and scale will need to improve accordingly.

hueving 10y ago
>who have deployed OpenFlow at scale I didn't say it was OpenFlow everywhere. I meant custom software in some way (i.e. not an off the shelf Juniper/Cisco/whatever). OpenFlow has pretty narrow use cases once you dump reactive flows so it's definitely rare in the wild. For other customers using whitebox switches with different OS's booted, look at whoever is paying companies like Cumulus/Big Switch.
pas 10y ago
Probably every OpenStack hosting company uses something like OpenFlow to provide the virtual/tenant/overlay network, so Rackspace et al. Also see Project Calico for containers, they use the linux kernel as FIB, but obviously they could just as well use OpenFlow to program switches for bare metal machines (which then might run containers).
- pquerna 10y ago
  As deployed, most OpenStack SDNs will use OVS or similar software on top of a (xen,kvm) Hypervisor, connected to some kind of central controller, which just maybe would look kinda like OpenFlow. But all of this is running as an overlay on top of a much more traditional network. This is very different than a TOR switch or core-router running OpenFlow. Only a few places are running more than a couple thousand VMs in a single SDN, and its nothing like what is being discussed above.

hackney 10y ago

Personally, I think what needs to be rethought is the fact they are a separate device from the actual pc. Imo, they should exist inside the computer, cable modem included. As far as switches are concerned, obviously those will take more time to absorb into the general infrastructure of things. Simply put from a user perspective, it would be nice to have the 'router' inside the box and not separated from it.

forgottenpass 10y ago
they should exist inside the computer, cable modem included Not _that_ kind of router. The type of router connected to you cable modem is a box that performs NAT, not IP routing.
- hackney 10y ago
  Yes, that kind. I have a cable modem router that has no buttons or controls on it whatsoever, separate from the pc, obviously. Then, I have the actual wireless router attached to that. I could have bought a single cable modem/wireless router. Regardless, depending upon the application there are essentially all one and the same in intent.
  - icebraining 10y ago
    There are many such devices, e.g. the Netgear N600. In any case, you're not talking about the kind of routers being discussed in this thread.
    - hackney 10y ago
      If the 'thread' applies to rethinking routers then I have every right to have posted what I have. Seems you are the one that wants to make an argument while I was making my own personal suggestion. Instead of assuming that you are smart while I am not, I suggest you let it go.
      - deleted 10y ago

walrus01 10y ago

> Right now you can buy a whitebox 'switch' with 40gbps interfaces and load various operating systems that enable different management styles (e.g. OpenFlow control like Google http://opennetsummit.org/archives/apr12/hoelzle-tue-openflow... http://opennetsummit.org/archives/apr12/hoelzle-tue-openflow...). This is all very useful for in-building traffic until you get the traffic to the edge of your datacenter/colo/hosting environment network and need to exchange traffic with other ISPs... At which point you need a serious chassis based router with redundancy and full layer 3 capabilities (example: Juniper MX960, Cisco ASR9006 or 9010). whitebox datacenter switches are a great way to get a lot of 10, 40 and 100 Gbps layer 2 ethernet switch capacity within a datacenter cheaply, they're NOT routers and should not be mistaken with them. They are things that you connect your hypervisor hardware platforms to (example: a whole shitload of 1U servers or facebook/OCP type servers each with dual socket, 16-core xeons in them), is a switch.

wmf 10y ago
It's only a matter of time until whitebox routers exist; Accton recently submitted one to OCP. Then it will take a few years for the software stack to be built.