3 ms·

It's very unlikely. You must consider that those routers aren't standardized embedded systems, but they are highly specialized with ASIC chips, in-house operati

It's very unlikely. You must consider that those routers aren't standardized embedded systems, but they are highly specialized with ASIC chips, in-house operation systems with its homebrew network stack. Even with perfect documentation, it's not something that you can just fire up a BSD on - getting it all supported and running requires many years of works. Many of the functionalities of those routers make special assumptions to boost performance and don't even fit into a standard network stack, so minimally, you would need to develop your own functional network stack for that. Or at least that's my impression of them, correct me if I'm misinformed.

topranks 7y ago
There are projects like ONIE that provide a common framework to bootstrap “white box” switches, and there are operating systems like Cumulus that can run on numerous platforms. We’re not there yet but the situation is definitely changing / improving.
- kingosticks 7y ago
  > We’re not there yet but the situation is definitely changing / improving. But only if you can use that commodity hardware. If you need the stuff running on ASICs, it's as the parent describes and I don't see how that could change. Can whitebox solutions ever offer the same performance as boxes from cisco etc? If not then I'm not convinced there will be much more improvement coming.
  - halbritt 7y ago
    "whitebox solutions" are merchant silicon designs, typically Broadcom. Cisco's entire Nexus line of switches was there entry into the market with merchant silicon. Their costs were much lower, but their pricing per port stayed roughly the same as their Catalyst line driving up margins. Adoption was pretty lackluster however until they started offering massive discounts on the Nexus products. Anyway, you can generally get non-blocking line rate on all ports out of any Broadcom reference design. The only distinction between that and a Cisco is the how the device is managed.
    - kingosticks 7y ago
      I confused 'whitebox' and those switches based on x86-based hardware. Isn't the price differentiation also due to a reduced feature set?
      - halbritt 7y ago
        Kinda depends. Layer 3 forwarding is all the same. Whitebox stuff generally comes with simple routing protocol implementations. Cisco, Juniper, et al. have routing protocol implementations generally guaranteed to work with their other network elements. That's how the L3 forwarding tables get populated. Whether that makes a difference depends on the network architecture. FAANG companies and others are deploying whitebox switches, building "underlay" networks using eBGP, and using SDN as an overlay basically obviating the need to have much intelligence built into the network element. Cumulus Linux replicates these efforts to some extent. One gets a whitebox switch running Linux with open source routing protocol implementations and configuration via something like ansible.
        oneplane 7y ago
        Yep, and SDN is also possible if you get switches that support things like being an openvswitch-slave or OpenFlow/NetFlow/whateverflow switch. You basically replace the control plane. That's why a lot of engineering is based around getting data planes working, and ignoring most of the control plane that runs on the application processor. Even if you only get that to control the ASICs and run a simple REST server to accept data plane control commands it's already enough to turn a switch into something useful forever.
oneplane 7y ago
Yes, the ASICs require highly specialised setup and it's the main issue with creating firmware from a reverse-engineering perspective. Most firmwares are either created from reverse-engineering projects which due to the time consuming nature often target older switching chips like Broadcom chips from around 2009. Setting them up is a PITA and almost an order of magnitude harder than DDR memory training or x86 bootstrapping without an FSP/BSP. But it's possible and has been shown to work. Other projects like Open Compute and ONIE give a lot of insight for reverse engineering of newer fabric and data plane components which significantly shortens the process of reversing and re-using existing hardware. The control plane side of things is much easier in that regard as they are commonly generic ARM or POWER architecture SoCs with generic Flash/PROM components running a more generic RTOS (i.e. one that is used across an entire line of products and supping many data planes) or even a Linux based OS. That, combined with easy to extract firmwares and JTAG + Serial makes the control plane a lot less complex than the data plane. This is also why control planes are usually left alone because you can drop in any control plane software we already have and focus on interfacing with the ASICs instead.
- segfaultbuserr 7y ago
  > Setting them up is a PITA and almost an order of magnitude harder than DDR memory training or x86 bootstrapping without an FSP/BSP. But it's possible and has been shown to work. Wow, an order of magnitude harder that booting x86 without FSP, but it's still possible... I can't imagine that. I salute the efforts. Eyeopening! Thanks.
  - oneplane 7y ago
    It's possible but because there are other fabric/data-plane ASICs are are not an order of magnitude harder than Cisco's chips most time is spent on those instead. The problem with the Cisco ones is mostly because they sit somewhere in between FPGAs and a Barrel CPU. Without the custom loading and init they are essentially useless. Think of them as an FPGA with a softcore CPU. If they are empty they have no CPU so you can't run code on them. If you fill them with Cisco core data you get no use out of them without logging in to their DRM server. So you need to feed them the softcore but without the cisco runtime code.

topranks 7y ago

There are projects like ONIE that provide a common framework to bootstrap “white box” switches, and there are operating systems like Cumulus that can run on numerous platforms. We’re not there yet but the situation is definitely changing / improving.

kingosticks 7y ago
> We’re not there yet but the situation is definitely changing / improving. But only if you can use that commodity hardware. If you need the stuff running on ASICs, it's as the parent describes and I don't see how that could change. Can whitebox solutions ever offer the same performance as boxes from cisco etc? If not then I'm not convinced there will be much more improvement coming.
- halbritt 7y ago
  "whitebox solutions" are merchant silicon designs, typically Broadcom. Cisco's entire Nexus line of switches was there entry into the market with merchant silicon. Their costs were much lower, but their pricing per port stayed roughly the same as their Catalyst line driving up margins. Adoption was pretty lackluster however until they started offering massive discounts on the Nexus products. Anyway, you can generally get non-blocking line rate on all ports out of any Broadcom reference design. The only distinction between that and a Cisco is the how the device is managed.
  - kingosticks 7y ago
    I confused 'whitebox' and those switches based on x86-based hardware. Isn't the price differentiation also due to a reduced feature set?
    - halbritt 7y ago
      Kinda depends. Layer 3 forwarding is all the same. Whitebox stuff generally comes with simple routing protocol implementations. Cisco, Juniper, et al. have routing protocol implementations generally guaranteed to work with their other network elements. That's how the L3 forwarding tables get populated. Whether that makes a difference depends on the network architecture. FAANG companies and others are deploying whitebox switches, building "underlay" networks using eBGP, and using SDN as an overlay basically obviating the need to have much intelligence built into the network element. Cumulus Linux replicates these efforts to some extent. One gets a whitebox switch running Linux with open source routing protocol implementations and configuration via something like ansible.
      - oneplane 7y ago
        Yep, and SDN is also possible if you get switches that support things like being an openvswitch-slave or OpenFlow/NetFlow/whateverflow switch. You basically replace the control plane. That's why a lot of engineering is based around getting data planes working, and ignoring most of the control plane that runs on the application processor. Even if you only get that to control the ASICs and run a simple REST server to accept data plane control commands it's already enough to turn a switch into something useful forever.

oneplane 7y ago

Yes, the ASICs require highly specialised setup and it's the main issue with creating firmware from a reverse-engineering perspective. Most firmwares are either created from reverse-engineering projects which due to the time consuming nature often target older switching chips like Broadcom chips from around 2009. Setting them up is a PITA and almost an order of magnitude harder than DDR memory training or x86 bootstrapping without an FSP/BSP. But it's possible and has been shown to work. Other projects like Open Compute and ONIE give a lot of insight for reverse engineering of newer fabric and data plane components which significantly shortens the process of reversing and re-using existing hardware. The control plane side of things is much easier in that regard as they are commonly generic ARM or POWER architecture SoCs with generic Flash/PROM components running a more generic RTOS (i.e. one that is used across an entire line of products and supping many data planes) or even a Linux based OS. That, combined with easy to extract firmwares and JTAG + Serial makes the control plane a lot less complex than the data plane. This is also why control planes are usually left alone because you can drop in any control plane software we already have and focus on interfacing with the ASICs instead.

segfaultbuserr 7y ago
> Setting them up is a PITA and almost an order of magnitude harder than DDR memory training or x86 bootstrapping without an FSP/BSP. But it's possible and has been shown to work. Wow, an order of magnitude harder that booting x86 without FSP, but it's still possible... I can't imagine that. I salute the efforts. Eyeopening! Thanks.
- oneplane 7y ago
  It's possible but because there are other fabric/data-plane ASICs are are not an order of magnitude harder than Cisco's chips most time is spent on those instead. The problem with the Cisco ones is mostly because they sit somewhere in between FPGAs and a Barrel CPU. Without the custom loading and init they are essentially useless. Think of them as an FPGA with a softcore CPU. If they are empty they have no CPU so you can't run code on them. If you fill them with Cisco core data you get no use out of them without logging in to their DRM server. So you need to feed them the softcore but without the cisco runtime code.