5 ms·

Facebook aims to knock Cisco down a peg with open network hardware

83 points by velodrome 13y ago

quesera 13y ago
Long overdue. Cisco is in trouble for a lot of reasons, but this might be the straw that breaks the backplane. Routers and switches make the world go 'round on Tasman Drive... The timer is running on Chambers' resignation. I don't think he'll be fired -- he's too established and congenial -- but I also don't think he wants to run a company 1/3rd of Cisco's size
- lmg643 13y ago
  Agreed. Plus, this may be the coolest thing going on at Facebook.
bluedino 13y ago
This is just like the big server vendors (IBM, Dell and HP) losing all their sales to big companies, because they are buying their own designs direct from companies like ZT and Hyve. What's left, storage systems?
- wmf 13y ago
  No, they're also working on storage.
- smackfu 13y ago
  I can't imagine that the margins on sales to big companies are that great, considering how much volume they can throw around.
- bitwize 13y ago
  IBM big systems will still be running long after you and I are dead. Running the same software that was first written before we were born.
- druiid 13y ago
  Already being worked on really. It's more blob-storage oriented, but take a look at Openstack Swift if you haven't before or Ceph. Lots of other projects out there. If you don't specifically mean block-storage type software and more open replacements for Netapp/EMC gear, someone else might be the better one to chime in.
- e12e 13y ago
  For the hardware (storage) part, there is the storage pod from Backblaze: http://blog.backblaze.com/2013/02/20/180tb-of-good-vibrations-storage-pod-3-0/ http://blog.backblaze.com/2013/02/20/180tb-of-good-vibration...
Ironlink 13y ago
For those interested, Google has been using OpenFlow for some time now. This presentation by Urs Hölzle is about a year old: http://www.youtube.com/watch?v=VLHJUfgxEO4 http://www.youtube.com/watch?v=VLHJUfgxEO4 Slides: http://www.opennetsummit.org/archives/apr12/hoelzle-tue-openflow.pdf http://www.opennetsummit.org/archives/apr12/hoelzle-tue-open...
- deelowe 13y ago
  Funny how this didn't make as much waves back then when they announced it. I honestly think it went over everyone's head.
minimax 13y ago
What is the advantage you get from having a sophisticated software stack on a top of rack switch? I was under the impression that you only really configured vlans and maybe a span port every now and again. Someone care to enlighten me?
- fourspace 13y ago
  It isn't always the software. Having open, controlled, customizable hardware can offer a huge advantage over what you're able to purchase from Cisco/Juniper/etc. Also, a software switch allows you potentially massively scale the number of ports on a single switch.
- wmf 13y ago
  In that case, eliminating the complexity of existing control plane software will be a win. Some people may want to run OSPF or IS-IS, or do some kind of performance analysis, or do ARP/ND/DHCP proxying, or perhaps use switches to aggregate BMC monitoring & control, or who knows what.
  - minimax 13y ago
    I get that you would want all that stuff in a router, but the article is mostly about a new type of top-of-rack switch. My understanding (based on experience that is now a few years old) is that a top-of-rack switch basically only handles layer two switching duties and a separate core router handles the layer three business.
    - wmf 13y ago
      Some people perform routing in the ToR because STP doesn't scale. And TRILL/SPB have been proposed to make Ethernet scale, but then they end up having similar complexity as routing (even using IS-IS).
- csears 13y ago
  In many common environments (single-tenant, fairly static configs) it probably doesn't make sense to use an SDN. However, if your environment looks more like EC2's, with lots of VMs/apps/services, lots of tenants, and lots of virtual networks, you need some way create an overlay that provides logical layer 2 switching beyond what simple VLANs can provide. For example, if I want to migrate a running VM from one host to another, it would be nice if the top of rack switches could be made aware of the change as it happens so CAM tables could be updated on the fly. Cisco and VMware do something similar today with the DVS and Nexus 1000v virtual switches, and a lot of the OpenFlow/SDN vendors are hoping to extend that type of dynamic orchestration out to physical switches as well.
dhess 13y ago
I would love to buy a server that conforms to Open Compute and a rack that conforms to Open Rack. Where can I do that?
- ericcoleman 13y ago
  Quick google search turned up: Penguin Computing: http://www.penguincomputing.com/Products/OpenComputeSolutions http://www.penguincomputing.com/Products/OpenComputeSolution... Silicon Mechanics: http://www.siliconmechanics.com/c1345 http://www.siliconmechanics.com/c1345
  - dhess 13y ago
    Thanks! Last time I looked, I came up with nothing but the specs on the Open Compute page.
  - wmf 13y ago
    That Silicon Mechanics stuff is obsolete and the Penguin servers don't look like OCP at all.
    - throwaway1701 13y ago
      Former Silicon Mechanics employee here. While I was working there, we had between zero and no interest whatsoever from anybody in actually seriously trying to purchase anything OpenCompute. Apparently some board members used to work with some higher-ups at Facebook, so the project is sticking around for a while despite the total lack of interest. If there's any actual real-world interest in OpenCompute stuff, you HAVE to reach out and ask or it's going to disappear.
      - wmf 13y ago
        Good point, although I suspect people who are willing to reach out are going to the ODMs and people who have interest in clicking the "buy now" button cannot express that interest because there is no "buy now" button.
- 13y ago
mprovost 13y ago
SDN and OpenFlow are great if you're a company full of programmers like Facebook and Google are. They approach everything as a software problem. For many (most?) other companies the network is just something that has to work and not a core part of their business. You need to be able to hire a network engineer that can come in cold, with a bunch of vendor certifications on their CV, and understand your network. Most places don't want to hire a programmer to sit around and code up new spanning tree algorithms for their network. They just want to plug servers in and have it work. The people that are interested in SDN are running into the limits of what OSPF/BGP/MPLS can do and are going to work around it but many places don't run networks that complex in the first place. I think there are a lot of gains to be made from changing the industry though, for the first time the hardware is becoming a commodity made by Broadcom and Intel (Fulcrum) just like servers. In the past you bought Ciscos or Junipers to get the custom ASICs that made the thing work, those days are coming to and end. But I wouldn't count the big companies out just yet, they have a massive head start writing software and have huge service organisations and a whole industry full of engineers that have paid to get certified on their hardware and aren't going to want to see the value of that education fall to zero. The winners are going to be the hardware companies that really are software companies at heart - Juniper and Arista are the best examples. Cisco has always loved its hardware and been way behind on software development.
- insaneirish 13y ago
  This switch does not require SDN or OpenFlow. The idea is you run whatever OS on it you want that provides the features you need. For instance, you can go buy an OS from Cumulus (mentioned in the article) and run it on an ODM switch and run traditional routing protocols and topologies. It is only a matter of time before other somewhat traditional vendors start selling their OS for commodity hardware as well. Arista will likely be the first.
- illuminate 13y ago
  "They approach everything as a software problem. For many (most?) other companies the network is just something that has to work and not a core part of their business. You need to be able to hire a network engineer that can come in cold, with a bunch of vendor certifications on their CV, and understand your network. Most places don't want to hire a programmer to sit around and code up new spanning tree algorithms for their network. They just want to plug servers in and have it work" Do you not need network engineers to implement Cisco products?
nixisfun 13y ago
People don't buy Cisco just for the box with ports and blinky lights. They buy them for the supportability. As for a business they depend on here network to run and as mprovost mentioned you can take some one cold who has networking experience and If they have half a brain they should be able to fix your network. All Cisco has spawned a whole Eco system of vars who exist throughout the world and can have replacement parts onsite in 4 hours or less. Also Cisco provides code scrubs and a range of other features. So unless some vendor can build these magical switches and provide support they will never be a serious option in any environment.

83 points by velodrome 13y ago

Long overdue. Cisco is in trouble for a lot of reasons, but this might be the straw that breaks the backplane. Routers and switches make the world go 'round on Tasman Drive... The timer is running on Chambers' resignation. I don't think he'll be fired -- he's too established and congenial -- but I also don't think he wants to run a company 1/3rd of Cisco's size

lmg643 13y ago
Agreed. Plus, this may be the coolest thing going on at Facebook.

bluedino 13y ago

This is just like the big server vendors (IBM, Dell and HP) losing all their sales to big companies, because they are buying their own designs direct from companies like ZT and Hyve. What's left, storage systems?

wmf 13y ago
No, they're also working on storage.
smackfu 13y ago
I can't imagine that the margins on sales to big companies are that great, considering how much volume they can throw around.
bitwize 13y ago
IBM big systems will still be running long after you and I are dead. Running the same software that was first written before we were born.
druiid 13y ago
Already being worked on really. It's more blob-storage oriented, but take a look at Openstack Swift if you haven't before or Ceph. Lots of other projects out there. If you don't specifically mean block-storage type software and more open replacements for Netapp/EMC gear, someone else might be the better one to chime in.
e12e 13y ago
For the hardware (storage) part, there is the storage pod from Backblaze: http://blog.backblaze.com/2013/02/20/180tb-of-good-vibrations-storage-pod-3-0/ http://blog.backblaze.com/2013/02/20/180tb-of-good-vibration...

Ironlink 13y ago

For those interested, Google has been using OpenFlow for some time now. This presentation by Urs Hölzle is about a year old: http://www.youtube.com/watch?v=VLHJUfgxEO4 http://www.youtube.com/watch?v=VLHJUfgxEO4 Slides: http://www.opennetsummit.org/archives/apr12/hoelzle-tue-openflow.pdf http://www.opennetsummit.org/archives/apr12/hoelzle-tue-open...

deelowe 13y ago
Funny how this didn't make as much waves back then when they announced it. I honestly think it went over everyone's head.

minimax 13y ago

What is the advantage you get from having a sophisticated software stack on a top of rack switch? I was under the impression that you only really configured vlans and maybe a span port every now and again. Someone care to enlighten me?

fourspace 13y ago
It isn't always the software. Having open, controlled, customizable hardware can offer a huge advantage over what you're able to purchase from Cisco/Juniper/etc. Also, a software switch allows you potentially massively scale the number of ports on a single switch.
wmf 13y ago
In that case, eliminating the complexity of existing control plane software will be a win. Some people may want to run OSPF or IS-IS, or do some kind of performance analysis, or do ARP/ND/DHCP proxying, or perhaps use switches to aggregate BMC monitoring & control, or who knows what.
- minimax 13y ago
  I get that you would want all that stuff in a router, but the article is mostly about a new type of top-of-rack switch. My understanding (based on experience that is now a few years old) is that a top-of-rack switch basically only handles layer two switching duties and a separate core router handles the layer three business.
  - wmf 13y ago
    Some people perform routing in the ToR because STP doesn't scale. And TRILL/SPB have been proposed to make Ethernet scale, but then they end up having similar complexity as routing (even using IS-IS).
csears 13y ago
In many common environments (single-tenant, fairly static configs) it probably doesn't make sense to use an SDN. However, if your environment looks more like EC2's, with lots of VMs/apps/services, lots of tenants, and lots of virtual networks, you need some way create an overlay that provides logical layer 2 switching beyond what simple VLANs can provide. For example, if I want to migrate a running VM from one host to another, it would be nice if the top of rack switches could be made aware of the change as it happens so CAM tables could be updated on the fly. Cisco and VMware do something similar today with the DVS and Nexus 1000v virtual switches, and a lot of the OpenFlow/SDN vendors are hoping to extend that type of dynamic orchestration out to physical switches as well.

dhess 13y ago

I would love to buy a server that conforms to Open Compute and a rack that conforms to Open Rack. Where can I do that?

ericcoleman 13y ago
Quick google search turned up: Penguin Computing: http://www.penguincomputing.com/Products/OpenComputeSolutions http://www.penguincomputing.com/Products/OpenComputeSolution... Silicon Mechanics: http://www.siliconmechanics.com/c1345 http://www.siliconmechanics.com/c1345
- dhess 13y ago
  Thanks! Last time I looked, I came up with nothing but the specs on the Open Compute page.
- wmf 13y ago
  That Silicon Mechanics stuff is obsolete and the Penguin servers don't look like OCP at all.
  - throwaway1701 13y ago
    Former Silicon Mechanics employee here. While I was working there, we had between zero and no interest whatsoever from anybody in actually seriously trying to purchase anything OpenCompute. Apparently some board members used to work with some higher-ups at Facebook, so the project is sticking around for a while despite the total lack of interest. If there's any actual real-world interest in OpenCompute stuff, you HAVE to reach out and ask or it's going to disappear.
    - wmf 13y ago
      Good point, although I suspect people who are willing to reach out are going to the ODMs and people who have interest in clicking the "buy now" button cannot express that interest because there is no "buy now" button.
13y ago

mprovost 13y ago

SDN and OpenFlow are great if you're a company full of programmers like Facebook and Google are. They approach everything as a software problem. For many (most?) other companies the network is just something that has to work and not a core part of their business. You need to be able to hire a network engineer that can come in cold, with a bunch of vendor certifications on their CV, and understand your network. Most places don't want to hire a programmer to sit around and code up new spanning tree algorithms for their network. They just want to plug servers in and have it work. The people that are interested in SDN are running into the limits of what OSPF/BGP/MPLS can do and are going to work around it but many places don't run networks that complex in the first place. I think there are a lot of gains to be made from changing the industry though, for the first time the hardware is becoming a commodity made by Broadcom and Intel (Fulcrum) just like servers. In the past you bought Ciscos or Junipers to get the custom ASICs that made the thing work, those days are coming to and end. But I wouldn't count the big companies out just yet, they have a massive head start writing software and have huge service organisations and a whole industry full of engineers that have paid to get certified on their hardware and aren't going to want to see the value of that education fall to zero. The winners are going to be the hardware companies that really are software companies at heart - Juniper and Arista are the best examples. Cisco has always loved its hardware and been way behind on software development.

insaneirish 13y ago
This switch does not require SDN or OpenFlow. The idea is you run whatever OS on it you want that provides the features you need. For instance, you can go buy an OS from Cumulus (mentioned in the article) and run it on an ODM switch and run traditional routing protocols and topologies. It is only a matter of time before other somewhat traditional vendors start selling their OS for commodity hardware as well. Arista will likely be the first.
illuminate 13y ago
"They approach everything as a software problem. For many (most?) other companies the network is just something that has to work and not a core part of their business. You need to be able to hire a network engineer that can come in cold, with a bunch of vendor certifications on their CV, and understand your network. Most places don't want to hire a programmer to sit around and code up new spanning tree algorithms for their network. They just want to plug servers in and have it work" Do you not need network engineers to implement Cisco products?

nixisfun 13y ago

People don't buy Cisco just for the box with ports and blinky lights. They buy them for the supportability. As for a business they depend on here network to run and as mprovost mentioned you can take some one cold who has networking experience and If they have half a brain they should be able to fix your network. All Cisco has spawned a whole Eco system of vars who exist throughout the world and can have replacement parts onsite in 4 hours or less. Also Cisco provides code scrubs and a range of other features. So unless some vendor can build these magical switches and provide support they will never be a serious option in any environment.