3 ms·

How does this compare with existing switch OSs from Cisco and Juniper? Linux seems like it would be a bit heavy for a switch.

wmf 6y ago
SONIC is less mature but it's free and open source with the benefits and drawbacks that entails. For example, SONIC runs on a variety of different hardware so you could take advantage of that diversity without having to learn different NOSes. Ten years ago switches were using 800 MHz single-core PowerPCs which was adequate to run Linux (although many were using VxWorks or whatever). Now the $400 switches are still wimpy but more expensive "disaggregated" switches are using Atoms or low-end Xeons.
iptrans 6y ago
Adding to what wmf said, JunOS is based on BSD and JunoOS evolved will be based on Linux. Most network equipment run their control plane using standard Intel CPUs, so running Linux isn't much of a strech.
monocasa 6y ago
Eh, it's probably not that bad. On the low end, Linux isn't a slouch with XDP, etc. On the high end, it's game over if your kernel is in the data plane at all anyway, and Linux is a great option for the control plane of a switch.
tw04 6y ago
Cisco is already Linux based. IOS-XE and NX-OS. https://en.m.wikipedia.org/wiki/Cisco_IOS_XE https://en.m.wikipedia.org/wiki/Cisco_IOS_XE https://en.m.wikipedia.org/wiki/Cisco_NX-OS https://en.m.wikipedia.org/wiki/Cisco_NX-OS
- 12bits 6y ago
  I like this comparison https://medium.com/@Meela349588204/using-cows-to-explain-the-differences-of-the-cisco-operating-systems-876d14145827 https://medium.com/@Meela349588204/using-cows-to-explain-the...
  - myrandomcomment 6y ago
    If you want to enjoy even more pain, ah trivia I mean, NXOS and NXOS are not the same thing. Read it again. There is an NXOS train for the original 2/5/7K and current kit in that line, and there is another version for the 3/9K new kit from the ACI spin in. They are just different enough that it makes things difficult when you are trying to develop tools for them. Cisco has promised to merge the two for a long time now.
- Cyph0n 6y ago
  IOS-XR is also Linux-based.
  - p_l 6y ago
    IOS-XR at least used to be QNX based, with somewhat distributed architecture (line cards running separate OS instances etc.)
    - Cyph0n 6y ago
      Yes, the QNX version (cXR) is approaching end-of-life afaik. The distributed architecture remains with the Linux version (eXR).
      - p_l 6y ago
        TIL. Thanks a lot :) is eXR used for the XRv?
- p_l 6y ago
  IOS-XE is essentially IOS Classic running as an application on top of Linux, communicating with the actual forwarding hardware. NX-OS is similar but without legacy of having been a standalone OS. Linux generally stays away from all networking work and just provides a nicer deployment target with nicer APIs than bare bones MIPS/PPC/x86
dahfizz 6y ago
A switch operating system doesn't do the actual networking stuff, all that happens in the switching ASIC. All the OS does is apply configurations and basic management tasks. If it runs on a $10 raspberry pi it will run fine on a $20,000 switch.
- Cyph0n 6y ago
  Well, that's not really the case with higher-scale router operating systems like IOS-XR. You have a ton of protocols running in software, so performance and memory requirements increase quite a bit. And then there are the high-availability (HA) requirements which typically lead to redundancy in software and hardware.
  - turtlecrush1337 6y ago
    Quite right. Control plane traffic is punted to the CPU, and a Raspi CPU cannot really handle that volume of traffic at enterprise scale.
deleted 6y ago
[deleted]
myrandomcomment 6y ago
Arista is Fedora. Routing stack started on NextHop (gated) which Arista bought in 2008 IIRC. Has been re-written majorly since. Cisco NXOS is Yocto. They planned to move to Fedora at some point. Might have by now. Cumulus is Debian based. switchd is closed sourced ASIC driver. Junos is now a FreeBSD VM running on a Linux boot not sure what version. Sonic is Debian based IIRC. OS10 (Dell / Force10 was NextBSD) but I think with OPX (open source OS10 that SONiC will replace - personal option) moved to Linux. FoundryOS was custom (VXworks?) Current version is Broadcom Strata. Extreme original was VXworks. Current is Linux based. Cisco IOS XE/XR is Linux (Debian IIRC). SwtichLight is Linux as well as the BSNOS with their open flow stuff on top. Ubiquiti is Vyatta running on Linux. That is a quick dump from meat cache.
musicale 6y ago
> Linux seems like it would be a bit heavy for a switch Linux (and other Unix or Unix-like) kernels (and indeed full OS distributions) run fine on many low-end and embedded CPUs and hardware, and network switches are no exception. OpenWRT is Linux-based and runs on extremely low-end switches such as home routers and access points. Arista EOS is based on Fedora. (Of course Arista switches have real server CPUs and lots of memory. People do crazy things like running KVM on them.) Juniper's Junos is based on BSD. Remember that on a high-speed switch packets usually pass through the switching hardware without touching the switch CPU. Programmable switching chips like Tofino typically run pre-compiled pipelines that execute on-chip at line rate. The switch OS is primarily used for running management software that programs the hardware, runs the CLI, and/or provides other services. The OS can also run software to provide higher-level protocols and services such as BGP, DNS, or DHCP.

wmf 6y ago

SONIC is less mature but it's free and open source with the benefits and drawbacks that entails. For example, SONIC runs on a variety of different hardware so you could take advantage of that diversity without having to learn different NOSes. Ten years ago switches were using 800 MHz single-core PowerPCs which was adequate to run Linux (although many were using VxWorks or whatever). Now the $400 switches are still wimpy but more expensive "disaggregated" switches are using Atoms or low-end Xeons.

iptrans 6y ago

Adding to what wmf said, JunOS is based on BSD and JunoOS evolved will be based on Linux. Most network equipment run their control plane using standard Intel CPUs, so running Linux isn't much of a strech.

monocasa 6y ago

Eh, it's probably not that bad. On the low end, Linux isn't a slouch with XDP, etc. On the high end, it's game over if your kernel is in the data plane at all anyway, and Linux is a great option for the control plane of a switch.

tw04 6y ago

Cisco is already Linux based. IOS-XE and NX-OS. https://en.m.wikipedia.org/wiki/Cisco_IOS_XE https://en.m.wikipedia.org/wiki/Cisco_IOS_XE https://en.m.wikipedia.org/wiki/Cisco_NX-OS https://en.m.wikipedia.org/wiki/Cisco_NX-OS

12bits 6y ago
I like this comparison https://medium.com/@Meela349588204/using-cows-to-explain-the-differences-of-the-cisco-operating-systems-876d14145827 https://medium.com/@Meela349588204/using-cows-to-explain-the...
- myrandomcomment 6y ago
  If you want to enjoy even more pain, ah trivia I mean, NXOS and NXOS are not the same thing. Read it again. There is an NXOS train for the original 2/5/7K and current kit in that line, and there is another version for the 3/9K new kit from the ACI spin in. They are just different enough that it makes things difficult when you are trying to develop tools for them. Cisco has promised to merge the two for a long time now.
Cyph0n 6y ago
IOS-XR is also Linux-based.
- p_l 6y ago
  IOS-XR at least used to be QNX based, with somewhat distributed architecture (line cards running separate OS instances etc.)
  - Cyph0n 6y ago
    Yes, the QNX version (cXR) is approaching end-of-life afaik. The distributed architecture remains with the Linux version (eXR).
    - p_l 6y ago
      TIL. Thanks a lot :) is eXR used for the XRv?
p_l 6y ago
IOS-XE is essentially IOS Classic running as an application on top of Linux, communicating with the actual forwarding hardware. NX-OS is similar but without legacy of having been a standalone OS. Linux generally stays away from all networking work and just provides a nicer deployment target with nicer APIs than bare bones MIPS/PPC/x86

dahfizz 6y ago

A switch operating system doesn't do the actual networking stuff, all that happens in the switching ASIC. All the OS does is apply configurations and basic management tasks. If it runs on a $10 raspberry pi it will run fine on a $20,000 switch.

Cyph0n 6y ago
Well, that's not really the case with higher-scale router operating systems like IOS-XR. You have a ton of protocols running in software, so performance and memory requirements increase quite a bit. And then there are the high-availability (HA) requirements which typically lead to redundancy in software and hardware.
- turtlecrush1337 6y ago
  Quite right. Control plane traffic is punted to the CPU, and a Raspi CPU cannot really handle that volume of traffic at enterprise scale.

deleted 6y ago

[deleted]

myrandomcomment 6y ago

Arista is Fedora. Routing stack started on NextHop (gated) which Arista bought in 2008 IIRC. Has been re-written majorly since. Cisco NXOS is Yocto. They planned to move to Fedora at some point. Might have by now. Cumulus is Debian based. switchd is closed sourced ASIC driver. Junos is now a FreeBSD VM running on a Linux boot not sure what version. Sonic is Debian based IIRC. OS10 (Dell / Force10 was NextBSD) but I think with OPX (open source OS10 that SONiC will replace - personal option) moved to Linux. FoundryOS was custom (VXworks?) Current version is Broadcom Strata. Extreme original was VXworks. Current is Linux based. Cisco IOS XE/XR is Linux (Debian IIRC). SwtichLight is Linux as well as the BSNOS with their open flow stuff on top. Ubiquiti is Vyatta running on Linux. That is a quick dump from meat cache.

musicale 6y ago

> Linux seems like it would be a bit heavy for a switch Linux (and other Unix or Unix-like) kernels (and indeed full OS distributions) run fine on many low-end and embedded CPUs and hardware, and network switches are no exception. OpenWRT is Linux-based and runs on extremely low-end switches such as home routers and access points. Arista EOS is based on Fedora. (Of course Arista switches have real server CPUs and lots of memory. People do crazy things like running KVM on them.) Juniper's Junos is based on BSD. Remember that on a high-speed switch packets usually pass through the switching hardware without touching the switch CPU. Programmable switching chips like Tofino typically run pre-compiled pipelines that execute on-chip at line rate. The switch OS is primarily used for running management software that programs the hardware, runs the CLI, and/or provides other services. The OS can also run software to provide higher-level protocols and services such as BGP, DNS, or DHCP.