4 ms·

Uh, no. MPLS has had SOLID ASIC support for maybe two decades now. Everyone who is putting in VxLAN-capable hardware is doing so because they have oodles of E

by Nrsolis 11y ago

Uh, no. MPLS has had SOLID ASIC support for maybe two decades now. Everyone who is putting in VxLAN-capable hardware is doing so because they have oodles of ESXi deployed and VxLAN is the only technology that's supported by VMware. If you're not wedded to the ESXi hypervisor, you can deploy either MPLS or MPLSoverGREoverIP (for your non-MPLS capable endpoints) and capture a whole stack of goodness to include interworking across an entire WAN infrastructure for DC-to-DC VM movement and EVPN support. MPLS has been doing L2VPN support for ages. Man, NSX/VxLAN didn't even have a real control plane until recently. They were going to use MULTICAST for their table updates! MULTICAST!

jpgvm 11y ago
So much FUD. First of all VxLAN is already nicely offloaded by the majority of NICs and if not being terminated doesn't required hardware support in switching equipment. VxLAN is almost no lock-in. Currently to do MPLS in any shape or form you are going to be locked into proprietary network gear. MPLS has almost no software implementations, whereas Linux has native VxLAN support in vanilla bridging mode and OVS. Forget MPLS on Windows. MPLS is great tech don't get me wrong, however it's restricted to the carrier space because it doesn't play well with others.
- Nrsolis 11y ago
  You're thinking of a very narrow use-case for VxLAN. What happens when your VM needs to talk to legacy networking gear? What happens when your VM(s) need to talk to a legacy security infrastructure that's mandated because of regulatory concerns? What happens when you need to route between two VxLAN domains? You're going to need to provide the VxLAN VTEP functionality someplace and that requires hardware support in the network someplace. Waving your hands and saying "that's not a concern" won't cut it. I have a customer that's facing this issue RIGHT NOW. And no, MPLS isn't a proprietary standard. It's open and it's available on oodles of networking hardware. It's interoperable and it's a proven technology that scales. Developing a tunneling protocol is the easy part. Developing a scalable control plane to handle RIB and FIB state is another matter entirely. Getting all of that to operate on a chipset that can do 2-4Tbps per slot is another thing altogether. Go check out some of the HUGE cloud infrastructure guys. They aren't using BrandX networking hardware to build their data-centers. They're using the Big-3 players.
  - jpgvm 11y ago
    No, they are not using the Big-3 players. ISPs are using Alcatel and Juniper still sure, big enterprise shops are still swearing by Cisco. Big cloud infrastructure has all moved to Linux on merchant silicon, either stuff like Quanta, Pico8 or even more DIY like OCP. As for terminating VxLAN on legacy gear, yeah that is probably a bad idea and someone probably made a bad decision to end up with an architecture that requires that. As for control plane. Most people using VxLAN at scale have their own control planes. They are actually stupidly easy to build because the edge is so easy to work with. I built my own that integrates with the Linux native VxLAN implementation using netlink to program the forwarding table. At the end of the day VxLAN isn't a great replacement for MPLS but it's a great encapsulation system for fully software defined datacenters that have already invested into a full control plane for all compute and networking (think Mesos, Kubernetes). It's also a good choice for smaller scale stuff because it scales down nicely. Multicast forwarding might suck in a real DC situation but for a lot of newbies playing around it's a good way to get into doing L2oL3. I don't think the WAN cases matter that much. At the point where it's a problem you have the people around to make said problem go away. Specifically trying to do cross DC IP address mobility is dumb in the first place. Most models that actually work well cross datacenter simply terminate it in one place and bring it up the workload on an entirely new set of resources in the new DC. This is much easier and is shared nothing usually (except maybe the dataset, but that is usually stored in S3/HDFS/Ceph/other distributed store here). Long story short, it does it's job fine. Use it for something it wasn't built for and yes it will hurt you.
    - Nrsolis 11y ago
      You're downvoting me but you're so wrong on so many points I don't know where to begin. 1. Cisco, Juniper, Alcatel, Arista all have merchant silicon platforms. They marry those to their own control planes because the customers want that kind of continuity and support. Pico8 and others are a FRACTION of the market place for switching (including datacenter switching). My own company has so many racks full of gear at AMZN and GOOG that it's hard to understand why you feel like you know that infrastructure better. 2. Writing a scalable control plane isn't as easy as you are representing. GOOG took many years to write theirs and it's still problematic for them in anything in lots of their use cases. I don't know why you think that quite literally the entire Internet along with all of the protocol work that has built it is a drag on novel architectures. 3. Mesos/Kubernetes are vanishingly small parts of the larger ecosystems out there. You can't just wave you hands and disregard quite literally the billions of dollars of infrastructure that are deployed each and every year by major service providers and corporations. Tiny cloud providers are not the largest slice of the pie when it comes to dollars spent on networking gear right now. 4. "At the point where it's a problem you have the people around to make said problem go away." HUH???? I can think of maybe 8 or 9 use-cases off hand that absolutely REQUIRE this kind of functionality because billions of dollars of transactions are handled by the infrastructure within and between those datacenters. Sorry buddy. You're way wrong here. Either you've never built an infrastructure that anyone cares about losing for an hour or two while you figure out what went wrong or you're so tanked up on "cloud" kool-aid that you've forgotten how we got to this point and have failed to understand how large systems scale up from small ones. Show me a large multinational bank that's storing their transaction data on S3 or Ceph and I'll show you a bank that's not "systemically important". Most large enterprises don't have the luxury of simply discarding 100% of their working, proven architectures on the promise of a few small startups hoping to cash in big. Things like operational stability, redundancy, fault isolation, and monitoring are not "nice to haves", they are mission critical requirements. They aren't up to the whim of some bright-eyed CTO, but the watchful eye of umpteen nations of regulators.

by Nrsolis 11y ago

jpgvm 11y ago

So much FUD. First of all VxLAN is already nicely offloaded by the majority of NICs and if not being terminated doesn't required hardware support in switching equipment. VxLAN is almost no lock-in. Currently to do MPLS in any shape or form you are going to be locked into proprietary network gear. MPLS has almost no software implementations, whereas Linux has native VxLAN support in vanilla bridging mode and OVS. Forget MPLS on Windows. MPLS is great tech don't get me wrong, however it's restricted to the carrier space because it doesn't play well with others.

Nrsolis 11y ago
You're thinking of a very narrow use-case for VxLAN. What happens when your VM needs to talk to legacy networking gear? What happens when your VM(s) need to talk to a legacy security infrastructure that's mandated because of regulatory concerns? What happens when you need to route between two VxLAN domains? You're going to need to provide the VxLAN VTEP functionality someplace and that requires hardware support in the network someplace. Waving your hands and saying "that's not a concern" won't cut it. I have a customer that's facing this issue RIGHT NOW. And no, MPLS isn't a proprietary standard. It's open and it's available on oodles of networking hardware. It's interoperable and it's a proven technology that scales. Developing a tunneling protocol is the easy part. Developing a scalable control plane to handle RIB and FIB state is another matter entirely. Getting all of that to operate on a chipset that can do 2-4Tbps per slot is another thing altogether. Go check out some of the HUGE cloud infrastructure guys. They aren't using BrandX networking hardware to build their data-centers. They're using the Big-3 players.
- jpgvm 11y ago
  No, they are not using the Big-3 players. ISPs are using Alcatel and Juniper still sure, big enterprise shops are still swearing by Cisco. Big cloud infrastructure has all moved to Linux on merchant silicon, either stuff like Quanta, Pico8 or even more DIY like OCP. As for terminating VxLAN on legacy gear, yeah that is probably a bad idea and someone probably made a bad decision to end up with an architecture that requires that. As for control plane. Most people using VxLAN at scale have their own control planes. They are actually stupidly easy to build because the edge is so easy to work with. I built my own that integrates with the Linux native VxLAN implementation using netlink to program the forwarding table. At the end of the day VxLAN isn't a great replacement for MPLS but it's a great encapsulation system for fully software defined datacenters that have already invested into a full control plane for all compute and networking (think Mesos, Kubernetes). It's also a good choice for smaller scale stuff because it scales down nicely. Multicast forwarding might suck in a real DC situation but for a lot of newbies playing around it's a good way to get into doing L2oL3. I don't think the WAN cases matter that much. At the point where it's a problem you have the people around to make said problem go away. Specifically trying to do cross DC IP address mobility is dumb in the first place. Most models that actually work well cross datacenter simply terminate it in one place and bring it up the workload on an entirely new set of resources in the new DC. This is much easier and is shared nothing usually (except maybe the dataset, but that is usually stored in S3/HDFS/Ceph/other distributed store here). Long story short, it does it's job fine. Use it for something it wasn't built for and yes it will hurt you.
  - Nrsolis 11y ago
    You're downvoting me but you're so wrong on so many points I don't know where to begin. 1. Cisco, Juniper, Alcatel, Arista all have merchant silicon platforms. They marry those to their own control planes because the customers want that kind of continuity and support. Pico8 and others are a FRACTION of the market place for switching (including datacenter switching). My own company has so many racks full of gear at AMZN and GOOG that it's hard to understand why you feel like you know that infrastructure better. 2. Writing a scalable control plane isn't as easy as you are representing. GOOG took many years to write theirs and it's still problematic for them in anything in lots of their use cases. I don't know why you think that quite literally the entire Internet along with all of the protocol work that has built it is a drag on novel architectures. 3. Mesos/Kubernetes are vanishingly small parts of the larger ecosystems out there. You can't just wave you hands and disregard quite literally the billions of dollars of infrastructure that are deployed each and every year by major service providers and corporations. Tiny cloud providers are not the largest slice of the pie when it comes to dollars spent on networking gear right now. 4. "At the point where it's a problem you have the people around to make said problem go away." HUH???? I can think of maybe 8 or 9 use-cases off hand that absolutely REQUIRE this kind of functionality because billions of dollars of transactions are handled by the infrastructure within and between those datacenters. Sorry buddy. You're way wrong here. Either you've never built an infrastructure that anyone cares about losing for an hour or two while you figure out what went wrong or you're so tanked up on "cloud" kool-aid that you've forgotten how we got to this point and have failed to understand how large systems scale up from small ones. Show me a large multinational bank that's storing their transaction data on S3 or Ceph and I'll show you a bank that's not "systemically important". Most large enterprises don't have the luxury of simply discarding 100% of their working, proven architectures on the promise of a few small startups hoping to cash in big. Things like operational stability, redundancy, fault isolation, and monitoring are not "nice to haves", they are mission critical requirements. They aren't up to the whim of some bright-eyed CTO, but the watchful eye of umpteen nations of regulators.