7 ms·

I have to disagree with the primary point being offered up in this post. In my career I’ve met quite a few network engineers who don’t think sufficiently outsi

by jgowdy 9y ago

I have to disagree with the primary point being offered up in this post. In my career I’ve met quite a few network engineers who don’t think sufficiently outside the box. They typically say “this is how it’s done, this is how it works.” The idea that we can’t come up with a better protocol than BGP to manage the interconnections of the Internet is absurd. We don’t say “well security isn’t a programming problem, it’s a human problem, so there’s no point in making safer programming languages.” BGP is a ridiculous system of complete trust just like many older protocols. If we sat down and started designing a routing protocol to handle the advertisement of routing tables on the internet, would we come up with BGP again? Would the new protocol potentially offer up fixes for some of the most common and obvious problems with BGP? Then there’s potential for improvement. The network industry resists and slows the horribly needed disruption in several key areas through the dogmatic defense of the status quo that we typically hear from its constituents. This is why so many of us pray for SDN. It’s the only way to wrest control of networking away from those who typically have dogmatic vendor based training rather than a “network science and theory” education that leads to innovation.

d33 9y ago
> BGP is a ridiculous system of complete trust just like many older protocols. This sounds related: https://security.stackexchange.com/q/56069/15648 https://security.stackexchange.com/q/56069/15648
jtiri1883 9y ago
I look forward to reading the first draft of your groups proposal
ra1n85 9y ago
>This is why so many of us pray for SDN. It’s the only way to wrest control of networking Why not take the standards approach and draft a protocol spec? At some point, you'll need to interoperate with those horribly dogmatic vendor hugging networks that serve the majority of traffic on the internet. >who typically have dogmatic vendor based training rather than a “network science and theory” education that leads to innovation. I've seen this as well, and it can be frustrating. Part of me believes this is a defense mechanism used to prevent developers from pushing problems into the network. The other part believes that network engineering has drawn individuals to the field based on earnings potential rather than interest in the work, and the talent just isn't there. Both are anecdotal, so grain of salt and all.
- jgowdy 9y ago
  > At some point, you'll need to interoperate with those horribly dogmatic vendor hugging networks that serve the majority of traffic on the internet. And this has been their defense all along. It serves the status quo to imply that transitions must be difficult. IPv6 being the most obvious example today. Of course as DJB pointed out, backwards compatible IPv6 would have already rolled out by now, at the cost of a single /64. Dual stack was literally the most difficult way to accomplish IPv6. Now let’s look at the BGP situation. As long as the outcome is the same (same routing tables result in nominal situations), any two ISPs could choose to use a different protocol for their interconnection. Same thing that has happened with a lot of supplemental networking protocols. There are BGP alternatives on deck. What we don’t need is high profile network engineers speaking out against them and in favor of the status quo. The truth is, networking is the one part of the industry that isn’t improving at the same pace as the rest. And all the improvements that are happening seem to be related to single links and switching, whereas the defense of networking dogma is related to routing. According to the holy bible of networking, switching is easy, routing is hard. Routers must be memory restricted with small memory footprints to artificially create SKUs. Routers must operate on multi-minute timers rather than seconds. Building routing tables is an insanely intense and time consuming process. Blah blah blah. The reality is, almost all of these problems are contrived to maintain the status quo and so network vendors can ignore the pace of hardware and instead set SKUs at resource levels that force you to use the “appropriate” piece of hardware for the appropriate task. Another smell that should make this obvious to everyone is that almost every major company has had high level engineers make some statement of how network companies are ripping you off, and explaining how they bought white box network solutions that route layer 3 at wire speed, converge rapidly, and have tons of memory. Not to mention the vast improvement something like well designed SDN can provide. The network companies rely on the fact that most companies don’t have the labor or expertise to use white label solutions. But that doesn’t disprove the point. That proves the point that the state of hardware is “super cheap components provide all the switching and routing performance and memory needed for almost 99% of networking purposes.” Companies like Cisco are playing the Intel +25Mhz game. Supposedly disrupting companies do so at an awfully measured pace. And the network staff generally will open up to off brand switches but start to shy away at routing, especially edge routing. As an industry, networking needs to be disrupted and it needs a severe model change. People involved in networking need to get on board, and those who don’t and insist on fighting for the status quo need to be left behind.
convolvatron 9y ago
i can see where SDN might be applied to build your own machinery for inter domain routing, but how does it help in the broader internet where bgp is applied? sure, I guess if you developed a new mechanism for resolving transitive policy with baked in authentication, and got everyone to agree to use it, you wouldn't need to convince the router vendor. but it seems like the first two things are harder than the third.
- icedchai 9y ago
  The biggest problem with replacing BGP is the adoption. BGP is "good enough." What's the incentive? If you inject BGP routes into the new protocols, you'll have some of the same issues, right?
hhw 9y ago
Although this may very well be the mentality of many rank and file network engineers that you may come across, that wouldn't at all be an accurate representation of those in charge of the architecture and design of larger networks. However, even the more open minded engineers tend to be pragmatic, and there have already been numerous proposals or initiatives which are reached near consensus level of acceptance as good ideas such as BCP38, IRR generated prefix lists, etc., but still have abysmal adoption rates. Solutions already exist for most problems, with the ability to implement them on a case by case basis, which would be a much lesser barrier than upending BGP altogether. Throwing BGP out in lieu of a new protocol altogether has an order of magnitude even lower probability of seeing widespread adoption. What's lacking in many existing cases is simply (business) will. There are many businesses who will not allocate resources to make the Internet a better place if it doesn't improve their bottom line, and there are many more businesses who lack the resources to even be able to consider adopting anything beyond the bare minimum. At the heart of it, most carrier networks lose money most quarters, and smaller ISPs barely scrape by. It's only a handful of very large, oligopoly eyeball networks like Comcast that are quite profitable.
kanyawwah 9y ago
I'm a network engineer myself, and I can tell you your evaluation of the industry is pretty spot on. I believe a big part of the solution will be so-called white-box hardware. This will break the Cisco strangle-hold and force network operators to have deeper technical knowledge.
- zero_intp 9y ago
  As an NSP architect I can tell you the hardest resource to procure is experienced talent that can operate a production network without trying to move fast and break things. Buying hardware with usable software is pretty critical when dealing with stateful services.
mino 9y ago
> In my career I’ve met quite a few network engineers who don’t think sufficiently outside the box. They typically say “this is how it’s done, this is how it works.” Completely agree, it is my experience as well. I believe this is because, at last historically, very few network engineers are interested in software. This is slowly changing, but it will take "a generation".

by jgowdy 9y ago

d33 9y ago

> BGP is a ridiculous system of complete trust just like many older protocols. This sounds related: https://security.stackexchange.com/q/56069/15648 https://security.stackexchange.com/q/56069/15648

jtiri1883 9y ago

I look forward to reading the first draft of your groups proposal

ra1n85 9y ago

>This is why so many of us pray for SDN. It’s the only way to wrest control of networking Why not take the standards approach and draft a protocol spec? At some point, you'll need to interoperate with those horribly dogmatic vendor hugging networks that serve the majority of traffic on the internet. >who typically have dogmatic vendor based training rather than a “network science and theory” education that leads to innovation. I've seen this as well, and it can be frustrating. Part of me believes this is a defense mechanism used to prevent developers from pushing problems into the network. The other part believes that network engineering has drawn individuals to the field based on earnings potential rather than interest in the work, and the talent just isn't there. Both are anecdotal, so grain of salt and all.

jgowdy 9y ago
> At some point, you'll need to interoperate with those horribly dogmatic vendor hugging networks that serve the majority of traffic on the internet. And this has been their defense all along. It serves the status quo to imply that transitions must be difficult. IPv6 being the most obvious example today. Of course as DJB pointed out, backwards compatible IPv6 would have already rolled out by now, at the cost of a single /64. Dual stack was literally the most difficult way to accomplish IPv6. Now let’s look at the BGP situation. As long as the outcome is the same (same routing tables result in nominal situations), any two ISPs could choose to use a different protocol for their interconnection. Same thing that has happened with a lot of supplemental networking protocols. There are BGP alternatives on deck. What we don’t need is high profile network engineers speaking out against them and in favor of the status quo. The truth is, networking is the one part of the industry that isn’t improving at the same pace as the rest. And all the improvements that are happening seem to be related to single links and switching, whereas the defense of networking dogma is related to routing. According to the holy bible of networking, switching is easy, routing is hard. Routers must be memory restricted with small memory footprints to artificially create SKUs. Routers must operate on multi-minute timers rather than seconds. Building routing tables is an insanely intense and time consuming process. Blah blah blah. The reality is, almost all of these problems are contrived to maintain the status quo and so network vendors can ignore the pace of hardware and instead set SKUs at resource levels that force you to use the “appropriate” piece of hardware for the appropriate task. Another smell that should make this obvious to everyone is that almost every major company has had high level engineers make some statement of how network companies are ripping you off, and explaining how they bought white box network solutions that route layer 3 at wire speed, converge rapidly, and have tons of memory. Not to mention the vast improvement something like well designed SDN can provide. The network companies rely on the fact that most companies don’t have the labor or expertise to use white label solutions. But that doesn’t disprove the point. That proves the point that the state of hardware is “super cheap components provide all the switching and routing performance and memory needed for almost 99% of networking purposes.” Companies like Cisco are playing the Intel +25Mhz game. Supposedly disrupting companies do so at an awfully measured pace. And the network staff generally will open up to off brand switches but start to shy away at routing, especially edge routing. As an industry, networking needs to be disrupted and it needs a severe model change. People involved in networking need to get on board, and those who don’t and insist on fighting for the status quo need to be left behind.

convolvatron 9y ago

i can see where SDN might be applied to build your own machinery for inter domain routing, but how does it help in the broader internet where bgp is applied? sure, I guess if you developed a new mechanism for resolving transitive policy with baked in authentication, and got everyone to agree to use it, you wouldn't need to convince the router vendor. but it seems like the first two things are harder than the third.

icedchai 9y ago
The biggest problem with replacing BGP is the adoption. BGP is "good enough." What's the incentive? If you inject BGP routes into the new protocols, you'll have some of the same issues, right?

hhw 9y ago

Although this may very well be the mentality of many rank and file network engineers that you may come across, that wouldn't at all be an accurate representation of those in charge of the architecture and design of larger networks. However, even the more open minded engineers tend to be pragmatic, and there have already been numerous proposals or initiatives which are reached near consensus level of acceptance as good ideas such as BCP38, IRR generated prefix lists, etc., but still have abysmal adoption rates. Solutions already exist for most problems, with the ability to implement them on a case by case basis, which would be a much lesser barrier than upending BGP altogether. Throwing BGP out in lieu of a new protocol altogether has an order of magnitude even lower probability of seeing widespread adoption. What's lacking in many existing cases is simply (business) will. There are many businesses who will not allocate resources to make the Internet a better place if it doesn't improve their bottom line, and there are many more businesses who lack the resources to even be able to consider adopting anything beyond the bare minimum. At the heart of it, most carrier networks lose money most quarters, and smaller ISPs barely scrape by. It's only a handful of very large, oligopoly eyeball networks like Comcast that are quite profitable.

kanyawwah 9y ago

I'm a network engineer myself, and I can tell you your evaluation of the industry is pretty spot on. I believe a big part of the solution will be so-called white-box hardware. This will break the Cisco strangle-hold and force network operators to have deeper technical knowledge.

zero_intp 9y ago
As an NSP architect I can tell you the hardest resource to procure is experienced talent that can operate a production network without trying to move fast and break things. Buying hardware with usable software is pretty critical when dealing with stateful services.

mino 9y ago

> In my career I’ve met quite a few network engineers who don’t think sufficiently outside the box. They typically say “this is how it’s done, this is how it works.” Completely agree, it is my experience as well. I believe this is because, at last historically, very few network engineers are interested in software. This is slowly changing, but it will take "a generation".