4 ms·

I think with platforms like this it is now safe to say that the systems and services Google is deploying are no longer in the same category as classical network

by dkhenry 10y ago

I think with platforms like this it is now safe to say that the systems and services Google is deploying are no longer in the same category as classical networked systems. This is as foreign a concept from traditional networking and the seven layer OSI model as non von Neumann computing is from von neumann computing

insaneirish 10y ago
> This is as foreign a concept from traditional networking and the seven layer OSI model as non von Neumann computing is from von neumann computing Not really. The OSI model doesn't say anything about where I run my routing algorithm and BGP application vs. where my actual switches are. "Classical" networking is an artifact of viewing routers/switches as monolithic blocks that embed all of their functionality in one black box. I said BGP application above because that's what it is, an application for distributing/communicating state. The same can be said for many other parts of networking traditionally embedded in the monolithic blob we often call a router. Label switched fabrics provide inherent NFV, security functions, and allow you to influence paths (i.e. traffic engineer) from applications that are equipped to make decisions based on your priorities, not some rigid vendor implementation. You will see more of this.
- tyingq 10y ago
  I can't speak the the edge stuff they are doing, but the routing closer to the data centers was unique to the way they do things. In a way that wouldn't be generally usable. Not every application can put enough context in the request to make it work the way Google is working. Sort of app request context based routing. Also, they have the advantage that a lot of their apps are like search, in that no consistency is needed. Five consecutive searches for "some query" can return different results each time, with no adverse effects. That creates a lot of flexibility in routing requests to destinations.
- NetStrikeForce 10y ago
  As someone who works with SDN daily I couldn't agree more with you. Espresso is making decisions on how to use the application layer, but the underlying layers stay the same.
- walrus01 10y ago
  > The OSI model doesn't say anything about where I run my routing algorithm and BGP application vs. where my actual switches are. If you're $BIGASN and you set up an intra building singlemode crossconnect at $BIGCITY to establish settlement free peering (let's say for example a 4 x 10 Gbps bonded 802.3ad circuit) with $OTHERBIGASN, they most assuredly are going to notice if your BGP session and router is not directly on the other end of that cable. Because they are going to be expecting sub-1ms latency to your router, and not "we're taking this session and stuffing it in some sort of tunnel or encapsulation and sending it somewhere else, to where the thing that actually speaks BGP is located". It's bad juju to practice deceptive peering.
  - insaneirish 10y ago
    > Because they are going to be expecting sub-1ms latency to your router, and not "we're taking this session and stuffing it in some sort of tunnel or encapsulation and sending it somewhere else, to where the thing that actually speaks BGP is located". Why should they care? > It's bad juju to practice deceptive peering. I don't understand applying moral judgment to a technical design choice.
    - walrus01 10y ago
      I'm guessing you do not handle peering for a medium to large sized AS, so it's really hard to explain. First: they should care because the point of establishing peering in a given city is to give inter-AS traffic the absolute shortest path and shortest number of hops between two points. If I put a router in portland, OR and buy a 10 Gbps mpls tunnel to Vancouver BC and join the vanix, ask to set up with peers there, all traffic will be taking a multi hundred km round trip to Portland. Two: it's not moral judgment, it's a technical best practice to actually put routers in the city in which you set up new edge BGP sessions. Pretty basic ISP stuff in fact.
      - londons_explore 10y ago
        Just because they tunnel the BGP back to some software system to make routing decisions doesn't mean they tunnel all the user data to that same location. To do so would be silly. In fact, a good system would have a couple of systems handling BGP, with physical location fairly irrelevant, but acting as if they are local to the peer they are talking to.
joshbaptiste 10y ago
Yep.. the CEO of Fastly has some cool presentations on YouTube of their SDN configuration which does away with the traditional router/switch combination for the switch/SDN server combo, which provides a much smarter network at a fraction of the cost. The only caveat is your engineers need to understand and tube the SDN software stack from the network card to the kernel.
- pas 10y ago
  Even though this reply is 6 days late, I finally had time to watch the mentioned talk, and it's interesting and still very relevant (it's from 2015): https://www.youtube.com/watch?v=TLbzvbfWmfY https://www.youtube.com/watch?v=TLbzvbfWmfY interesting stuff starts around 7 minutes
walrus01 10y ago
> This is as foreign a concept from traditional networking and the seven layer OSI model cough cough what? One of the major challenges for a CDN is predicated upon OSI layers 1 and 2: You need to establish POPs with routers and caching servers geographically distributed near major IX points (L2 peering fabrics, and crucial buildings that host the same IX points, where you can run intra-building fiber crossconnects for network-to-network interfaces to provide settlement free peering to major ISPs). The internet is physically built out of a great deal of equipment at layer 1. In the case of Google, you need to have a team of people who care about things like cost-effectively building intra-datacenter 100GbE layer 2 connections between Google, and large content sinks (eyeball) ISPs such as Charter/TWTC or similar. Hand waving around and saying "we've built some new software to improve how we efficiently deliver BGP sessions to edge peers" is cool and all, but don't mistake it for some radical change. It is all still built on top of things like 2 megawatt diesel generators, massive battery plants, DWDM line terminals, dark fiber, etc.
- nickpsecurity 10y ago
  In other words, it's built on hundreds of millions to billions of dollars of typical, boring stuff on the bottom that allows all the cool stuff to work as well as it does.
  - jodrellblank 10y ago
    that allows all the cool stuff to work as well as it does. that allows all the advertising to draw in ad spend as well as it does.
  - smueller1234 10y ago
    It's really easily billions, not hundreds of millions. I work for a much smaller outfit and we (handwave) push the smaller number already.

by dkhenry 10y ago

insaneirish 10y ago

> This is as foreign a concept from traditional networking and the seven layer OSI model as non von Neumann computing is from von neumann computing Not really. The OSI model doesn't say anything about where I run my routing algorithm and BGP application vs. where my actual switches are. "Classical" networking is an artifact of viewing routers/switches as monolithic blocks that embed all of their functionality in one black box. I said BGP application above because that's what it is, an application for distributing/communicating state. The same can be said for many other parts of networking traditionally embedded in the monolithic blob we often call a router. Label switched fabrics provide inherent NFV, security functions, and allow you to influence paths (i.e. traffic engineer) from applications that are equipped to make decisions based on your priorities, not some rigid vendor implementation. You will see more of this.

tyingq 10y ago
I can't speak the the edge stuff they are doing, but the routing closer to the data centers was unique to the way they do things. In a way that wouldn't be generally usable. Not every application can put enough context in the request to make it work the way Google is working. Sort of app request context based routing. Also, they have the advantage that a lot of their apps are like search, in that no consistency is needed. Five consecutive searches for "some query" can return different results each time, with no adverse effects. That creates a lot of flexibility in routing requests to destinations.
NetStrikeForce 10y ago
As someone who works with SDN daily I couldn't agree more with you. Espresso is making decisions on how to use the application layer, but the underlying layers stay the same.
walrus01 10y ago
> The OSI model doesn't say anything about where I run my routing algorithm and BGP application vs. where my actual switches are. If you're $BIGASN and you set up an intra building singlemode crossconnect at $BIGCITY to establish settlement free peering (let's say for example a 4 x 10 Gbps bonded 802.3ad circuit) with $OTHERBIGASN, they most assuredly are going to notice if your BGP session and router is not directly on the other end of that cable. Because they are going to be expecting sub-1ms latency to your router, and not "we're taking this session and stuffing it in some sort of tunnel or encapsulation and sending it somewhere else, to where the thing that actually speaks BGP is located". It's bad juju to practice deceptive peering.
- insaneirish 10y ago
  > Because they are going to be expecting sub-1ms latency to your router, and not "we're taking this session and stuffing it in some sort of tunnel or encapsulation and sending it somewhere else, to where the thing that actually speaks BGP is located". Why should they care? > It's bad juju to practice deceptive peering. I don't understand applying moral judgment to a technical design choice.
  - walrus01 10y ago
    I'm guessing you do not handle peering for a medium to large sized AS, so it's really hard to explain. First: they should care because the point of establishing peering in a given city is to give inter-AS traffic the absolute shortest path and shortest number of hops between two points. If I put a router in portland, OR and buy a 10 Gbps mpls tunnel to Vancouver BC and join the vanix, ask to set up with peers there, all traffic will be taking a multi hundred km round trip to Portland. Two: it's not moral judgment, it's a technical best practice to actually put routers in the city in which you set up new edge BGP sessions. Pretty basic ISP stuff in fact.
    - londons_explore 10y ago
      Just because they tunnel the BGP back to some software system to make routing decisions doesn't mean they tunnel all the user data to that same location. To do so would be silly. In fact, a good system would have a couple of systems handling BGP, with physical location fairly irrelevant, but acting as if they are local to the peer they are talking to.

joshbaptiste 10y ago

Yep.. the CEO of Fastly has some cool presentations on YouTube of their SDN configuration which does away with the traditional router/switch combination for the switch/SDN server combo, which provides a much smarter network at a fraction of the cost. The only caveat is your engineers need to understand and tube the SDN software stack from the network card to the kernel.

pas 10y ago
Even though this reply is 6 days late, I finally had time to watch the mentioned talk, and it's interesting and still very relevant (it's from 2015): https://www.youtube.com/watch?v=TLbzvbfWmfY https://www.youtube.com/watch?v=TLbzvbfWmfY interesting stuff starts around 7 minutes

walrus01 10y ago

> This is as foreign a concept from traditional networking and the seven layer OSI model cough cough what? One of the major challenges for a CDN is predicated upon OSI layers 1 and 2: You need to establish POPs with routers and caching servers geographically distributed near major IX points (L2 peering fabrics, and crucial buildings that host the same IX points, where you can run intra-building fiber crossconnects for network-to-network interfaces to provide settlement free peering to major ISPs). The internet is physically built out of a great deal of equipment at layer 1. In the case of Google, you need to have a team of people who care about things like cost-effectively building intra-datacenter 100GbE layer 2 connections between Google, and large content sinks (eyeball) ISPs such as Charter/TWTC or similar. Hand waving around and saying "we've built some new software to improve how we efficiently deliver BGP sessions to edge peers" is cool and all, but don't mistake it for some radical change. It is all still built on top of things like 2 megawatt diesel generators, massive battery plants, DWDM line terminals, dark fiber, etc.

nickpsecurity 10y ago
In other words, it's built on hundreds of millions to billions of dollars of typical, boring stuff on the bottom that allows all the cool stuff to work as well as it does.
- jodrellblank 10y ago
  that allows all the cool stuff to work as well as it does. that allows all the advertising to draw in ad spend as well as it does.
- smueller1234 10y ago
  It's really easily billions, not hundreds of millions. I work for a much smaller outfit and we (handwave) push the smaller number already.