3 ms·

Many of the things Netflix does here google does too and has been doing for a long time like GGC (Google Global Cache) for free at ISP NOC or datacenters - thei

by pavs 9y ago

Many of the things Netflix does here google does too and has been doing for a long time like GGC (Google Global Cache) for free at ISP NOC or datacenters - their GGC nodes are many orders of magnitude time is distribution. 90% of the Google traffic through these devices are youtube videos. They mention it in the article like Netflix pioneered this idea with open-connect. If anything it's harder to get OC devices from Netflix then it is from Google. Even Akamai does have a similar program where they let you put their CDN on ISP NOC.

amelius 9y ago
That sounds a bit like a violation of net neutrality avant la lettre.
- pavs 9y ago
  There is a whole world outside the United States.
- virtuallynathan 9y ago
  It's good for everyone -- if you are an ISP with 35%+ of your traffic coming from Netflix, you might be begging for these boxes in your network.
  - pavs 9y ago
    Yup, as an ISP owner, I can tell you 70-75% of my traffic come through GGC, for every 1gbps traffic from cache nodes 100mbps on average is pulled from the internet. Even with GGC nodes, it's still very expensive, BW-wise.
    - PuffinBlue 9y ago
      I saw another comment on here in the distant past that said an ISP owner decided to pull out the GGC boxes because they would sync vast amounts of data (largely YouTube IIRC) and it wasn't worth it bandwdith wise for them. It sounds like 10:1 is a decent saving but still objectively a lot of data.
      - pavs 9y ago
        You can't survive as an ISP if you don't have GGC nodes or directly peered with google DC. If your customer wants to watch youtube videos and you serve them through internet only, which will either mean you will have to buy more BW (not financially sustainable) or you will have to cap youtube traffic resulting in inferior user experience and a customer leaving your service. Or just ban youtube - might as well stop running an ISP. The best solution is to peer with their DC - let them do the heavy lifting and you can pull from their cache, then you can chuck the GGC nodes. Google is building a small DC close to our NOC - which means lots of saving for us and a better experience for our customers. We can see us chucking the nodes or in this case, we most likely have to give it back to them or something - they are very expensive and large supermicro servers. 10:1 is not a lot if you are serving a lot of traffic. For 50gbps, thats 5gbps bw - that's a lot of bw for free service - we don't cap youtube traffic (users likely to watch high res 1080p, 4k videos nowadays), we can't sell youtube traffic (license agreement from google) - so as far as ISPs are concerned its pissing away money. To be honest services like youtube, facebook and Netflix, even with their on-premise cache solution are a huge headache and expense for ISPs - it's not just the BW, rack-space, distribution switches, power, they all add up. There is a reason why Google with all their expertise, knowledge, money and influence can't even make a dent in their ISP business - because even with all their money - it insanely expensive when you try to cover most of the country.
        foepys 9y ago
        > You can't survive as an ISP if you don't have GGC nodes or directly peered with google DC. Believe me, you can. Deutsche Telekom, Germany's AT&T and Europe's biggest ISP, does this and in the evening YouTube videos regularly stutter on FullHD. But the customers just don't care enough.
        pavs 9y ago
        If all the big players do it, you really don't have a choice.
Terretta 9y ago
You react like Google pioneered these ideas. Many of the things Netflix does here, Google does too... but VDNs did before YouTube was a twinkle in Chad, Steve, and Jawed’s eye. The custom appliance proactive multi-tier video placement process outlined here, along with more techniques not mentioned (some of the most significant are only just now being rediscovered by non-CDNs such as Netflix or even AWS), were generally invented and deployed between 1998 and 2003 - 2005.
- pavs 9y ago
  My beef was not more like "who did it first", I honestly don't know and I couldn't find a reference online of anyone did something like this at this scale before google/youtube/akamai. The Point is the global scale of this on-premise network and the amount of data they move. I think Netflix is a blip compare to youtube/google/Akamai/facebook (I recently learned facebook also has a similar program, but I think they also use Akamai), on a global scale. I didn't like that glossed on over this as if Netflix is doing something exceptional or even unique. Obviously, it takes some serious engineering chops scale this amount of traffic globally and routes the traffic through on-premise nodes. But at this point, it's not exceptional - if anything its commodity.
  - Terretta 9y ago
    Netflix isn't a blip. For instance: https://fortunedotcom.files.wordpress.com/2015/10/apps-by-internet-traffic.png https://fortunedotcom.files.wordpress.com/2015/10/apps-by-in... Still true in 2017: https://techcrunch.com/2017/07/05/u-s-cord-cutters-watch-more-netflix-than-amazon-video-hulu-and-youtube-combined/ https://techcrunch.com/2017/07/05/u-s-cord-cutters-watch-mor... Regardless, long form video streamed globally without buffering is incredibly difficult. Short form clips are drastically easier. Do the math on how much content at what quality can fit in a given cache, or how long it takes to swap a piece of content in a cache, you'll start to see why. It's surprising to me that streaming well is still not a commodity. Even less so if you consider transactional (paid) video, which is what Netflix does. For example, just recently, tech press talked about cable companies losing $10B a year from account sharing. This was a problem VDNs solved in late 90s / early 00s for streaming protocols, but the technique for handling this across a global footprint was sidelined when YouTube's "http" progressive downloads / pseudo streaming and 'ad supported' video became vogue. Now that streams need to act more like streams again, and people want to charge for the content itself again, the solution is again needed, and no http-based "commodity" CDN seems to have it sorted out. Netflix does. PS. A couple refs from before Netflix/Google/YouTube: - https://www.google.com/patents/US20010027491 https://www.google.com/patents/US20010027491 - https://www.google.com/patents/US20010047275 https://www.google.com/patents/US20010047275 - https://news.microsoft.com/2000/06/12/microsofts-new-digital-broadcast-manager-enables-next-wave-of-e-commerce/ https://news.microsoft.com/2000/06/12/microsofts-new-digital... Those were first iterations. Later iterations were more sophisticated, and not patented in order to not disclose.

by pavs 9y ago

amelius 9y ago

That sounds a bit like a violation of net neutrality avant la lettre.

pavs 9y ago
There is a whole world outside the United States.
virtuallynathan 9y ago
It's good for everyone -- if you are an ISP with 35%+ of your traffic coming from Netflix, you might be begging for these boxes in your network.
- pavs 9y ago
  Yup, as an ISP owner, I can tell you 70-75% of my traffic come through GGC, for every 1gbps traffic from cache nodes 100mbps on average is pulled from the internet. Even with GGC nodes, it's still very expensive, BW-wise.
  - PuffinBlue 9y ago
    I saw another comment on here in the distant past that said an ISP owner decided to pull out the GGC boxes because they would sync vast amounts of data (largely YouTube IIRC) and it wasn't worth it bandwdith wise for them. It sounds like 10:1 is a decent saving but still objectively a lot of data.
    - pavs 9y ago
      You can't survive as an ISP if you don't have GGC nodes or directly peered with google DC. If your customer wants to watch youtube videos and you serve them through internet only, which will either mean you will have to buy more BW (not financially sustainable) or you will have to cap youtube traffic resulting in inferior user experience and a customer leaving your service. Or just ban youtube - might as well stop running an ISP. The best solution is to peer with their DC - let them do the heavy lifting and you can pull from their cache, then you can chuck the GGC nodes. Google is building a small DC close to our NOC - which means lots of saving for us and a better experience for our customers. We can see us chucking the nodes or in this case, we most likely have to give it back to them or something - they are very expensive and large supermicro servers. 10:1 is not a lot if you are serving a lot of traffic. For 50gbps, thats 5gbps bw - that's a lot of bw for free service - we don't cap youtube traffic (users likely to watch high res 1080p, 4k videos nowadays), we can't sell youtube traffic (license agreement from google) - so as far as ISPs are concerned its pissing away money. To be honest services like youtube, facebook and Netflix, even with their on-premise cache solution are a huge headache and expense for ISPs - it's not just the BW, rack-space, distribution switches, power, they all add up. There is a reason why Google with all their expertise, knowledge, money and influence can't even make a dent in their ISP business - because even with all their money - it insanely expensive when you try to cover most of the country.
      - foepys 9y ago
        > You can't survive as an ISP if you don't have GGC nodes or directly peered with google DC. Believe me, you can. Deutsche Telekom, Germany's AT&T and Europe's biggest ISP, does this and in the evening YouTube videos regularly stutter on FullHD. But the customers just don't care enough.
        pavs 9y ago
        If all the big players do it, you really don't have a choice.

Terretta 9y ago

You react like Google pioneered these ideas. Many of the things Netflix does here, Google does too... but VDNs did before YouTube was a twinkle in Chad, Steve, and Jawed’s eye. The custom appliance proactive multi-tier video placement process outlined here, along with more techniques not mentioned (some of the most significant are only just now being rediscovered by non-CDNs such as Netflix or even AWS), were generally invented and deployed between 1998 and 2003 - 2005.

pavs 9y ago
My beef was not more like "who did it first", I honestly don't know and I couldn't find a reference online of anyone did something like this at this scale before google/youtube/akamai. The Point is the global scale of this on-premise network and the amount of data they move. I think Netflix is a blip compare to youtube/google/Akamai/facebook (I recently learned facebook also has a similar program, but I think they also use Akamai), on a global scale. I didn't like that glossed on over this as if Netflix is doing something exceptional or even unique. Obviously, it takes some serious engineering chops scale this amount of traffic globally and routes the traffic through on-premise nodes. But at this point, it's not exceptional - if anything its commodity.
- Terretta 9y ago
  Netflix isn't a blip. For instance: https://fortunedotcom.files.wordpress.com/2015/10/apps-by-internet-traffic.png https://fortunedotcom.files.wordpress.com/2015/10/apps-by-in... Still true in 2017: https://techcrunch.com/2017/07/05/u-s-cord-cutters-watch-more-netflix-than-amazon-video-hulu-and-youtube-combined/ https://techcrunch.com/2017/07/05/u-s-cord-cutters-watch-mor... Regardless, long form video streamed globally without buffering is incredibly difficult. Short form clips are drastically easier. Do the math on how much content at what quality can fit in a given cache, or how long it takes to swap a piece of content in a cache, you'll start to see why. It's surprising to me that streaming well is still not a commodity. Even less so if you consider transactional (paid) video, which is what Netflix does. For example, just recently, tech press talked about cable companies losing $10B a year from account sharing. This was a problem VDNs solved in late 90s / early 00s for streaming protocols, but the technique for handling this across a global footprint was sidelined when YouTube's "http" progressive downloads / pseudo streaming and 'ad supported' video became vogue. Now that streams need to act more like streams again, and people want to charge for the content itself again, the solution is again needed, and no http-based "commodity" CDN seems to have it sorted out. Netflix does. PS. A couple refs from before Netflix/Google/YouTube: - https://www.google.com/patents/US20010027491 https://www.google.com/patents/US20010027491 - https://www.google.com/patents/US20010047275 https://www.google.com/patents/US20010047275 - https://news.microsoft.com/2000/06/12/microsofts-new-digital-broadcast-manager-enables-next-wave-of-e-commerce/ https://news.microsoft.com/2000/06/12/microsofts-new-digital... Those were first iterations. Later iterations were more sophisticated, and not patented in order to not disclose.