6 ms·
Researchers Map Locations of Servers in Netflix’s Content Delivery Network
- socmag 10y agoCan anyone provide more information on the server infrastructure than this article: http://www.techhive.com/article/2158040/how-netflix-streams-movies-to-your-tv.html http://www.techhive.com/article/2158040/how-netflix-streams-... It claims each server serves up to 10,000 streams simultaneously. So operational peak capacity is at ~50M simultaneous streams. Also, 10k streams at 1080p would be ~70Gbps per machine at full HD, or ~20Gbps per machine at SD. With some blend of course being the norm. There is of course a big difference between being capable of and actual and FUD. I'd really love to know what the internals of one of their streaming servers might look like. Anyone here who has architected a box with that capacity?
- socmag 10y agoSomeone pointed out the original paper, which has better estimates than my back of the napkin calculation for traffic throughput. Good stuff. Still would love to know more about the internals of their servers and support infrastructure.
- jlgaddis 10y agoI recall seeing presentations at various FreeBSD events from Netflix folks talking about them, so a search for something like that might find what you're looking for. If memory serves... SuperMicro motherboards, multiple Mellanox/Chelsio 10 Gbps NICs (probably upgraded to 40 Gbps by now, or getting close to that point), a bunch (up to 28, perhaps?) of hard drives (with no/little redundancy for thecontent), and a ton of RAM (for ZFS ARC). They run on (heavily tuned) FreeBSD operating system.
- jakozaur 10y agoDirect link: http://arxiv.org/pdf/1606.05519v1.pdf http://arxiv.org/pdf/1606.05519v1.pdf
- socmag 10y agoThanks, much better!
- anc84 10y agoHow many did they find in Kansas?
- boulos 10y agoIt's interesting to contrast this with the Google Global Cache nodes at the edge (https://peering.google.com/#/infrastructure https://peering.google.com/#/infrastructure). As Netflix expands globally, will they be in just as many locations? (Perhaps not because streaming video is less latency sensitive than API responses and search results). I'm also curious if the public disclosure of this info means the researchers didn't bother to do the same reverse engineering for say YouTube? Disclosure: I work at Google on Cloud, but don't work on edge caching.
- eatbitseveryday 10y agoZero severs in Russia, China, Turkey, India..? Over 2 bn. people left out of the market? I know not all have network access, but still. Not Russia or India at least?
- throwaway1974 10y agohttps://torrentfreak.com/vpn-provider-pia-exits-russia-server-seizures-160712/ https://torrentfreak.com/vpn-provider-pia-exits-russia-serve... Russia is not a friendly country to do business in China is closed off behind the Great Firewall and probably already has a local netflix clone India, I do not know, could be cost issues?
- inopinatus 10y agoIndia could be a tax issue. As a tech sector APAC manager I have been instructed more than once not to visit. Not for any internal conference, and definitely not for any customer contact, and to take verifiable steps to avoid making "management decisions" if visiting for a holiday. Typical quote, "You do not want to be the guy that triggered tax obligations for us in India". IANAL but I believe it's due to the broad framing of India's "place of effective management" rules. This sort of thing ends up being mostly to Singapore's benefit, transnationals won't place their APAC region office in India as a result.
- zhte415 10y agoAs long as your main revenue base is not India, and you're not visiting to make board decisions visiting India, on the business of a subsidiary, should not be an issue. You're looking at taxation for resident income, and source income. Sure, a local entity should pay tax at source, in your case India, but unless you're sitting in India and taking global decisions or decisions creating significant global impact, no need to pay on resident income. IANAL but this is my experience interacting mainly with large corporate. You might want to ask legal in your company or have an international independent auditor advise; don't want the wool may well being pulled over your eyes. And that would be bad, as it would be a dereliction of trust. And I've seen that a lot when, non India specific, but offshored locations try to build ill-founded silos.
- milliams 10y agoIn a similar vein there's a project at the moment to map the world's internet infrastructure called The New Cloud Atlas (http://newcloudatlas.org http://newcloudatlas.org) which describes itself thus: > The New Cloud Atlas is a global effort to map each data place that makes up the cloud in an open and accountable way. We have set out to find and map each warehouse data centre, each internet exchange, each connecting cable and switch. Anything of any physical significance in the operation of the cloud should be observed is some way, and recorded for everyone to see and use.
- juandazapata 10y agoI thought that the "Netflix CDN" was AWS.
- SteveNuts 10y agoHasn't been for a while, the egress would be insanely expensive. They use devices they call "Openconnect" for peering in ISPs https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/
- juandazapata 10y agoThanks for clarifying. Very informative.
- joezydeco 10y agoI have a friend that runs IT for a large college campus in the Midwest. They not only have a Netflix cache, but an Apple one as well. The Netflix one runs at a few gigabit/sec at all times. The Apple one keeps the WAN from going down on update day.
- Something1234 10y agoWhat cache server do they use?
- philjohn 10y agoNetflix provides their own - https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/ I would imagine Apple provides something similar.
- joezydeco 10y agoFrom what I've been told, it's a rack-mounted unit that's sealed. That's about it. The Apple unit contains iTunes content as well as iOS/OSX system updates.
- betaby 10y agoHonestly, why that's research worthy? That's not a new knowledge, that's information is somewhat known to all employees, partners, various NMS vendors, ISPs. 'Nice' graph and latex formatted article doesn't make that scientific research. I guess there is to much useless "since" in style of 'I run ping/traceroute from the couple of locations and here is my scientific research'.
- kafkaesq 10y agoIt's not about "ping and traceroute"; it's about organizing and mapping the data (both logically and geographically) that come out tools like ping and traceroute. They aren't saying they've demonstrated quantum teleportation, or anything. But for the subject matter at hand (and given that Netflix runs one of the largest CDNs in the world), it's quite impressive.
- bjt 10y agoIt sounds like Netflix's host naming scheme is what made it possible for the researchers to do this. I assume that means using regions and incrementing counters in the hostnames. Who wants to bet that in the next year we'll see their hostnames change to be random-looking instead?
- toomuchtodo 10y agoThis won't happen. It would cause havoc when attempting to work with peering partners and transit providers when troubleshooting.