7 ms·
Netflix Shuts Down Final Bits of Own Data Center Infrastructure
- iofj 11y agoKeep in mind that all but the business rules actually run in their own custom CDN nodes around the world, not in these cloud setups.
- 2trill2spill 11y agoI thought everything but the video streaming was done on AWS. Like the Website, the recommendation engine, video encoding, etc, So I think thats a lot more then business rules. The FreeBSD CDN just serves the video.[1] The video below is from AsiaBSDCon 2014. [1]: https://www.youtube.com/watch?v=4sZZN8Szh14 https://www.youtube.com/watch?v=4sZZN8Szh14
- jedberg 11y agoYou have that backwards. Everything runs in AWS except serving raw video bits from very stripped down boxes in data centers around the world. Those boxes in the CDN do literally nothing but serve video files via https very quickly.
- 2trill2spill 11y agoOpenconnect does not serve https it's http, although they are evaluating using https.
- jedberg 11y agoI'm 100% sure it is https (I worked there). But if you don't believe me you can sniff your network traffic the next time you play a movie on Netflix. :)
- late2part 11y agoGreat story, brah. Except for the CDN. https://openconnect.netflix.com/peeringLocations/ https://openconnect.netflix.com/peeringLocations/
- justincormack 11y agoYes, a data centre in every ISP. If it was all in one place it would be a huge data centre.
- deleted 11y ago[deleted]
- michaelbuckbee 11y agoWasn't familiar with OpenConnect. It is the program where Netflix puts a caching server in an ISP's datacenter so that the traffic is more "local" and cheaper for all involved.
- 2trill2spill 11y agoOpenConnect is the CDN that serves the actual Netflix video, it's Based on FreeBSD 10 and Nginx. Netflix puts them in ISP's and internet peering locations across the internet.
- yardie 11y agoWas going to say exactly this. They've decentralized their datacenter by pushing it to their users ISP.
- late2part 11y agoA lot of folks responded to this by talking about how Netflix asks ISPs to put caches in the ISP locations. That is true but it's not the point. Netflix runs a large CDN on their own infrastructure, and rents space in various datacenters. They "peer" from these locations to distribute their traffic, and they also buy transit to distribute their traffic. They also send traffic to ISP caches from these locations. So they are running their own data centers, in many locations.
- danhou 11y agoIs it just me, or is it a massive risk to have your entire infrastructure hosted by a company who is also one of your chief competitors?
- late2part 11y agocorrect. 'cooperatition.'
- Scaevolus 11y agoAWS might make more money from Netflix than they do from Amazon Prime Video.
- late2part 11y agofalse. http://www.business-management-degree.net/features/amazon-vs-netflix/ http://www.business-management-degree.net/features/amazon-vs...
- SteveNuts 11y agoI think they meant Netflix pays Amazon more per year for infrastructure than Amazon makes on Prime instant. It's hard to say how much Prime streaming makes per year since it's also part of the 2 day shipping.
- Scaevolus 11y agoThat infographic isn't very clear. It's hard to separate the impact of Prime Video from Prime and from normal Amazon operations. How many people buy Amazon Prime for the video streaming? What percentage of the $800M Prime revenue is attributable to the video service? I'd wager less than 10%. Does Netflix spend more than $80M a year on AWS? The numbers needed to answer the question aren't publicly available. It's a bunch of guesswork.
- bsimpson 11y agoConsidering the bills I've seen for AWS at even modestly-sized companies, I would not be surprised if Netflix spends more than $80M a year on AWS. In fact, considering the amount of bandwidth they use, I'd be surprised if they didn't. They spent $1.3B on "General/Administrative Expenses" last year - think more than 6% of that was infrastructure?
- trequartista 11y agoActual blog post on the Netflix site - https://media.netflix.com/en/company-blog/completing-the-netflix-cloud-migration https://media.netflix.com/en/company-blog/completing-the-net...
- mattdeboard 11y agoIs Netflix at the top end of some curve? My understanding of moving from cloud -> datacenter is, you know, there is some line where it's cheaper to just lease your hardware than pay for cloud infrastructure. At that size you tend to have DBAs, dedicated infrastructure people, etc., to make scaling up go. But now has Netflix come through that line where the cost savings is outweighed by the uptime/availability etc.?
- phamilton 11y agoNetflix doesn't pay what you or I would pay. Netflix is at a point in their AWS usage where Amazon calls them to help track down AWS bugs.
- fleitz 11y agoNetflix has hugely 'spikey' load across the globe, AWS is perfect for that use case. They need to spike for prime time and then don't really need much infrastructure.
- throwaway_exer 11y agoNetflix tried operating their own DCs, and weren't good enough at it. Hence the move to the cloud. They do have discounted pricing, and they get the agility to add a new region overnite.
- jedberg 11y agoIt's important to keep this in mind: For most companies, their infrastructure is the second largest expense after their employees. For Netflix, the cost of both pales in comparison to the content licensing (which in 2016 they have said is 6 billion). Cost isn't a huge concern, agility is far more important. That being said, the all in cost for Netflix on AWS is still cheaper than trying to run nine datacenters around the world, which is what they would have to do to replace their usage of AWS.
- easytiger 11y agoIt would be beneficial for amazon to sell their services at 50% off to take netflix out of the game of running the DC themselves for the next 15 years
- ghostek 11y agoInteresting bits about the microsrvices approach. Other companies like Microsoft still need to wake up and understand the value of small teams, microservices deployed in isolation, real devops etc.
- struct 11y agoThey do have Azure Service Fabric however, which does seem incredibly promising.
- panamafrank 11y agocheck out MS Orleans, it's similar to Erlang. https://github.com/dotnet/orleans https://github.com/dotnet/orleans
- ghostek 11y agoI heard about Orleans but how much is it in use internally at MS? Lot of services are not even on Azure, yet? Ever? It's well known that new fancy modern approaches take years to be understood and absorbed int the company strategy, think Git, Agile, elastic services, REST, mobile... Don't get me wrong I'd love to see MS going back to be an innovator, and Orleans might be an oppotunity. How long has it been cooking, 2 3 years? Where's the disruptive plan to make it the new paradigm?
- us0r 11y ago>I heard about Orleans but how much is it in use internally at MS They have a small game called Halo which uses it.
- ksec 11y agoWell they still own ( I think ) their CDN hardware. Which is essentially the Core and most heavy part of Global Video Delivery.
- Animats 11y agoAh. So 99% of the traffic doesn't flow through AWS.
- tetraverse 11y agoWhat did it cost to transfer Netflix to AWS and what are the running costs charged to Netflix. I recall reading somewhere that the first attempt was less than successful, and Amazon had to virtually rebuild the service from scratch.
- tetraverse 11y agoAnswering me own question .. "NETFLIX OpenConnect Appliance deployment guide:" http://oc.nflxvideo.net/docs/OpenConnect-Deployment-Guide.pdf http://oc.nflxvideo.net/docs/OpenConnect-Deployment-Guide.pd...
- dzdt 11y agoNetflix is in the video distribution business. Their usage is hugely variable over the course of a day. AWS is in the cloud computing business. They are good at selling unused server time. If netflix ran their own datacenters, they would have to pay for infrastructure to sit unused for most of the day to meet their peak demand. With AWS someone else is using and paying for that infrastructure in off hours. The cost-savings for netflix and synergy for AWS make the deal make sense.