6 ms·

How much does Netflix spend on Amazon AWS?

77 points by gibbiv 8y ago

Waterluvian 8y ago
Since I'm a team of one working on some Amazon services for a far smaller customer base, I get a broad slice of different technologies and problems to solve. I can't imagine being the person whose job is to manage some slice of the part of the stack that monitors just the billing portion of their aws usage. But contemplating that scale also makes me better understand how these companies end up with hundreds of engineers to manage a product that on the surface really doesn't do a whole lot.
- chrisco255 8y ago
  Are you trying to say that Netflix as a product doesn't do a whole lot?
  - falcon620 8y ago
    https://en.wikipedia.org/wiki/Embarrassingly_parallel https://en.wikipedia.org/wiki/Embarrassingly_parallel
  - yeahsure 8y ago
    Well, there are plenty of sites hosting pirated movies that are run by a couple of people. At the end of the day -unless you really know how many tech/human resources something like netflix needs- you could wrongly asume a few guys could host the content on AWS, use a third party for billing and be done with it.
    - StudentStuff 8y ago
      AWS is totally uncompetitive if your looking to serve video though, Amazon's bandwidth pricing is something straight out of 1999, it is one of the largest pieces of their moat to lock people into AWS.
      - deleted 8y ago
        [deleted]
      - mayniac 8y ago
        I'm one of those people the comment above you mentioned who runs a site hosting pirate media (it's all private before anyone asks) Looked into AWS a while ago so I could throw some encrypted containers on it thinking it would be cheaper. Storage on S3 would have cost about four times as much as I'm paying now in VPS costs, and EFS (the service I actualy wanted) about ten times as much, and that's before looking at bandwidth. Oh and I'd still need at least one VPS to handle transcoding, content delivery etc. I think Netflix save money by using S3 for the majority of their storage but having their own CDN so they use less bandwidth from AWS.
    - zaphirplane 8y ago
      Their technology isn’t about here is a list of movies enjoy, their tech is about enticing people to watch thru List, recommendations, trends ... etc just like Uber isn’t a taxi company most of their tech is about optimising the pricing
  - ghostbrainalpha 8y ago
    He's saying that on the surface it doesn't do a lot. But that he understands why it would take 100's of engineers. You seem to be preparing to argue with someone, that definitely agrees with you. BUT.. for if you just need someone to argue with..... Netflix is a video library organization system, and a video player. It really doesn't do much. They should consider building the whole thing on WordPress and managing it with a single person.
    - chrisco255 8y ago
      He seems to be inferring that's mostly due to AWS management. Not going to take the bait, but it is an interesting phenomenon that a lot of people will look at an app that has a simple surface layer with an intuitive UX and not appreciate the difficulty of engineering that at scale.
      - aidos 8y ago
        I think you might need to read it again more charitably. To me it reads that it’s just the scale that requires a big team, nothing to do with AWS
- hibikir 8y ago
  AWS is expensive enough, and the pricing is hard enough, that you don't need a crazy scale before you have a team monitoring AWS expenses. It all depends on how much more revenue you'll get out of each engineer, but ignoring the most magical of companies, you probably want at least one person dedicated to this before you hit 200 engineers deploying to AWS. And yes, a lot of the expense is invisible to the end user. Some well known companies end up spending significant percentages of their AWS budgets on building and running integration tests. Suddenly you also end up getting teams dedicated to improving just that.
paxy 8y ago
This is impossible to even ballpark without some insider info. My company for example runs a ton of user facing services on AWS, but the vast majority of costs come from internal data storage and processing pipelines (server logs, analytics, BI/SI dashboards).
Alex3917 8y ago
I just hope someone there is getting weekly emails like: "Dear AWS customer: AWS Trusted Advisor currently shows alerts for 12 checks (1 red and 11 yellow) and $18,922,425.23 of potential monthly savings based on your usage."
- Artemis2 8y ago
  “AWS Free Tier limit alert”
alttab 8y ago
$665M. Just guessing.
- stevewilhelm 8y ago
  "Amazon Web Services reported sales on Thursday of $5.44 billion (for Q1 2018)" [1] So there is your upper bound. [1] https://www.cnbc.com/2018/04/26/aws-earnings-q1-2018.html https://www.cnbc.com/2018/04/26/aws-earnings-q1-2018.html
blantonl 8y ago
I've got to think bandwidth is their largest expenditure... even with all the CDN infrastructure they have in place. AWS bandwidth is extremely expensive - regardless of the negotiated discounts they probably get.
- jonny_eh 8y ago
  Netflix doesn't host any video content on AWS, none. It only serves up their API + site.
  - whoisjuan 8y ago
    Saying that they host none of their video content in AWS it's kind of a stretch. The original video and encoded versions of those videos are probably all hosted in S3. Every Netflix Open Connect Appliance still needs to copy and update its content from somewhere and as far as I know, they do it from S3 which it's ultimately their core data lake. How often and how efficiently they do it is probably unknown, but I can't imagine any other strategy to distribute their video across their appliances network that doesn't involve S3. It's also very unlikely that they are using 'warm' storage to replicate their content across their appliances network since the objects would be retrieved and replicated behind the scenes (no user-facing speed required), so they could be as well using Amazon Glacier to host their content video and just retrieve it slowly at a fraction of S3's cost. You can read more about Netflix Open Connect network and their storage appliances here: https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/
CyanLite2 8y ago
TLDR: Netflix gives over $250mil to their competition every year.
- sn41 8y ago
  Is it that simple? To replace AWS with in-house expertise producing acceptable quality cloud services for Netflix, how much will Netflix have to spend? My guess is that it will be significantly costlier to go self-sufficient (during something like the coming decade) considering AWS' almost 12+ year lead on these technologies.
  - Terretta 8y ago
    It’s not the tech. It’s the people. If you can only get so many people, what do you want them working on? Undifferentiated heavy lifting, or something that directly makes you better than your competitors in your customers’ minds?
- mr_toad 8y ago
  It’s not like that’s pure profit for Amazon.

77 points by gibbiv 8y ago

Since I'm a team of one working on some Amazon services for a far smaller customer base, I get a broad slice of different technologies and problems to solve. I can't imagine being the person whose job is to manage some slice of the part of the stack that monitors just the billing portion of their aws usage. But contemplating that scale also makes me better understand how these companies end up with hundreds of engineers to manage a product that on the surface really doesn't do a whole lot.

chrisco255 8y ago
Are you trying to say that Netflix as a product doesn't do a whole lot?
- falcon620 8y ago
  https://en.wikipedia.org/wiki/Embarrassingly_parallel https://en.wikipedia.org/wiki/Embarrassingly_parallel
- yeahsure 8y ago
  Well, there are plenty of sites hosting pirated movies that are run by a couple of people. At the end of the day -unless you really know how many tech/human resources something like netflix needs- you could wrongly asume a few guys could host the content on AWS, use a third party for billing and be done with it.
  - StudentStuff 8y ago
    AWS is totally uncompetitive if your looking to serve video though, Amazon's bandwidth pricing is something straight out of 1999, it is one of the largest pieces of their moat to lock people into AWS.
    - deleted 8y ago
      [deleted]
    - mayniac 8y ago
      I'm one of those people the comment above you mentioned who runs a site hosting pirate media (it's all private before anyone asks) Looked into AWS a while ago so I could throw some encrypted containers on it thinking it would be cheaper. Storage on S3 would have cost about four times as much as I'm paying now in VPS costs, and EFS (the service I actualy wanted) about ten times as much, and that's before looking at bandwidth. Oh and I'd still need at least one VPS to handle transcoding, content delivery etc. I think Netflix save money by using S3 for the majority of their storage but having their own CDN so they use less bandwidth from AWS.
  - zaphirplane 8y ago
    Their technology isn’t about here is a list of movies enjoy, their tech is about enticing people to watch thru List, recommendations, trends ... etc just like Uber isn’t a taxi company most of their tech is about optimising the pricing
- ghostbrainalpha 8y ago
  He's saying that on the surface it doesn't do a lot. But that he understands why it would take 100's of engineers. You seem to be preparing to argue with someone, that definitely agrees with you. BUT.. for if you just need someone to argue with..... Netflix is a video library organization system, and a video player. It really doesn't do much. They should consider building the whole thing on WordPress and managing it with a single person.
  - chrisco255 8y ago
    He seems to be inferring that's mostly due to AWS management. Not going to take the bait, but it is an interesting phenomenon that a lot of people will look at an app that has a simple surface layer with an intuitive UX and not appreciate the difficulty of engineering that at scale.
    - aidos 8y ago
      I think you might need to read it again more charitably. To me it reads that it’s just the scale that requires a big team, nothing to do with AWS
hibikir 8y ago
AWS is expensive enough, and the pricing is hard enough, that you don't need a crazy scale before you have a team monitoring AWS expenses. It all depends on how much more revenue you'll get out of each engineer, but ignoring the most magical of companies, you probably want at least one person dedicated to this before you hit 200 engineers deploying to AWS. And yes, a lot of the expense is invisible to the end user. Some well known companies end up spending significant percentages of their AWS budgets on building and running integration tests. Suddenly you also end up getting teams dedicated to improving just that.

paxy 8y ago

This is impossible to even ballpark without some insider info. My company for example runs a ton of user facing services on AWS, but the vast majority of costs come from internal data storage and processing pipelines (server logs, analytics, BI/SI dashboards).

Alex3917 8y ago

I just hope someone there is getting weekly emails like: "Dear AWS customer: AWS Trusted Advisor currently shows alerts for 12 checks (1 red and 11 yellow) and $18,922,425.23 of potential monthly savings based on your usage."

Artemis2 8y ago
“AWS Free Tier limit alert”

alttab 8y ago

$665M. Just guessing.

stevewilhelm 8y ago
"Amazon Web Services reported sales on Thursday of $5.44 billion (for Q1 2018)" [1] So there is your upper bound. [1] https://www.cnbc.com/2018/04/26/aws-earnings-q1-2018.html https://www.cnbc.com/2018/04/26/aws-earnings-q1-2018.html

blantonl 8y ago

I've got to think bandwidth is their largest expenditure... even with all the CDN infrastructure they have in place. AWS bandwidth is extremely expensive - regardless of the negotiated discounts they probably get.

jonny_eh 8y ago
Netflix doesn't host any video content on AWS, none. It only serves up their API + site.
- whoisjuan 8y ago
  Saying that they host none of their video content in AWS it's kind of a stretch. The original video and encoded versions of those videos are probably all hosted in S3. Every Netflix Open Connect Appliance still needs to copy and update its content from somewhere and as far as I know, they do it from S3 which it's ultimately their core data lake. How often and how efficiently they do it is probably unknown, but I can't imagine any other strategy to distribute their video across their appliances network that doesn't involve S3. It's also very unlikely that they are using 'warm' storage to replicate their content across their appliances network since the objects would be retrieved and replicated behind the scenes (no user-facing speed required), so they could be as well using Amazon Glacier to host their content video and just retrieve it slowly at a fraction of S3's cost. You can read more about Netflix Open Connect network and their storage appliances here: https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/

CyanLite2 8y ago

TLDR: Netflix gives over $250mil to their competition every year.

sn41 8y ago
Is it that simple? To replace AWS with in-house expertise producing acceptable quality cloud services for Netflix, how much will Netflix have to spend? My guess is that it will be significantly costlier to go self-sufficient (during something like the coming decade) considering AWS' almost 12+ year lead on these technologies.
- Terretta 8y ago
  It’s not the tech. It’s the people. If you can only get so many people, what do you want them working on? Undifferentiated heavy lifting, or something that directly makes you better than your competitors in your customers’ minds?
mr_toad 8y ago
It’s not like that’s pure profit for Amazon.