3 ms·

You’re right. Because every company you have seen with a large scale implementation on AWS must mean that all large companies - including Netflix - are dumb.

coldtea 3y ago
Is there an argument in your comment? For my argument not "all large" companies need to be dumb, as in the strawman above. It's enough to make more sense for the average company to do so. And there are tons of companies, large, SME and others, with big data to store, that have no ops or IT department as such, or just a couple of people. They're not going to build datacenters of their own, distribute them globally, research into reliability and failover solutions, add some custom or adapt some third party stack, and so on. The salary for just an extra person could tower their AWS costs.
- scarface_74 3y ago
  I can’t quite grok from your original comment are you saying that every company who tried to duplicate what AWS does on prem themselves end up spending too much money on consultants etc or are you saying that companies that use AWS still spend too much money on consultants and they don’t have any cost savings?
  - coldtea 3y ago
    Am saying that: - most companies will do better to not try to replicate AWS (or the share of AWS functionality that they need) - most companies aren't tech companies (not talking about the Netflixes and co of the world here) and would botch it, and at best get a costly + high maintenance bad copy
- vidarh 3y ago
  When I did contracting in this space, my AWS based customers invariably needed far more of my time for similar sized setups than the non-cloud customers. The assumption that the non-cloud users will need to pay for any extra people is the opposite of my experience actually running both types of setup. I used to offer to cut their costs by moving them off AWS, but didn't push it very hard because I made far more money from the ones that insisted on using AWS. Some of them had good reasons. Many were just cargo-culting. You don't need to build datacenters, or rack servers yourself, or even own servers, or even ever having seen your servers in order to have racks of dedicated bare metal colo racks of servers. Plenty of service-providers will rent you those services by the hour, and plenty of server providers will rent or lease-to-own hardware for you, so you get monthly bills. I started working in this space in 1995, and while we've often had servers actually on prem, I've also managed racks worth of servers I've never seen in person, in data centres I've never seen, with the help of "remote hands" provided by colo provider employees I've never met.
  - scarface_74 3y ago
    So did they have multiple redundant data centers worldwide?
    - vidarh 3y ago
      They who? I've had dozens of customers, and multiple employers, and co-founded several companies. Most of them had servers in multiple redundant data centres worldwide. None owned any data centres because it rarely pays - owning data centres is a real estate play, not a tech play, which is why you also increasingly see real-estate management companies pick up data centre operators. Unless you're spending in the billions, it's rarely worth it when the markup is a tiny fraction of what you're paying for space and power. Some w/rented servers. Some leased. Some owned. Some with staff on retainer just for us. Some just w/support contracts and remote hands. In some cases we rented office space for ops people in the colo facility, because again: They're real estate plays and the big ones have office suites available.
pclmulqdq 3y ago
I always find it funny that Netflix is considered a poster child of "all cloud all the time." If you look at Netflix's architecture, it actually uses the cloud in a very cost-effective way. They mostly deploy containers rather than using "cloudy" things like serverless, and Netflix's data plane is actually mostly their own CDN (Open Connect CDN), deployed in colo facilities around the world. By data volume, I would guess that only 1% of Netflix's traffic (measured in gigabits per second) actually hits their cloud, if even that.
- scarface_74 3y ago
  Just for reference: you are replying to someone who spent 3 years at AWS (Professional Services)…. I can tell you just from digging into the publicly available videos let alone some of the not publicly available information that you are greatly underestimating the complexity and costs of Netflix’s architecture. You can even spend some time watching some of the interviews about Netflix’s transition from their own data centers to AWS and their reasons. And before you accuse me of being a fanboy or shill. I got Amazoned last year. I have no love loss for Amazon as a former employee
  - coldtea 3y ago
    >I can tell you just from digging into the publicly available videos let alone some of the not publicly available information that you are greatly underestimating the complexity and costs of Netflix’s architecture. Why do they need all that complexity to host a bunch of (not even that many) videos and user profiling?
    - scarface_74 3y ago
      Search for Netflix’s reinvent talks on YouTube
  - pclmulqdq 3y ago
    I agree with you that Netflix's architecture in the cloud is incredibly complex. I know they do a lot of cool things with cloud storage, for example, but also scale breeds complexity. However, complexity is not scale. Netflix users reportedly stream about 200 million hours per day, and at 1 GB/hour, that's an egress of 200 Petabytes/hour, or about 5 exabytes per day. There is no reason for Netflix to pay AWS to handle that, and they don't. That would be $100 million per day from CloudFront. You can read about their data plane online: https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/ If you count how much data goes to their cloud vs their CDN appliances, I would be surprised if the cloud gets very much of their actual scale at all.
vidarh 3y ago
I've seen the size of discounts companies with magnitudes smaller discounts than Netflix has managed to get. I'm sure it's cost-effective for Netflix. That means nothing with respect to whether it will be cost-effective for someone else.

coldtea 3y ago

Is there an argument in your comment? For my argument not "all large" companies need to be dumb, as in the strawman above. It's enough to make more sense for the average company to do so. And there are tons of companies, large, SME and others, with big data to store, that have no ops or IT department as such, or just a couple of people. They're not going to build datacenters of their own, distribute them globally, research into reliability and failover solutions, add some custom or adapt some third party stack, and so on. The salary for just an extra person could tower their AWS costs.

scarface_74 3y ago
I can’t quite grok from your original comment are you saying that every company who tried to duplicate what AWS does on prem themselves end up spending too much money on consultants etc or are you saying that companies that use AWS still spend too much money on consultants and they don’t have any cost savings?
- coldtea 3y ago
  Am saying that: - most companies will do better to not try to replicate AWS (or the share of AWS functionality that they need) - most companies aren't tech companies (not talking about the Netflixes and co of the world here) and would botch it, and at best get a costly + high maintenance bad copy
vidarh 3y ago
When I did contracting in this space, my AWS based customers invariably needed far more of my time for similar sized setups than the non-cloud customers. The assumption that the non-cloud users will need to pay for any extra people is the opposite of my experience actually running both types of setup. I used to offer to cut their costs by moving them off AWS, but didn't push it very hard because I made far more money from the ones that insisted on using AWS. Some of them had good reasons. Many were just cargo-culting. You don't need to build datacenters, or rack servers yourself, or even own servers, or even ever having seen your servers in order to have racks of dedicated bare metal colo racks of servers. Plenty of service-providers will rent you those services by the hour, and plenty of server providers will rent or lease-to-own hardware for you, so you get monthly bills. I started working in this space in 1995, and while we've often had servers actually on prem, I've also managed racks worth of servers I've never seen in person, in data centres I've never seen, with the help of "remote hands" provided by colo provider employees I've never met.
- scarface_74 3y ago
  So did they have multiple redundant data centers worldwide?
  - vidarh 3y ago
    They who? I've had dozens of customers, and multiple employers, and co-founded several companies. Most of them had servers in multiple redundant data centres worldwide. None owned any data centres because it rarely pays - owning data centres is a real estate play, not a tech play, which is why you also increasingly see real-estate management companies pick up data centre operators. Unless you're spending in the billions, it's rarely worth it when the markup is a tiny fraction of what you're paying for space and power. Some w/rented servers. Some leased. Some owned. Some with staff on retainer just for us. Some just w/support contracts and remote hands. In some cases we rented office space for ops people in the colo facility, because again: They're real estate plays and the big ones have office suites available.

pclmulqdq 3y ago

I always find it funny that Netflix is considered a poster child of "all cloud all the time." If you look at Netflix's architecture, it actually uses the cloud in a very cost-effective way. They mostly deploy containers rather than using "cloudy" things like serverless, and Netflix's data plane is actually mostly their own CDN (Open Connect CDN), deployed in colo facilities around the world. By data volume, I would guess that only 1% of Netflix's traffic (measured in gigabits per second) actually hits their cloud, if even that.

scarface_74 3y ago
Just for reference: you are replying to someone who spent 3 years at AWS (Professional Services)…. I can tell you just from digging into the publicly available videos let alone some of the not publicly available information that you are greatly underestimating the complexity and costs of Netflix’s architecture. You can even spend some time watching some of the interviews about Netflix’s transition from their own data centers to AWS and their reasons. And before you accuse me of being a fanboy or shill. I got Amazoned last year. I have no love loss for Amazon as a former employee
- coldtea 3y ago
  >I can tell you just from digging into the publicly available videos let alone some of the not publicly available information that you are greatly underestimating the complexity and costs of Netflix’s architecture. Why do they need all that complexity to host a bunch of (not even that many) videos and user profiling?
  - scarface_74 3y ago
    Search for Netflix’s reinvent talks on YouTube
- pclmulqdq 3y ago
  I agree with you that Netflix's architecture in the cloud is incredibly complex. I know they do a lot of cool things with cloud storage, for example, but also scale breeds complexity. However, complexity is not scale. Netflix users reportedly stream about 200 million hours per day, and at 1 GB/hour, that's an egress of 200 Petabytes/hour, or about 5 exabytes per day. There is no reason for Netflix to pay AWS to handle that, and they don't. That would be $100 million per day from CloudFront. You can read about their data plane online: https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/ If you count how much data goes to their cloud vs their CDN appliances, I would be surprised if the cloud gets very much of their actual scale at all.

vidarh 3y ago

I've seen the size of discounts companies with magnitudes smaller discounts than Netflix has managed to get. I'm sure it's cost-effective for Netflix. That means nothing with respect to whether it will be cost-effective for someone else.