5 ms·

I'm trying to understand why large companies use AWS for their primary infrastructure. Both AWS and Azure are obscenely expensive when compared to buying serve

I'm trying to understand why large companies use AWS for their primary infrastructure. Both AWS and Azure are obscenely expensive when compared to buying servers and colocating them somewhere. The cloud model works well for bootstrapped startups, but anyone with the resources to buy and manage their own servers is crazy if they use AWS or Azure as their primary server platform.

jon-wood 11y ago
I feel like this is a myth that just won't go away. If you run your 24/7 steady state load using on-demand hourly charged instances, and don't factor in the cost of managing your own hardware and data centres then yes, EC2 is hideously expensive. If you plan your spend (much like you would in buying your own hardware and DC space) it really isn't - reserved instances considerably reduce prices, and come with bulk discounts when you buy enough of them. It's been a while, but when costing out our migration to EC2 it came out cheaper than any other VM provider, and was in the same ballpark as leasing our own hardware.
- nulltype 11y ago
  What about bandwidth costs?
  - jon-wood 11y ago
    Frankly, they're a rounding error with our current workload. If you're in the business of running a CDN or streaming video you may want to look at other options - Netflix with their caching servers in ISP's data centres are a pretty good example of this.
lostcolony 11y ago
Is it? Cloud providers bring you not just localized infrastructure, but distributed infrastructure, with clear mechanisms for scaling as needed, plus support, etc. With AWS, you have a predictable cost, predictable up-time, and ability to scale at peak, and your costs are spread over time, rather than all up front. With your own servers you have to build out to handle peak load (and run mostly dormant during non-peak hours), across multiple locations, to achieve the same availability. I'm not sure the costs for the convenience of AWS are so obscene in such a case. They may still be higher, I honestly don't know, but the convenience benefit is huge. They key bit is that AWS' costs are largely -known-, whereas the costs of DIYing aren't. Per Netflix's blog post about why they use AWS for serving everything except video content - "We could have chosen to build out new data centers, build our own redundancy and failover, data synchronization systems, etc. Or, we could opt to write a check to someone else to do that instead." Oftentimes for enterprise, having all your infrastructure in place, done right the first time, with someone else eating any costs associated with the unexpected, and ready to start building your application on, is worth the cost. For enterprise, paying a known amount to avoid risk is oftentimes worth it.
- msravi 11y ago
  So why would you not have localized infrastructure to handle average loads and use AWS to scale to peak only when necessary? Why use AWS even for your average loads?
  - lostcolony 11y ago
    Because then you have to target two different sets of infrastructure. The extra work and complexity carries with it a rather large cost too, -and- greater risk. It's almost the worst of both worlds, rather than the best, as you have to solve all the problems of rolling your own, -and- pay Amazon for the extra utilization.
- downandout 11y ago
  I'm guessing that Netflix is getting truly massive discounts from their published rates. For most everyone else, it won't make sense.
- latch 11y ago
  Yes, it is. "If you’re used to designing and deploying applications in your own data centers, you need to be prepared to unlearn a lot of what you know." Which is it? Did it save them money because they didn't have to "build their own redundancy and failover" or did they have to build a bunch of custom tools like Chaos Monkey because "I knew to expect higher rates of individual instance failure in AWS, but I hadn’t thought through some of these sorts of implications." ??? From reading Netflix' blog, it's clear that it was a huge learning and engineering effort. That cost has to be taken into account and added to the fact that across many AWS offerings, you're looking at as much as a 10x price / performance penalty versus dedicated or collocation. I'm unconvinced that they couldn't have done it cheaper and better through more transitional approaches, and moreso that it's a meaningful indicator for anyone else. AWS has been massively innovative. But it's much more expensive, and a huge part of their business is sales and getting CXOs onboard, not necessarily providing good value.
  - amazon_not 11y ago
    "That cost has to be taken into account and added to the fact that across many AWS offerings, you're looking at as much as a 10x price / performance penalty versus dedicated or collocation." This. The pricing is especially egrerious in bandwidth charges which for AWS are almost pure profit. Amazon is only cheap if you don't use it.
asgard1024 11y ago
> I'm trying to understand why large companies use AWS for their primary infrastructure. Because they are corrupt? Could be? Large companies are basically centrally controlled economies, and we know how that turns out.. It seems to me that large companies live in a special bubble where they are willing to pay extra for everything. On the other hand, they also charge extra, so externally, it will mostly cancel out. The winners of this scheme are probably the people involved in running (either directly or as shareholders) these large companies, because their salaries (and other bonuses) are artificially inflated compared to the rest of the society. The losers are probably the government and the taxpayer (i.e. the public), these can be easily made to pay extra but it's difficult for them to charge extra.
chx 11y ago
Why? Why colo and not rent dedicated servers...? I tried to understand the logic behind colo but I can't.
- BillinghamJ 11y ago
  It's cheaper, drastically so at scale. You just need to be running enough for it to be cost efficient to have an ops person.
  - chx 11y ago
    Care to point to an analysis supporting this? There's surely a point where the bother with replacing spare parts is cheaper than letting someone else deal with it but it must be some scale.
smackfu 11y ago
>anyone with the resources to buy and manage their own servers Eh, I think the bar is a lot higher than that, especially if you have to meet external compliance standards with yearly audits, or have to have good disaster recovery with multiple datacenters. Those kind of things get very expensive if you are under a certain scale.

jon-wood 11y ago

I feel like this is a myth that just won't go away. If you run your 24/7 steady state load using on-demand hourly charged instances, and don't factor in the cost of managing your own hardware and data centres then yes, EC2 is hideously expensive. If you plan your spend (much like you would in buying your own hardware and DC space) it really isn't - reserved instances considerably reduce prices, and come with bulk discounts when you buy enough of them. It's been a while, but when costing out our migration to EC2 it came out cheaper than any other VM provider, and was in the same ballpark as leasing our own hardware.

nulltype 11y ago
What about bandwidth costs?
- jon-wood 11y ago
  Frankly, they're a rounding error with our current workload. If you're in the business of running a CDN or streaming video you may want to look at other options - Netflix with their caching servers in ISP's data centres are a pretty good example of this.

lostcolony 11y ago

Is it? Cloud providers bring you not just localized infrastructure, but distributed infrastructure, with clear mechanisms for scaling as needed, plus support, etc. With AWS, you have a predictable cost, predictable up-time, and ability to scale at peak, and your costs are spread over time, rather than all up front. With your own servers you have to build out to handle peak load (and run mostly dormant during non-peak hours), across multiple locations, to achieve the same availability. I'm not sure the costs for the convenience of AWS are so obscene in such a case. They may still be higher, I honestly don't know, but the convenience benefit is huge. They key bit is that AWS' costs are largely -known-, whereas the costs of DIYing aren't. Per Netflix's blog post about why they use AWS for serving everything except video content - "We could have chosen to build out new data centers, build our own redundancy and failover, data synchronization systems, etc. Or, we could opt to write a check to someone else to do that instead." Oftentimes for enterprise, having all your infrastructure in place, done right the first time, with someone else eating any costs associated with the unexpected, and ready to start building your application on, is worth the cost. For enterprise, paying a known amount to avoid risk is oftentimes worth it.

msravi 11y ago
So why would you not have localized infrastructure to handle average loads and use AWS to scale to peak only when necessary? Why use AWS even for your average loads?
- lostcolony 11y ago
  Because then you have to target two different sets of infrastructure. The extra work and complexity carries with it a rather large cost too, -and- greater risk. It's almost the worst of both worlds, rather than the best, as you have to solve all the problems of rolling your own, -and- pay Amazon for the extra utilization.
downandout 11y ago
I'm guessing that Netflix is getting truly massive discounts from their published rates. For most everyone else, it won't make sense.
latch 11y ago
Yes, it is. "If you’re used to designing and deploying applications in your own data centers, you need to be prepared to unlearn a lot of what you know." Which is it? Did it save them money because they didn't have to "build their own redundancy and failover" or did they have to build a bunch of custom tools like Chaos Monkey because "I knew to expect higher rates of individual instance failure in AWS, but I hadn’t thought through some of these sorts of implications." ??? From reading Netflix' blog, it's clear that it was a huge learning and engineering effort. That cost has to be taken into account and added to the fact that across many AWS offerings, you're looking at as much as a 10x price / performance penalty versus dedicated or collocation. I'm unconvinced that they couldn't have done it cheaper and better through more transitional approaches, and moreso that it's a meaningful indicator for anyone else. AWS has been massively innovative. But it's much more expensive, and a huge part of their business is sales and getting CXOs onboard, not necessarily providing good value.
- amazon_not 11y ago
  "That cost has to be taken into account and added to the fact that across many AWS offerings, you're looking at as much as a 10x price / performance penalty versus dedicated or collocation." This. The pricing is especially egrerious in bandwidth charges which for AWS are almost pure profit. Amazon is only cheap if you don't use it.

asgard1024 11y ago

> I'm trying to understand why large companies use AWS for their primary infrastructure. Because they are corrupt? Could be? Large companies are basically centrally controlled economies, and we know how that turns out.. It seems to me that large companies live in a special bubble where they are willing to pay extra for everything. On the other hand, they also charge extra, so externally, it will mostly cancel out. The winners of this scheme are probably the people involved in running (either directly or as shareholders) these large companies, because their salaries (and other bonuses) are artificially inflated compared to the rest of the society. The losers are probably the government and the taxpayer (i.e. the public), these can be easily made to pay extra but it's difficult for them to charge extra.

chx 11y ago

Why? Why colo and not rent dedicated servers...? I tried to understand the logic behind colo but I can't.

BillinghamJ 11y ago
It's cheaper, drastically so at scale. You just need to be running enough for it to be cost efficient to have an ops person.
- chx 11y ago
  Care to point to an analysis supporting this? There's surely a point where the bother with replacing spare parts is cheaper than letting someone else deal with it but it must be some scale.

smackfu 11y ago

>anyone with the resources to buy and manage their own servers Eh, I think the bar is a lot higher than that, especially if you have to meet external compliance standards with yearly audits, or have to have good disaster recovery with multiple datacenters. Those kind of things get very expensive if you are under a certain scale.