7 ms·

I've mentioned it before and will again: AWS spent a decade openly and privately advertising how much "cheaper" the cloud was. How economies of sclae meant you

by tw04 5y ago

I've mentioned it before and will again: AWS spent a decade openly and privately advertising how much "cheaper" the cloud was. How economies of sclae meant you'd save boatloads of money. Except, it was never true. You will potentially save money if you're just starting out and have no idea if your idea will be successful. A massive capital outlay is difficult and stupid at that point. You will also potentially save money if the tool you use to make money is something you can easily scale up and down, and actually DOES scale up and down. IE: I host a fantasy football stats app and every Monday night, Thursday night, and all day Sunday I've got millions of users, and thousands the rest of the week. If you don't meet those criteria, unless you are absolutely horrible at negotiating with your vendors, you aren't saving money going into the cloud. I don't care that "datacenters are expensive to build". If you are large enough to need to build your own datacenter instead of renting floorspace for pennies on the dollar from Switch or Equinix (although Equinix is kind of pricey) or Windstream or insert provider - then the datacenter is a rounding error in your 10-year IT budget. The problem is "I moved company XYZ to the cloud" is now a requirement for an IT exec to show up to the country club on Saturday. So whether it makes any fiscal sense at all, the board wants to know "what our cloud plans are" and most execs are more than happy to oblige in order to pad that resume. In my experience, most of them leave long before the bill comes due, and people start asking why the IT budget quadrupled in 3 years. The best part is, most of these companies can't pass along that increased cost to end-customers like a Netflix can because they aren't actually making money on the apps they're hosting in the cloud, those are all internal business facing apps.

prpl 5y ago
A few things are relatively cheap: Compute on preemptable VMs/spot instances is cheaper in cloud. Object storage, if you would have required a vendor support contract on prem, up to a PB or so. managed kubernetes, if you would have required a vendor support contract on prem.
manigandham 5y ago
The money is not the issue. Cost is not the same as value.
- Closi 5y ago
  Cost is a key component of value though.
- tw04 5y ago
  >The money is not the issue. Money is ALWAYS the issue. Whether you can pass along increased cost to your customers or not is entirely dependent on what your business is and the competitive landscape of it. I don't care how "agile" the cloud makes you, if your cost of IT is more expensive than the revenue you bring in, you're bankrupt. It's the running joke around here about people who build autoscaling k8s based infrastructure for a site that could be hosted on a $5/month VPS with even a little bit of tuning and caching. Complexity for the sake of complexity is called stupidity unless you're doing it as part of an educational experience. Going to production with needless complexity is a great way to ruin a good idea. >Cost is not the same as value. I guess it's a good thing I didn't state that cost is the same thing as value?
  - exdsq 5y ago
    There's more to it than just the tech and what is cheapest though. Like risk management. Cost might be higher for a less-risky thing.
  - londons_explore 5y ago
    > build autoscaling k8s based infrastructure for a site that could be hosted on a $5/month VPS I think the goal is that it should be the same amount of engineering time to set up both options, and both should cost $5 per month. Today, that is broadly untrue. Although a few niches like appengine offer this.
  - devoutsalsa 5y ago
    It's all fun & games until you can't make payroll.
  - pm90 5y ago
    If we’re discussing major corporations, they are willing to pay a premium for predictability and reliability of service. Cloud vendors provide just that: if you want more resources, you can scale up until you hit capacity, then you can call the account manager to increase it. Very little downtime (except for the occasional outages). All of these are known processes. With operating your own DC, you have to hire really great people and hope they don’t fuck up. You have to plan much in advance for capacity changes. It’s very much possible that something goes wrong in your compute infrastructure and simply nobody in your team has the expertise to fix it. Put it another way: can you pay/hire the same kinds of engineers that work at AWS/GCP? If not, cloud is the safer route. Cost does matter, and this is why companies sign multi year contracts to get discounts. But business continuity is critical.
mbesto 5y ago
You have some really good points, but your premise is simply not based in reality. You're painting the picture as if AWS is the 2nd version of the IBM saying: "No one ever got fired for using AWS". This is simply not true...at least not the part where "IBM" (i.e. AWS now) is somehow inferior to the competition. > If you are large enough to need to build your own datacenter instead of renting floorspace for pennies on the dollar from Switch or Equinix (although Equinix is kind of pricey) or Windstream or insert provider - then the datacenter is a rounding error in your 10-year IT budget. Then you clearly don't understand the TCO of using a data center vs cloud. The datacenter cost itself isn't the cost - the people who operate it are. On a computing resource basis can you operate SQL Server on a rack in Equinix for cheaper than that in AWS? Sure, but who is going to pay for the person to patch it every week, or upgrade it to the latest version, or upgrade the SAN disk, or upgrade the hardware every 3 years, etc. etc. Let's take this a step further - who is going to hire these people? And fire them? And train them? These are all transactions costs that get hidden in the "$0.20/hr cost" of spinning up an EC2 instance. If you think cloud computing it's just about a lift and shift from a traditional data center, then you don't understand how to calculate TCO. P.S. - For the record, many large internal IT groups do in fact do simple lift and shifts from traditional data centers and don't fully realize the value of cloud computing resources. However, these companies will usually mature eventually and evolve services to ones that are more advantageous (i.e. setup SQL Server on EC2 and the migrate to RDS). AWS (et al) have the ability to do this with ease.
- pbalau 5y ago
  > On a computing resource basis can you operate SQL Server on a rack in Equinix for cheaper than that in AWS? Sure, but who is going to pay for the person to patch it every week, or upgrade it to the latest version, or upgrade the SAN disk, or upgrade the hardware every 3 years, etc. etc. Let's take this a step further - who is going to hire these people? And fire them? And train them? These are all transactions costs that get hidden in the "$0.20/hr cost" of spinning up an EC2 instance. Who cares? This is IT and is not relevant. And they have Brent and he will solve everything.
- stuff4ben 5y ago
  Surely for any company of size you understand there will still be people managing the AWS infra. Having tens to hundreds of small dev groups go at it willy nilly is a recipe for disaster and overspending. Someone still has to patch the server instances, manage the people who access it, train them on how to use AWS, etc. Those costs don't go away because you've moved to the cloud.
_huayra_ 5y ago
Cloud providers (especially Amazon) can definitely be a bit greedy / shady about pricing and lock-in, but the key thing is that they have room to lower prices whereas a bespoke data center does not. Most data center machines run at a tiny fraction of their capabilities; seeing a CPU running at 25% is definitely something that stands out. Because of this, cloud operators can oversubscribe these machines in various ways, packing workloads of multiple tenants onto a single box while a single-tenant bespoke DC can't. They can also do things that schedule DC-wide resources (namely power) in ways that renting space in a colo cannot; they can shed load in insightful ways when a power overtopping event happens that can minimize the impact (e.g. throttle / nuke VMs that can operate in a degraded mode before impacting other critical services). No hetergeneous, multiple-private tenant DC is going to oversubscribe their rack- or row-level power delivery to tenants and require them to hook in to a smart system to manage overtopping events; they just won't trust their tenants to do the right thing, as they have no power over the tenants' machines and power draws; the cloud providers do. At the end of the day, the economies of scale are just hard to beat, in terms of COGS. The real question is whether they'll pass the savings from this on to the customers or the shareholders' dividends, and so far it seems like it is the latter.
- ryanjkirk 5y ago
  > Most data center machines run at a tiny fraction of their capabilities; seeing a CPU running at 25% is definitely something that stands out. Are you familiar with VMware?
  - rstupek 5y ago
    Is VMware free?
- ec109685 5y ago
  AWS doesn’t oversubscribe on EC2 instances outside the T family.
- hodgesrm 5y ago
  > Most data center machines run at a tiny fraction of their capabilities; seeing a CPU running at 25% is definitely something that stands out. Kubernetes fixes that, as does VMware. They can both distribute workloads efficiently to ensure good utilization. VMware has done that for 20 years.
aenis 5y ago
Wholehartedly disagree. I've been in large scale IT for 20+ years, working for deep pocketed companies. The costs of running old school IT with on prem data centers were orders of magnitude higher vs. what we pay for well architected serverless solutions these days. I remember having a $30MM/year line item for Oracle alone, running on expensive hardware that was sitting idle 90% of the time. Time sharing expensive hardware is the way to go. Another thing is costs of building something new. In the old world it would mean doing the math on resource needs now and in the future, since adding Hw meant capital expenditure. No such problem now and its easier to get started, see if it works, and scale or kill early. And of course as others have mentioned, keeping it all running requires expensive expertise. Esp. now with the internet that looks like automated wild west. You cant release a service without it being auto-probed and auto-attacked within 2-3hrs. Nope. With the cloud, the costs have went down for any large coMpany, even those that just moved the old crap to VMs running on someone elses hw.
- darksaints 5y ago
  > I remember having a $30MM/year line item for Oracle alone I think I found your problem...
- nojito 5y ago
  >Nope. With the cloud, the costs have went down for any large coMpany, even those that just moved the old crap to VMs running on someone elses hw. Are you implying that there isn't idle cloud hardware?
  - dagw 5y ago
    If a cloud machine is idle is you can shut it down and stop paying for it.
  - acdha 5y ago
    There is, but there's less if you're using managed services where possible and it's visible in a way which wasn't true for many large organizations. I can go into any cloud provider's console and see a detailed breakdown of what I'm paying and why, and how much of it is over/under-utilized, but getting that same report from an enterprise IT department is usually a project.
opportune 5y ago
The problem is not just the datacenter itself but 1. all the personnel needed to operate the datacenter/private cloud 2. the lack of agility when everything needs to go through internal IT provisioning (at some companies this is a huge bottleneck).
ec109685 5y ago
Fantasy football is actually interesting in that it happens “off hours”. So if you are a company that has an “on hours” business, running Fantasy Football is free if you utilize multi tenant hardware.
- handrous 5y ago
  > Fantasy football is actually interesting in that it happens “off hours”. I entirely associate fantasy football with working hours. Ditto a variety of other kinds of sports betting. Like, I know people do these at home, too, but there's a lot of it happening during business hours, at offices.
  - ec109685 5y ago
    The peaks though (by a significant margins) are when games are on and people are watching the games unfold.
41209 5y ago
It's all good until you remember you have to now higher an entire team to manage your bare metal servers. Companies don't like adding new employees, even when it's comparable in cost to hiring an outside vendor or paying for an outside service. Firing 10 people is very messy, reducing your AWS budget by 20%, however is not.
slaymaker1907 5y ago
Are you also factoring in that you often need 2+ DCs for redundancy? That seems like a lot of operational difficulty unless your company already had a physical presence in multiple locations. I know there are also a bunch of requirements for where you can store user data these days due to privacy regulations.

by tw04 5y ago

prpl 5y ago

A few things are relatively cheap: Compute on preemptable VMs/spot instances is cheaper in cloud. Object storage, if you would have required a vendor support contract on prem, up to a PB or so. managed kubernetes, if you would have required a vendor support contract on prem.

manigandham 5y ago

The money is not the issue. Cost is not the same as value.

Closi 5y ago
Cost is a key component of value though.
tw04 5y ago
>The money is not the issue. Money is ALWAYS the issue. Whether you can pass along increased cost to your customers or not is entirely dependent on what your business is and the competitive landscape of it. I don't care how "agile" the cloud makes you, if your cost of IT is more expensive than the revenue you bring in, you're bankrupt. It's the running joke around here about people who build autoscaling k8s based infrastructure for a site that could be hosted on a $5/month VPS with even a little bit of tuning and caching. Complexity for the sake of complexity is called stupidity unless you're doing it as part of an educational experience. Going to production with needless complexity is a great way to ruin a good idea. >Cost is not the same as value. I guess it's a good thing I didn't state that cost is the same thing as value?
- exdsq 5y ago
  There's more to it than just the tech and what is cheapest though. Like risk management. Cost might be higher for a less-risky thing.
- londons_explore 5y ago
  > build autoscaling k8s based infrastructure for a site that could be hosted on a $5/month VPS I think the goal is that it should be the same amount of engineering time to set up both options, and both should cost $5 per month. Today, that is broadly untrue. Although a few niches like appengine offer this.
- devoutsalsa 5y ago
  It's all fun & games until you can't make payroll.
- pm90 5y ago
  If we’re discussing major corporations, they are willing to pay a premium for predictability and reliability of service. Cloud vendors provide just that: if you want more resources, you can scale up until you hit capacity, then you can call the account manager to increase it. Very little downtime (except for the occasional outages). All of these are known processes. With operating your own DC, you have to hire really great people and hope they don’t fuck up. You have to plan much in advance for capacity changes. It’s very much possible that something goes wrong in your compute infrastructure and simply nobody in your team has the expertise to fix it. Put it another way: can you pay/hire the same kinds of engineers that work at AWS/GCP? If not, cloud is the safer route. Cost does matter, and this is why companies sign multi year contracts to get discounts. But business continuity is critical.

mbesto 5y ago

You have some really good points, but your premise is simply not based in reality. You're painting the picture as if AWS is the 2nd version of the IBM saying: "No one ever got fired for using AWS". This is simply not true...at least not the part where "IBM" (i.e. AWS now) is somehow inferior to the competition. > If you are large enough to need to build your own datacenter instead of renting floorspace for pennies on the dollar from Switch or Equinix (although Equinix is kind of pricey) or Windstream or insert provider - then the datacenter is a rounding error in your 10-year IT budget. Then you clearly don't understand the TCO of using a data center vs cloud. The datacenter cost itself isn't the cost - the people who operate it are. On a computing resource basis can you operate SQL Server on a rack in Equinix for cheaper than that in AWS? Sure, but who is going to pay for the person to patch it every week, or upgrade it to the latest version, or upgrade the SAN disk, or upgrade the hardware every 3 years, etc. etc. Let's take this a step further - who is going to hire these people? And fire them? And train them? These are all transactions costs that get hidden in the "$0.20/hr cost" of spinning up an EC2 instance. If you think cloud computing it's just about a lift and shift from a traditional data center, then you don't understand how to calculate TCO. P.S. - For the record, many large internal IT groups do in fact do simple lift and shifts from traditional data centers and don't fully realize the value of cloud computing resources. However, these companies will usually mature eventually and evolve services to ones that are more advantageous (i.e. setup SQL Server on EC2 and the migrate to RDS). AWS (et al) have the ability to do this with ease.

pbalau 5y ago
> On a computing resource basis can you operate SQL Server on a rack in Equinix for cheaper than that in AWS? Sure, but who is going to pay for the person to patch it every week, or upgrade it to the latest version, or upgrade the SAN disk, or upgrade the hardware every 3 years, etc. etc. Let's take this a step further - who is going to hire these people? And fire them? And train them? These are all transactions costs that get hidden in the "$0.20/hr cost" of spinning up an EC2 instance. Who cares? This is IT and is not relevant. And they have Brent and he will solve everything.
stuff4ben 5y ago
Surely for any company of size you understand there will still be people managing the AWS infra. Having tens to hundreds of small dev groups go at it willy nilly is a recipe for disaster and overspending. Someone still has to patch the server instances, manage the people who access it, train them on how to use AWS, etc. Those costs don't go away because you've moved to the cloud.

_huayra_ 5y ago

Cloud providers (especially Amazon) can definitely be a bit greedy / shady about pricing and lock-in, but the key thing is that they have room to lower prices whereas a bespoke data center does not. Most data center machines run at a tiny fraction of their capabilities; seeing a CPU running at 25% is definitely something that stands out. Because of this, cloud operators can oversubscribe these machines in various ways, packing workloads of multiple tenants onto a single box while a single-tenant bespoke DC can't. They can also do things that schedule DC-wide resources (namely power) in ways that renting space in a colo cannot; they can shed load in insightful ways when a power overtopping event happens that can minimize the impact (e.g. throttle / nuke VMs that can operate in a degraded mode before impacting other critical services). No hetergeneous, multiple-private tenant DC is going to oversubscribe their rack- or row-level power delivery to tenants and require them to hook in to a smart system to manage overtopping events; they just won't trust their tenants to do the right thing, as they have no power over the tenants' machines and power draws; the cloud providers do. At the end of the day, the economies of scale are just hard to beat, in terms of COGS. The real question is whether they'll pass the savings from this on to the customers or the shareholders' dividends, and so far it seems like it is the latter.

ryanjkirk 5y ago
> Most data center machines run at a tiny fraction of their capabilities; seeing a CPU running at 25% is definitely something that stands out. Are you familiar with VMware?
- rstupek 5y ago
  Is VMware free?
ec109685 5y ago
AWS doesn’t oversubscribe on EC2 instances outside the T family.
hodgesrm 5y ago
> Most data center machines run at a tiny fraction of their capabilities; seeing a CPU running at 25% is definitely something that stands out. Kubernetes fixes that, as does VMware. They can both distribute workloads efficiently to ensure good utilization. VMware has done that for 20 years.

aenis 5y ago

Wholehartedly disagree. I've been in large scale IT for 20+ years, working for deep pocketed companies. The costs of running old school IT with on prem data centers were orders of magnitude higher vs. what we pay for well architected serverless solutions these days. I remember having a $30MM/year line item for Oracle alone, running on expensive hardware that was sitting idle 90% of the time. Time sharing expensive hardware is the way to go. Another thing is costs of building something new. In the old world it would mean doing the math on resource needs now and in the future, since adding Hw meant capital expenditure. No such problem now and its easier to get started, see if it works, and scale or kill early. And of course as others have mentioned, keeping it all running requires expensive expertise. Esp. now with the internet that looks like automated wild west. You cant release a service without it being auto-probed and auto-attacked within 2-3hrs. Nope. With the cloud, the costs have went down for any large coMpany, even those that just moved the old crap to VMs running on someone elses hw.

darksaints 5y ago
> I remember having a $30MM/year line item for Oracle alone I think I found your problem...
nojito 5y ago
>Nope. With the cloud, the costs have went down for any large coMpany, even those that just moved the old crap to VMs running on someone elses hw. Are you implying that there isn't idle cloud hardware?
- dagw 5y ago
  If a cloud machine is idle is you can shut it down and stop paying for it.
- acdha 5y ago
  There is, but there's less if you're using managed services where possible and it's visible in a way which wasn't true for many large organizations. I can go into any cloud provider's console and see a detailed breakdown of what I'm paying and why, and how much of it is over/under-utilized, but getting that same report from an enterprise IT department is usually a project.

opportune 5y ago

The problem is not just the datacenter itself but 1. all the personnel needed to operate the datacenter/private cloud 2. the lack of agility when everything needs to go through internal IT provisioning (at some companies this is a huge bottleneck).

ec109685 5y ago

Fantasy football is actually interesting in that it happens “off hours”. So if you are a company that has an “on hours” business, running Fantasy Football is free if you utilize multi tenant hardware.

handrous 5y ago
> Fantasy football is actually interesting in that it happens “off hours”. I entirely associate fantasy football with working hours. Ditto a variety of other kinds of sports betting. Like, I know people do these at home, too, but there's a lot of it happening during business hours, at offices.
- ec109685 5y ago
  The peaks though (by a significant margins) are when games are on and people are watching the games unfold.

41209 5y ago

It's all good until you remember you have to now higher an entire team to manage your bare metal servers. Companies don't like adding new employees, even when it's comparable in cost to hiring an outside vendor or paying for an outside service. Firing 10 people is very messy, reducing your AWS budget by 20%, however is not.

slaymaker1907 5y ago

Are you also factoring in that you often need 2+ DCs for redundancy? That seems like a lot of operational difficulty unless your company already had a physical presence in multiple locations. I know there are also a bunch of requirements for where you can store user data these days due to privacy regulations.