6 ms·

As a CTO of a number of small startups, I am still struggling to understand what exactly AWS and other cloud providers give you to justify the markup. And yes

by oersted 11mo ago

As a CTO of a number of small startups, I am still struggling to understand what exactly AWS and other cloud providers give you to justify the markup. And yes we’ve been heavy users of both AWS and Google Cloud for years, mainly because of the credits they initially provided, but also used VMs, dedicated servers and other services from Hetzner and OVH extensively. In my experience, in terms of availability and security there’s not much difference in practice. There are tons of good tools nowadays to treat a physical server or a cluster of them as a cloud or a PaaS, it’s not really more work or responsibility, often it is actually simpler depending on the setup you choose. Most workloads do not require flexible compute capability and it’s also easy and fast to get it from these cheaper providers when you need to. I feel like the industry has collectively accepted that Cloud prices are a cost of doing business and unquestionable, “nobody ever got fired for choosing IBM”. Thinking about costs from first principles is an important part of being an engineer.

mgaunard 11mo ago
What AWS gives you is the ability to spin up dozens if not thousands of hosts in a single click. If you run your own hardware, getting stuff shipped to a datacenter and installed is 2 to 4 weeks (and potentially much longer based on how efficient your pipeline is)
- continuational 11mo ago
  But on Hetzner, you can usually get a dedicated server installed and ready tomorrow.
  - h33t-l4x0r 11mo ago
    Hetzner is oversold. It's not appropriate for production in the same sense that EC2 obviously is. It's fine for staging though.
    - nake89 11mo ago
      Out of curiosity, how is a dedicated server oversold?
      - tempestn 11mo ago
        I'm not familiar with Hetzner personally, but maybe they mean the uplink? I've found that with some smaller providers, advertising 10Gbit, but you rarely get close to that speed in reality.
  - giuseony 11mo ago
    Even sooner than that. I bought a dedicated server on auction and it was ready 2h after the payment.
- tempestn 11mo ago
  If you own your own hardware, but you can provision a leased dedicated server from many different providers in an hour or three, and still pay far less than for comparable hardware from AWS.
  - mgaunard 11mo ago
    That sounds like a good deal, what providers offer this?
    - graemep 11mo ago
      OVH, Hetzner, Scaleway. Lots of smaller providers. Most people who offer dedicated servers do this.
- noir_lord 11mo ago
  Which is an awesome capability, if you need it. I suspect that if you broke projects on AWS down by the numbers, the vast majority don't needed it. There are other benefits to using AWS (and drawbacks) bit "easy scaling" isn't just premature optimisation because if you build something to do something it's never going to do that's not optimisation it's simply waste.
  - mgaunard 11mo ago
    They need it at the beginning to get started quickly, then they don't but don't bother moving out. Not too different from how many other lines of business get their clients in the door.
- oersted 11mo ago
  What really needs thousands of hosts nowadays? Even if you have millions of users. Computers are plenty fast now and leveraging that is not any harder if you choose the right stack. And even if you are building with microservices, most standard servers can handle dozens in a single machine at once. They are all mostly doing network calls with minimal compute. Even better actually if they are in the same host and the network doesn’t get involved. If you want to, there are simple tools to hook a handful of them as a cluster and/or instantly spawn extra slightly costlier VMs in case of failure or a spike in usage, if a short outage is really a world-ending event, which it isn’t for almost every software system or business. These capabilities have not been exclusive to the major cloud providers for years. Of course we are generalizing a lot by this point, I’d be happy to discuss specific cases.
aurareturn 11mo ago
When your cheap dedicated server goes down and your admin is on holiday and you have hundreds of angry customers calling you, you'll get it. Or you need to restore your Postgres database and you find out that the backups didn't work. And finally you have a brilliant idea of hiring a second $150k/year dev ops admin so that at least one is always working and they can check each other's work. Suddenly, you're spending $300k on two dev ops admins alone and the cost savings of using cheaper dedicated servers are completely gone.
- esseph 11mo ago
  Sure, but AWS has more downtime than I do :-)
  - aurareturn 11mo ago
    Yes but you go on holidays but AWS does not.
  - fragmede 11mo ago
    If we assume that you're a human being that sleeps, say 8 hrs/day, and not an LLM, that leaves you with 16 hours of consciousness a day, for an uptime of 66%. That's upsidedown nines. You don't even crack one nine of uptime. If we assume you've a diet of meth and cocaine, and only sleep 2 hours a day, that still puts you at only like 92% uptime.
    - esseph 11mo ago
      > If we assume that you're a human being I'll have you know I am a cantaloupe, you insensitive clod!
- grey-area 11mo ago
  It is statistically far more likely that your cloud service will go down for hours or days, and you will have no recourse and will just have to wait till AWS manage to resolve it.
peanut-walrus 11mo ago
It makes sense to think about price in the context of your business. If your entire infra cost is a rounding error on your balance sheet, of course you would pick the provider with the best features and availability guarantees (choose IBM/AWS). If your infra cost makes up a significant percentage of your operating expenses, you will start spending engineering effort to lower the cost. That's why AWS can get away with charging the prices they do, even though it is expensive, for most companies it is not expensive enough to make it worth their while to look for cheaper alternatives.
- oersted 11mo ago
  It’s often less about engineering effort and more about taking some small risks to try less mainstream (but still relatively mature) alternatives by reasoning from first principles and doing a bit of homework. From our experience, you can actually end up in a situation that requires less engineering effort and be more stable, while saving on costs, if you dare to go to a bit lower abstraction layers. Sometimes being closer to the metal is simpler, not more complex. And in practice complexity is much more often the cause of outages rather than hardware reliability.
kamaal 11mo ago
>>As a CTO of a number of small startups, I am still struggling to understand what exactly AWS and other cloud providers give you to justify the markup. If you are having a company that warrants building a data center, then AWS does not add much. Other wise you face the 'if you want to build apple pie from scratch, you need to first invent the universe' problem. Simply put you can get started right on day one, in a pay as you go model. Like you can write code, deploy and ship from the very first day, instead of having to go deep down the infrastructure rabbit hole. Plus shutting down things is easy as well. Things don't workout? Good news! You can shut down the infrastructure that very day instead of having to worry about the capital expenditure spent to build infrastructure, and without having to worry about its use later. Simply put, AWS is infrastructure you can hire and fire at will.
paranoidrobot 11mo ago
> “nobody ever got fired for choosing IBM” Agreed, there's definitely a heavy element of that to it. But, at the risk of again being labelled as an AWS Shill - there's also other benefits. If your organisation needs to deploy some kind of security/compliance tools to help with getting (say) SOC2 certification - then there's a bunch of tools out there to help with that. All you have to do then is plug them into your AWS organisation. They can run a whole bunch of automated policy checks to say you're complying with whatever audit requirements. If you're self-hosting, or using Hetzner - well, you're going to spend a whole lot more time providing evidence to auditors. Same goes with integrating with vendors. Maybe you want someone to load/save data for you - no problems, create an AWS S3 bucket and hand them an AWS IAM Role and they can do that. No handing over of creds. There's a bunch of semi-managed services where a vendor will spin up EC2 instances running their special software, but since it's running in your account - you get more control/visiblity into it. Again, hand over an AWS IAM Role and off you go. It's the Slack of IAAS - it might not be the fastest, it's definitely not the cheapest, and you can roll your own for sure. But then you miss out on all these integrations that make life easier.
Ancapistani 11mo ago
As a Staff+ Engineer -- Principal in a couple of weeks -- for me it's about "batteries included", support availability, and ability to hire. Using a cloud platform means that while your needs are small, you're overpaying. Where it pays off is when you have a new requirement that needs to be met quickly. I've done my share of managing database instances in the past. I can spin up a new RDS Postgres instance in much less time than I can configure one from scratch, though. Do we need a read replica? Multi-site failover? Do we need to connect it to Okta, or Formal, so we can stand up a process to provision access to specific databases, tables, or even columns? All of those things I can do significantly faster and more quickly on AWS than I can do it by hand. What if a NoSQL database is the right solution for us? I have much less experience adminning those, so will either have to allocate a fair amount of my time to skill up or hire someone who already has those skills. Need a scheduled task? Sure, I could set up a Jenkins server somewhere and we could use that... or we could just add an ECS scheduled task to our existing cluster. Need an API endpoint to handle inbound Zoom events and forward them to an internal queue? Sure, I can set up a new VPC for that... that'll be a couple of days... or we whip up a Lambda, hook it up to API Gateway, and be up and running in a couple of hours. AWS helps me do more in less time - and my time is a cost to the business. It's also extremely flexible, and will let us add things far more quickly than we otherwise could. IMO, the correct comparison isn't "what would it cost to run this platform on Hetzner?" - it's "What would it cost to run it, plus what would cost to acquire the talent to build it, plus retain that talent to maintain it?" AWS isn't competing with other infrastructure providers. They're competing with other providers and the salaries of the engineers you need to make them work.

by oersted 11mo ago

mgaunard 11mo ago

What AWS gives you is the ability to spin up dozens if not thousands of hosts in a single click. If you run your own hardware, getting stuff shipped to a datacenter and installed is 2 to 4 weeks (and potentially much longer based on how efficient your pipeline is)

continuational 11mo ago
But on Hetzner, you can usually get a dedicated server installed and ready tomorrow.
- h33t-l4x0r 11mo ago
  Hetzner is oversold. It's not appropriate for production in the same sense that EC2 obviously is. It's fine for staging though.
  - nake89 11mo ago
    Out of curiosity, how is a dedicated server oversold?
    - tempestn 11mo ago
      I'm not familiar with Hetzner personally, but maybe they mean the uplink? I've found that with some smaller providers, advertising 10Gbit, but you rarely get close to that speed in reality.
- giuseony 11mo ago
  Even sooner than that. I bought a dedicated server on auction and it was ready 2h after the payment.
tempestn 11mo ago
If you own your own hardware, but you can provision a leased dedicated server from many different providers in an hour or three, and still pay far less than for comparable hardware from AWS.
- mgaunard 11mo ago
  That sounds like a good deal, what providers offer this?
  - graemep 11mo ago
    OVH, Hetzner, Scaleway. Lots of smaller providers. Most people who offer dedicated servers do this.
noir_lord 11mo ago
Which is an awesome capability, if you need it. I suspect that if you broke projects on AWS down by the numbers, the vast majority don't needed it. There are other benefits to using AWS (and drawbacks) bit "easy scaling" isn't just premature optimisation because if you build something to do something it's never going to do that's not optimisation it's simply waste.
- mgaunard 11mo ago
  They need it at the beginning to get started quickly, then they don't but don't bother moving out. Not too different from how many other lines of business get their clients in the door.
oersted 11mo ago
What really needs thousands of hosts nowadays? Even if you have millions of users. Computers are plenty fast now and leveraging that is not any harder if you choose the right stack. And even if you are building with microservices, most standard servers can handle dozens in a single machine at once. They are all mostly doing network calls with minimal compute. Even better actually if they are in the same host and the network doesn’t get involved. If you want to, there are simple tools to hook a handful of them as a cluster and/or instantly spawn extra slightly costlier VMs in case of failure or a spike in usage, if a short outage is really a world-ending event, which it isn’t for almost every software system or business. These capabilities have not been exclusive to the major cloud providers for years. Of course we are generalizing a lot by this point, I’d be happy to discuss specific cases.

aurareturn 11mo ago

When your cheap dedicated server goes down and your admin is on holiday and you have hundreds of angry customers calling you, you'll get it. Or you need to restore your Postgres database and you find out that the backups didn't work. And finally you have a brilliant idea of hiring a second $150k/year dev ops admin so that at least one is always working and they can check each other's work. Suddenly, you're spending $300k on two dev ops admins alone and the cost savings of using cheaper dedicated servers are completely gone.

esseph 11mo ago
Sure, but AWS has more downtime than I do :-)
- aurareturn 11mo ago
  Yes but you go on holidays but AWS does not.
- fragmede 11mo ago
  If we assume that you're a human being that sleeps, say 8 hrs/day, and not an LLM, that leaves you with 16 hours of consciousness a day, for an uptime of 66%. That's upsidedown nines. You don't even crack one nine of uptime. If we assume you've a diet of meth and cocaine, and only sleep 2 hours a day, that still puts you at only like 92% uptime.
  - esseph 11mo ago
    > If we assume that you're a human being I'll have you know I am a cantaloupe, you insensitive clod!
grey-area 11mo ago
It is statistically far more likely that your cloud service will go down for hours or days, and you will have no recourse and will just have to wait till AWS manage to resolve it.

peanut-walrus 11mo ago

It makes sense to think about price in the context of your business. If your entire infra cost is a rounding error on your balance sheet, of course you would pick the provider with the best features and availability guarantees (choose IBM/AWS). If your infra cost makes up a significant percentage of your operating expenses, you will start spending engineering effort to lower the cost. That's why AWS can get away with charging the prices they do, even though it is expensive, for most companies it is not expensive enough to make it worth their while to look for cheaper alternatives.

oersted 11mo ago
It’s often less about engineering effort and more about taking some small risks to try less mainstream (but still relatively mature) alternatives by reasoning from first principles and doing a bit of homework. From our experience, you can actually end up in a situation that requires less engineering effort and be more stable, while saving on costs, if you dare to go to a bit lower abstraction layers. Sometimes being closer to the metal is simpler, not more complex. And in practice complexity is much more often the cause of outages rather than hardware reliability.

kamaal 11mo ago

>>As a CTO of a number of small startups, I am still struggling to understand what exactly AWS and other cloud providers give you to justify the markup. If you are having a company that warrants building a data center, then AWS does not add much. Other wise you face the 'if you want to build apple pie from scratch, you need to first invent the universe' problem. Simply put you can get started right on day one, in a pay as you go model. Like you can write code, deploy and ship from the very first day, instead of having to go deep down the infrastructure rabbit hole. Plus shutting down things is easy as well. Things don't workout? Good news! You can shut down the infrastructure that very day instead of having to worry about the capital expenditure spent to build infrastructure, and without having to worry about its use later. Simply put, AWS is infrastructure you can hire and fire at will.

paranoidrobot 11mo ago

> “nobody ever got fired for choosing IBM” Agreed, there's definitely a heavy element of that to it. But, at the risk of again being labelled as an AWS Shill - there's also other benefits. If your organisation needs to deploy some kind of security/compliance tools to help with getting (say) SOC2 certification - then there's a bunch of tools out there to help with that. All you have to do then is plug them into your AWS organisation. They can run a whole bunch of automated policy checks to say you're complying with whatever audit requirements. If you're self-hosting, or using Hetzner - well, you're going to spend a whole lot more time providing evidence to auditors. Same goes with integrating with vendors. Maybe you want someone to load/save data for you - no problems, create an AWS S3 bucket and hand them an AWS IAM Role and they can do that. No handing over of creds. There's a bunch of semi-managed services where a vendor will spin up EC2 instances running their special software, but since it's running in your account - you get more control/visiblity into it. Again, hand over an AWS IAM Role and off you go. It's the Slack of IAAS - it might not be the fastest, it's definitely not the cheapest, and you can roll your own for sure. But then you miss out on all these integrations that make life easier.

Ancapistani 11mo ago

As a Staff+ Engineer -- Principal in a couple of weeks -- for me it's about "batteries included", support availability, and ability to hire. Using a cloud platform means that while your needs are small, you're overpaying. Where it pays off is when you have a new requirement that needs to be met quickly. I've done my share of managing database instances in the past. I can spin up a new RDS Postgres instance in much less time than I can configure one from scratch, though. Do we need a read replica? Multi-site failover? Do we need to connect it to Okta, or Formal, so we can stand up a process to provision access to specific databases, tables, or even columns? All of those things I can do significantly faster and more quickly on AWS than I can do it by hand. What if a NoSQL database is the right solution for us? I have much less experience adminning those, so will either have to allocate a fair amount of my time to skill up or hire someone who already has those skills. Need a scheduled task? Sure, I could set up a Jenkins server somewhere and we could use that... or we could just add an ECS scheduled task to our existing cluster. Need an API endpoint to handle inbound Zoom events and forward them to an internal queue? Sure, I can set up a new VPC for that... that'll be a couple of days... or we whip up a Lambda, hook it up to API Gateway, and be up and running in a couple of hours. AWS helps me do more in less time - and my time is a cost to the business. It's also extremely flexible, and will let us add things far more quickly than we otherwise could. IMO, the correct comparison isn't "what would it cost to run this platform on Hetzner?" - it's "What would it cost to run it, plus what would cost to acquire the talent to build it, plus retain that talent to maintain it?" AWS isn't competing with other infrastructure providers. They're competing with other providers and the salaries of the engineers you need to make them work.