5 ms·

EC2 is vastly more cost efficient than using all of those managed services unless you only have tiny on-demand usage scenarios.

andy_ppp 4y ago
Are you including the wages of the extra people to support building and managing such services?
- datalopers 4y ago
  extra people? no because I hire generalists actually capable of building software, not overpaid kids only capable of gluing aws services together.
  - etothepii 4y ago
    How much of your developer time gets lost to maintaining servers. I used to think this sort of thing was a valuable use of my time but I now have over $200k in annual revenue and we pay less that $5 a month in raw compute (and that's nearly all S3). My Co-founder is always worried about the cost of our AWS but so far we just haven't witnessed it. My mandate has always been the price should be able to scale to 0. So no EC2, has made building some longer running tasks a bit more complicated, but that's just because we didn't know how to do it before we started.
    - datalopers 4y ago
      Maintaining servers? Nah, you deploy a container and don't think about it. ECS is completely free to use. Anyway, you have a tiny on-demand business, as I caveated above. You could run it just fine on a nano ec2 instance if you needed, which is $2-3/mo.
    - moonchrome 4y ago
      The more I work with cloud the more I see this 'saves time on infrastructure' being a half-truth at best - we still have overwhelmed DevOps people who, unlike managing on-prem services, have 0 insight or control over what goes on inside AWS. And the services require a decent amount of hand holding, proprietary know how, etc. In the end you replaced sysadmin with DevOps and got up charged multiples.
      - zo1 4y ago
        Agreed, devops (and whatever the hell cloud wranglers are being sold as today) is the new priesthood. It's meta-level job creation and job security.
      - acdha 4y ago
        It's certainly possible to overspend on cloud services but in most cases when I see comparisons people tend to forget to fully include their true costs for things like staff time, infrastructure, etc. and especially things like opportunity cost for the delays caused by provisioning infrastructure, less capable interfaces (e.g. if you're swapping Terraform for a Jira ticket), or the technical decisions people make because they have fewer services available (this could be a Lambda function but we don't have that so now we need to manage a full VM). Some examples which come to mind: * Comparing S3 to the on-premise tape system but ignoring the fact that it involved an expensive tape robot, DR had access times measured in days, etc. * Comparing S3 to on-premise storage, ignoring the difference in redundancy, forcing users to handle bitrot at the application level, and the procurement process meaning that when they ran out of storage it took months of telling people they couldn't allocate more. * Saying their devops engineer cost twice as much as their sysadmins (true) but then when you look the devops engineer is using automation and managing literally a hundred times more systems than the “cheaper” ops team. * Saying their cost to run a VM was cheaper than EC2, which true if you looked only at the instance but not when you calculated how much they were spending on underutilized VM hosts, power / HVAC, facilities people, etc. It's totally possible to beat a major cloud provider on costs[1] but you usually need to be operating at fairly large scale to even approach the break-even point. This is especially true when you have regulatory or policy requirements for things like security and you include the cost of monitoring all of the things which fall under the cloud provider's responsibility — management networks, firmware version management, robust logging and IAM, etc. are all easy to accidentally exclude when making comparisons. 1. Network egress as the most obvious area to attack
        moonchrome 4y ago
        But that's the part that I'm not getting - it's either AWS or building storage shelters and tape robots. There are hosting providers like Hetzner that will rent you bare metal machines so you don't have to deal with actual hardware, their networking is basically unmetered (last I checked), they will also sell you spot VMs, blob storage service, etc. The pricing difference is in integer multiples and much less "gotchas" or lock-in (they are usually dumbed down). They won't sell you fancy SAAS - but the ammount of fiddling required to get most of AWS services working in practice and debugging it - you're not very far off from using the OSS solution they are reselling - chances are your devops/sysadmin team can do that as well. And there's plenty cloud agnostic SAAS as well. I mean if you're a big corp that can accept the inefficiency for the sake of having one bill to pay at the end of the month, SLA you can use to cover your ass to higher ups and have the "nobody got fired for picking AWS" mentality I get it. But for startups and small companies ? All this "magical simplicity" that AWS/Azure/GCP is supposedly brining to the table - I'm not seeing it. Real redundancy (across AZ) is still really hard and requires extensive planning - no magic there. Dynamic scaling "savings" is often offset by being multiples more expensive to begin with and I have to deal with warmup issues, and the pricing is sneaky/unpredictable, easy to fuck up. Lambda architecture is pushing towards distributed microservices by default - the hardest thing to debug in practice. So end of the day, I'm forced to work around arbitrary pricing models rather then technical limitations, debugging opaque black box services, not being "big enough" to get actual support, overly generic/complex systems built as "one size fits all" solutions, or get sold in to practically abandoned "service" because it's available etc. And it's not like being first party service is any indication of quality - as soon as you move off of the most used stuff you get to see some really shit services in my experience.
  - throwaway894345 4y ago
    You still have to pay your generalists for the time they spend writing and maintaining that bespoke software, and moreover you can hire competent developers to develop using cloud services. There’s no reason to assume that cloud services require incompetent developers, and indeed the cheapest solution is often to employ competent developers to integrate cloud services (whose development and operations costs are shared among all customers, whereas your company is footing the whole bill for your bespoke services).
etothepii 4y ago
Or any kind of demand spikes. If you are using lambda and suddenly need to run a million lambda you just can. If you want to have the ec2 support to run a million lambda at some time you are going to pay for a lot of sleeping computers.
- pclmulqdq 4y ago
  If your spikes go 5x over your base load, EC2 is still cheaper. Even if your load spikes are 10-20x over your base load, EC2 can still be cheaper. Past that, lambda is your better bet. If you are comparing to Hetzner or OVHCloud, you those numbers are 25x (definitely cheaper) and 50-100x (possibly cheaper).
acdha 4y ago
That's definitely not true for “all of those managed services” — you can't replicate CloudFront on EC2, for example, and it'd be extremely unlikely that you could build your own S3 replacement on top of EC2 storage without spending considerably more on operations than you save. On-premise storage is cheaper but you're still going to struggle to see a cost savings unless you either don't care about reliability or availability and/or buy your data by the petabyte. For the other services, it's still going to depend heavily on your usage, staff time, and operational efficiencies. For example, if you replace SQS with RabbitMQ you need to manage multiple EC2 servers for reliability but those servers might be significantly underutilized based on your traffic levels. Whether or not you save money depends on how much you pay your operators and how many messages you use: if you use less than the free tier's million messages per month, there's no way to beat it. Each million requests costs $0.40 or less, so if you pay your ops person $50k annually (haha) you'd need to be somewhat over 120 million messages per month to pay for them to spend a single hour on O&M even before factoring in your EC2 usage.
throwaway894345 4y ago
Maybe if you ignore the costs associated with operating EC2 or if you have a steady, predictable workload. Or perhaps if you have enormous (on the order of FAANG) scale to absorb those operational costs.
- devonkim 4y ago
  It gets odd in this distribution of potential customers because once your scale gets large enough again you're more than capable in theory of provisioning your own datacenters. MS, Google, and Amazon all have major public clouds while Apple and Facebook / Meta are deploying their own bare metal for their core infrastructure and some trace usage of the others most likely for compatibility reasons. Only Netflix is basically zero-DC in the new school infrastructure sense and even they used to have bare metal for old infrastructure like Oracle servers and Big Iron for I think accounting until so long ago. The real target customer IMO for these big public clouds are non-technical corporations whose primary revenue and personnel competencies aren't derived from mastery of software but primarily around other verticals like agriculture, manufacturing, etc. Insert Werner Vogels' criticism of the HBR article emphasizing focusing upon core competencies and outsourcing non-core competencies like IT (read: because technology is so important to every business at scale now).

andy_ppp 4y ago

Are you including the wages of the extra people to support building and managing such services?

datalopers 4y ago
extra people? no because I hire generalists actually capable of building software, not overpaid kids only capable of gluing aws services together.
- etothepii 4y ago
  How much of your developer time gets lost to maintaining servers. I used to think this sort of thing was a valuable use of my time but I now have over $200k in annual revenue and we pay less that $5 a month in raw compute (and that's nearly all S3). My Co-founder is always worried about the cost of our AWS but so far we just haven't witnessed it. My mandate has always been the price should be able to scale to 0. So no EC2, has made building some longer running tasks a bit more complicated, but that's just because we didn't know how to do it before we started.
  - datalopers 4y ago
    Maintaining servers? Nah, you deploy a container and don't think about it. ECS is completely free to use. Anyway, you have a tiny on-demand business, as I caveated above. You could run it just fine on a nano ec2 instance if you needed, which is $2-3/mo.
  - moonchrome 4y ago
    The more I work with cloud the more I see this 'saves time on infrastructure' being a half-truth at best - we still have overwhelmed DevOps people who, unlike managing on-prem services, have 0 insight or control over what goes on inside AWS. And the services require a decent amount of hand holding, proprietary know how, etc. In the end you replaced sysadmin with DevOps and got up charged multiples.
    - zo1 4y ago
      Agreed, devops (and whatever the hell cloud wranglers are being sold as today) is the new priesthood. It's meta-level job creation and job security.
    - acdha 4y ago
      It's certainly possible to overspend on cloud services but in most cases when I see comparisons people tend to forget to fully include their true costs for things like staff time, infrastructure, etc. and especially things like opportunity cost for the delays caused by provisioning infrastructure, less capable interfaces (e.g. if you're swapping Terraform for a Jira ticket), or the technical decisions people make because they have fewer services available (this could be a Lambda function but we don't have that so now we need to manage a full VM). Some examples which come to mind: * Comparing S3 to the on-premise tape system but ignoring the fact that it involved an expensive tape robot, DR had access times measured in days, etc. * Comparing S3 to on-premise storage, ignoring the difference in redundancy, forcing users to handle bitrot at the application level, and the procurement process meaning that when they ran out of storage it took months of telling people they couldn't allocate more. * Saying their devops engineer cost twice as much as their sysadmins (true) but then when you look the devops engineer is using automation and managing literally a hundred times more systems than the “cheaper” ops team. * Saying their cost to run a VM was cheaper than EC2, which true if you looked only at the instance but not when you calculated how much they were spending on underutilized VM hosts, power / HVAC, facilities people, etc. It's totally possible to beat a major cloud provider on costs[1] but you usually need to be operating at fairly large scale to even approach the break-even point. This is especially true when you have regulatory or policy requirements for things like security and you include the cost of monitoring all of the things which fall under the cloud provider's responsibility — management networks, firmware version management, robust logging and IAM, etc. are all easy to accidentally exclude when making comparisons. 1. Network egress as the most obvious area to attack
      - moonchrome 4y ago
        But that's the part that I'm not getting - it's either AWS or building storage shelters and tape robots. There are hosting providers like Hetzner that will rent you bare metal machines so you don't have to deal with actual hardware, their networking is basically unmetered (last I checked), they will also sell you spot VMs, blob storage service, etc. The pricing difference is in integer multiples and much less "gotchas" or lock-in (they are usually dumbed down). They won't sell you fancy SAAS - but the ammount of fiddling required to get most of AWS services working in practice and debugging it - you're not very far off from using the OSS solution they are reselling - chances are your devops/sysadmin team can do that as well. And there's plenty cloud agnostic SAAS as well. I mean if you're a big corp that can accept the inefficiency for the sake of having one bill to pay at the end of the month, SLA you can use to cover your ass to higher ups and have the "nobody got fired for picking AWS" mentality I get it. But for startups and small companies ? All this "magical simplicity" that AWS/Azure/GCP is supposedly brining to the table - I'm not seeing it. Real redundancy (across AZ) is still really hard and requires extensive planning - no magic there. Dynamic scaling "savings" is often offset by being multiples more expensive to begin with and I have to deal with warmup issues, and the pricing is sneaky/unpredictable, easy to fuck up. Lambda architecture is pushing towards distributed microservices by default - the hardest thing to debug in practice. So end of the day, I'm forced to work around arbitrary pricing models rather then technical limitations, debugging opaque black box services, not being "big enough" to get actual support, overly generic/complex systems built as "one size fits all" solutions, or get sold in to practically abandoned "service" because it's available etc. And it's not like being first party service is any indication of quality - as soon as you move off of the most used stuff you get to see some really shit services in my experience.
- throwaway894345 4y ago
  You still have to pay your generalists for the time they spend writing and maintaining that bespoke software, and moreover you can hire competent developers to develop using cloud services. There’s no reason to assume that cloud services require incompetent developers, and indeed the cheapest solution is often to employ competent developers to integrate cloud services (whose development and operations costs are shared among all customers, whereas your company is footing the whole bill for your bespoke services).

etothepii 4y ago

Or any kind of demand spikes. If you are using lambda and suddenly need to run a million lambda you just can. If you want to have the ec2 support to run a million lambda at some time you are going to pay for a lot of sleeping computers.

pclmulqdq 4y ago
If your spikes go 5x over your base load, EC2 is still cheaper. Even if your load spikes are 10-20x over your base load, EC2 can still be cheaper. Past that, lambda is your better bet. If you are comparing to Hetzner or OVHCloud, you those numbers are 25x (definitely cheaper) and 50-100x (possibly cheaper).

acdha 4y ago

That's definitely not true for “all of those managed services” — you can't replicate CloudFront on EC2, for example, and it'd be extremely unlikely that you could build your own S3 replacement on top of EC2 storage without spending considerably more on operations than you save. On-premise storage is cheaper but you're still going to struggle to see a cost savings unless you either don't care about reliability or availability and/or buy your data by the petabyte. For the other services, it's still going to depend heavily on your usage, staff time, and operational efficiencies. For example, if you replace SQS with RabbitMQ you need to manage multiple EC2 servers for reliability but those servers might be significantly underutilized based on your traffic levels. Whether or not you save money depends on how much you pay your operators and how many messages you use: if you use less than the free tier's million messages per month, there's no way to beat it. Each million requests costs $0.40 or less, so if you pay your ops person $50k annually (haha) you'd need to be somewhat over 120 million messages per month to pay for them to spend a single hour on O&M even before factoring in your EC2 usage.

throwaway894345 4y ago

Maybe if you ignore the costs associated with operating EC2 or if you have a steady, predictable workload. Or perhaps if you have enormous (on the order of FAANG) scale to absorb those operational costs.

devonkim 4y ago
It gets odd in this distribution of potential customers because once your scale gets large enough again you're more than capable in theory of provisioning your own datacenters. MS, Google, and Amazon all have major public clouds while Apple and Facebook / Meta are deploying their own bare metal for their core infrastructure and some trace usage of the others most likely for compatibility reasons. Only Netflix is basically zero-DC in the new school infrastructure sense and even they used to have bare metal for old infrastructure like Oracle servers and Big Iron for I think accounting until so long ago. The real target customer IMO for these big public clouds are non-technical corporations whose primary revenue and personnel competencies aren't derived from mastery of software but primarily around other verticals like agriculture, manufacturing, etc. Insert Werner Vogels' criticism of the HBR article emphasizing focusing upon core competencies and outsourcing non-core competencies like IT (read: because technology is so important to every business at scale now).