6 ms·

The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset inte

The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers, or even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. For those that do, your scaling example works against you. If today you can merge three services into one, then why do you need full time infrastructure staff to manage so few servers? And remember, you want 24/7 monitoring, replication for disaster recovery, etc. Most businesses do not have IT infrastructure as a core skill or differentiator, and so they want to farm it out.

boltzmann-brain 8mo ago
As opposed to talent to manage the AWS? Sorry, AWS loses here as well.
- carefree-bob 8mo ago
  I know of AWS's reputation as a business and what the devs say who work there, so I have no argument against your point, except to say that they do manage to make it work. Somewhere in there must be some unsung heroes keeping the whole thing online.
  - rcxdude 8mo ago
    The point being that AWS runs AWS, they don't run your business on AWS. You still need someone to actually set up AWS to do what you want, much like you would need someone to run your on-premises servers. And in my experience, the difference is not much.
    - boltzmann-brain 8mo ago
      The biggest issue is that with colo you're building a skill pool that can be used forever, with AWS you're building a skill pool centered around a corporate entity's business strategies and an inscrutable, closed-source system, which is not sustainable.
throwup238 8mo ago
> even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. This is really the core problem. Every time I’ve done the math on a sizable cloud vs on-prem deployment, there is so much money left on the table that the orgs can afford to pay FAANG-level salaries for several good SREs but never have we been able to find people to fill the roles or even know if we had found them. The numbers are so much worse now with GPUs. The cost of reserved instances (let alone on-demand) for an 8x H100 pod even with NVIDIA Enterprise licenses included leaves tens of thousands per pod for the salary of employees managing it. Assuming one SREs can manage at least four racks the hardware pays for itself, if you can find even a single qualified person.
- deleted 8mo ago
  [deleted]
- tgrowazay 8mo ago
  Self-hosted 8xH100 is ~$250k, depreciated across three years => $80k/year, with power and cooling => $90k/year (~$10/hour total). AWS charges $55/hour for EC2 p5.48xlarge instance, which goes down with 1 or 3 year commitments. With 1 year commitment, it costs ~$30/hour => $262k per year. 3-year commitment brings price down to $24/hour => $210k per year. This price does NOT include egress, and other fees. So, yeah, there is a $120k-$175k difference that can pay for a full-time on-site SRE, even if you only need one 8xH100 server. Numbers get better if you need more than one server like that.
  - Aurornis 8mo ago
    $120K isn't going to cover the fully loaded costs of an SRE who can set up and run that. Hiring 1 person to run the infrastructure means that 1 person is on-call 24/7 forever. If there's an issue with the server while they're sick or on vacation, you just stop and wait. If they take a new job, you need to find someone to take over or very quickly hire a replacement. There's a second bus factor: What happens when that 8xH100 starts to get flakey? You can't move the jobs to another server because you only have one. You can start diagnosing things and replacing parts and hope it gets to the root issue, but that's more downtime. Going on-prem like this is highly risky. It works well until the hardware starts developing problems or the person in charge gets a new job. The weeks and months lost to dealing with the server start to become a problem. The SRE team starts to get tired of having to do all of their work on weekends because they can't block active use during the week. Teams start complaining that they need to use cloud to keep their project moving forward.
ozgrakkurt 8mo ago
This factually did not play out like this in my experience. The company did need the same exact people to manage AWS anyway. And the cost difference was so high that it was possible to hire 5 more people which wasn't needed anyway. Not only the cost but not needing to worry about going over the bandwidth limit and having soo much extra compute power made a very big difference. Imo the cloud stuff is just too full of itself if you are trying to solve a problem that requires compute like hosting databases or similar. Just renting a machine from a provider like Hetzner and starting from there is the best option by far.
- Drunkfoowl 8mo ago
  [dead]
- LunaSea 8mo ago
  > The company did need the same exact people to manage AWS anyway. That is incorrect. On AWS you need a couple DevOps that will Tring together the already existing services. With on premise, you need someone that will install racks, change disks, setup high availability block storage or object storage, etc. Those are not DevOps people.
  - mgaunard 8mo ago
    Ops people are typically more useful given you probably already have devs.
  - ozgrakkurt 8mo ago
    To be clear, I'm not writing about on-premise. I mean difference between managed cloud and renting dedicated servers
    - Dylan16807 8mo ago
      Even if you do include physical server setup and maintenance, one or two days per month is probably enough enough for a couple hundred rack units.
    - 8mo ago
nszceta 8mo ago
Managing AWS is a ton of work anyway
highfrequency 8mo ago
Given how good Apple Silicon is these days, why not just buy a spec'd out Mac Studio (or a few) for $15k (512 GB RAM, 8 TB NVMe), maybe pay for S3 only to sync data across machines. No talent required to manage the gear. AWS EC2 costs for similar hardware would net out in something ridiculous like 4 months.
justsomehnguy 8mo ago
> main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear Not quite. If you hire a bad talent to manage your 'cloud gear' then you would find what the mistakes which would cost you nothing on-premises would cost you in the cloud. Sometimes - a lot.
barrkel 8mo ago
What about the cost of k8s and AWS experts etc.?
citrin_ru 8mo ago
> price of acquiring talent to manage the gear Is it still a problem in 2026 when unemployment in IT is rising? Reasons can be argued (the end of ZIRP or AI) but hiring should be easier than it was at any time during the last 10 years.
- bdangubic 8mo ago
  hiring in 2026 is 100x harder than ever before
- Figs 8mo ago
  Hiring people is still fucked in 2026 in my experience. HR processes are extremely dysfunctional at many organizations...
- PunchyHamster 8mo ago
  people with that set of skills are never looking for job for long.
tempaccount5050 8mo ago
You need the exact same people to run the infra in the cloud. If they don't have IT at all, they aren't spinning up cloud VMs. You're mixing together SaaS and actual cloud infra.
- danielheath 8mo ago
  I'm one of those people, and I don't agree. Before I drop 5 figures on a single server, I'd like to have some confidence in the performance numbers I'm likely to see. I'd expect folk who are experienced with on-prem have a good intuition about this - after a decade of cloud-only work, I don't. Also, cloud networking offers a bunch of really nice primitives which I'm not clear how I'd replicate on-prem. I've estimated our IT workload would roughly double if we were to add physically racking machines, replacing failed disks, monitoring backups/SMART errors etc. That's... not cheap in staff time. Moving things on-prem starts making financial sense around the point your cloud bills hit the cost of one engineers salary.
  - esseph 8mo ago
    > Also, cloud networking offers a bunch of really nice primitives which I'm not clear how I'd replicate on-prem. Like what?
    - danielheath 8mo ago
      The biggest one for me is the way AWS security groups & IAM work. In AWS, it's straightforward to say e.g. "permit traffic on port X from instances holding IAM role Y". You can easily e.g. get the firewall rules for all your ec2 instances in a structured format. I really would not look forward to building something even 1/10th as functional as that.
      - tempaccount5050 8mo ago
        And you think just anyone can set that up? No sys admin/infra guy needed? Seems pretty risky.
PunchyHamster 8mo ago
> The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers, or even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. That's partially true; managing cloud also takes skill, most people forget that with end result being "well we saved on hiring sysadmins, but had to have more devops guys". Hell I manage mostly physical infrastructure (few racks, few hundred VMs) and good 80% of my work is completely unrelated to that, it's just the devops gluing stuff together and helping developers to set their stuff up, which isn't all that different than it would be in cloud. > And remember, you want 24/7 monitoring, replication for disaster recovery, etc. And remember, you need that for cloud too. Plenty of cloud disaster stories to see where they copy pasted some tutorial thinking that's enough then surprise. There is also partial way of just getting some dedicated servers from say OVH and run infra on that, you cut out a bit of the hardware management from skillset and you don't have the CAPEX to deal with. But yes, if it is less than at least a rack, it's probably not worth looking for onprem unless you have really specific use case that is much cheaper there (I mean less than usual half)
dgxyz 8mo ago
This is not the case. We had to double staff count going from three cages to AWS. And AWS was a lot more expensive. And now we're stuck. On top of that no one really knows what the fuck they are doing in AWS anyway.
hattmall 8mo ago
To me this doesn't sound logical because you still have to hire someone to manage your cloud deployments which is an entire specialized discipline. Yeah you can get some leeway the job being fully remote I guess but ultimately you aren't reducing headcount as linearly as you seem to imply by going cloud vs on-prem.
briffle 8mo ago
While I agree with you, some solutions, such as Oxide Computing could come pretty close to having all the ease of cloud, one whole rack of computers at a time.
tiew9Vii 8mo ago
> The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers This comes up again and again. It was the original sales pitch from cloud vendors. Often the very same companies repeating this messaging are recruiting and paying large teams of platform developers to manage their cloud…and pay for them to be on call.
stego-tech 8mo ago
Except they do have IT infra as a skill - or did - and replaced it with higher-paid “cloud architects” who just manage VM fleets and DBaaS in the vendor they’re certified in, which could just as easily be done on-site nowadays. They’re lower-skilled overall but more specialized, and thus command higher pay. Generalists (it me) typically command lower market rates, remain far more flexible, and can replicate much of the experience on-prem while knowing when and why to put something in the public cloud. A few examples: * 24/7 Telemetry and Monitoring: if you have the talent to roll your own with OpenTelemetry, Grafana, Prometheus, and a database to store the telemetry involved, then great! If it’s me wrangling a hybrid environment though, I’m likely leaning on New Relic to save on headcount and deliver similar results. * DBaaS: this is increasingly just offered by hypervisor managers since it’s often just spooling up a container and pointing to storage. Not quite a “solved problem”, but enough of one that a single DBA can cover both estates if needed - or a moderately skilled generalist can at least secure it for internal use before offering it out to customers * Vulnerability Management: as much as I’d love to have an internal Red Team, it’s an order of magnitude cheaper to leverage Nessus, Wazuh, Wiz, or any of the other fleets of continuous scanners to identify vulnerabilities or misconfigurations That’s just me cherry-picking. The point is less “do everything possible on-prem”, and more “diversify workload placement depending on cost advantages relative to risk models”, and on-prem wins quite handily for a lot of LOB software that just quietly sits and does its thing with minimal fuss.

boltzmann-brain 8mo ago

As opposed to talent to manage the AWS? Sorry, AWS loses here as well.

carefree-bob 8mo ago
I know of AWS's reputation as a business and what the devs say who work there, so I have no argument against your point, except to say that they do manage to make it work. Somewhere in there must be some unsung heroes keeping the whole thing online.
- rcxdude 8mo ago
  The point being that AWS runs AWS, they don't run your business on AWS. You still need someone to actually set up AWS to do what you want, much like you would need someone to run your on-premises servers. And in my experience, the difference is not much.
  - boltzmann-brain 8mo ago
    The biggest issue is that with colo you're building a skill pool that can be used forever, with AWS you're building a skill pool centered around a corporate entity's business strategies and an inscrutable, closed-source system, which is not sustainable.

throwup238 8mo ago

> even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. This is really the core problem. Every time I’ve done the math on a sizable cloud vs on-prem deployment, there is so much money left on the table that the orgs can afford to pay FAANG-level salaries for several good SREs but never have we been able to find people to fill the roles or even know if we had found them. The numbers are so much worse now with GPUs. The cost of reserved instances (let alone on-demand) for an 8x H100 pod even with NVIDIA Enterprise licenses included leaves tens of thousands per pod for the salary of employees managing it. Assuming one SREs can manage at least four racks the hardware pays for itself, if you can find even a single qualified person.

deleted 8mo ago
[deleted]
tgrowazay 8mo ago
Self-hosted 8xH100 is ~$250k, depreciated across three years => $80k/year, with power and cooling => $90k/year (~$10/hour total). AWS charges $55/hour for EC2 p5.48xlarge instance, which goes down with 1 or 3 year commitments. With 1 year commitment, it costs ~$30/hour => $262k per year. 3-year commitment brings price down to $24/hour => $210k per year. This price does NOT include egress, and other fees. So, yeah, there is a $120k-$175k difference that can pay for a full-time on-site SRE, even if you only need one 8xH100 server. Numbers get better if you need more than one server like that.
- Aurornis 8mo ago
  $120K isn't going to cover the fully loaded costs of an SRE who can set up and run that. Hiring 1 person to run the infrastructure means that 1 person is on-call 24/7 forever. If there's an issue with the server while they're sick or on vacation, you just stop and wait. If they take a new job, you need to find someone to take over or very quickly hire a replacement. There's a second bus factor: What happens when that 8xH100 starts to get flakey? You can't move the jobs to another server because you only have one. You can start diagnosing things and replacing parts and hope it gets to the root issue, but that's more downtime. Going on-prem like this is highly risky. It works well until the hardware starts developing problems or the person in charge gets a new job. The weeks and months lost to dealing with the server start to become a problem. The SRE team starts to get tired of having to do all of their work on weekends because they can't block active use during the week. Teams start complaining that they need to use cloud to keep their project moving forward.

ozgrakkurt 8mo ago

This factually did not play out like this in my experience. The company did need the same exact people to manage AWS anyway. And the cost difference was so high that it was possible to hire 5 more people which wasn't needed anyway. Not only the cost but not needing to worry about going over the bandwidth limit and having soo much extra compute power made a very big difference. Imo the cloud stuff is just too full of itself if you are trying to solve a problem that requires compute like hosting databases or similar. Just renting a machine from a provider like Hetzner and starting from there is the best option by far.

Drunkfoowl 8mo ago
[dead]
LunaSea 8mo ago
> The company did need the same exact people to manage AWS anyway. That is incorrect. On AWS you need a couple DevOps that will Tring together the already existing services. With on premise, you need someone that will install racks, change disks, setup high availability block storage or object storage, etc. Those are not DevOps people.
- mgaunard 8mo ago
  Ops people are typically more useful given you probably already have devs.
- ozgrakkurt 8mo ago
  To be clear, I'm not writing about on-premise. I mean difference between managed cloud and renting dedicated servers
  - Dylan16807 8mo ago
    Even if you do include physical server setup and maintenance, one or two days per month is probably enough enough for a couple hundred rack units.
  - 8mo ago

nszceta 8mo ago

Managing AWS is a ton of work anyway

highfrequency 8mo ago

Given how good Apple Silicon is these days, why not just buy a spec'd out Mac Studio (or a few) for $15k (512 GB RAM, 8 TB NVMe), maybe pay for S3 only to sync data across machines. No talent required to manage the gear. AWS EC2 costs for similar hardware would net out in something ridiculous like 4 months.

justsomehnguy 8mo ago

> main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear Not quite. If you hire a bad talent to manage your 'cloud gear' then you would find what the mistakes which would cost you nothing on-premises would cost you in the cloud. Sometimes - a lot.

barrkel 8mo ago

What about the cost of k8s and AWS experts etc.?

citrin_ru 8mo ago

> price of acquiring talent to manage the gear Is it still a problem in 2026 when unemployment in IT is rising? Reasons can be argued (the end of ZIRP or AI) but hiring should be easier than it was at any time during the last 10 years.

bdangubic 8mo ago
hiring in 2026 is 100x harder than ever before
Figs 8mo ago
Hiring people is still fucked in 2026 in my experience. HR processes are extremely dysfunctional at many organizations...
PunchyHamster 8mo ago
people with that set of skills are never looking for job for long.

tempaccount5050 8mo ago

You need the exact same people to run the infra in the cloud. If they don't have IT at all, they aren't spinning up cloud VMs. You're mixing together SaaS and actual cloud infra.

danielheath 8mo ago
I'm one of those people, and I don't agree. Before I drop 5 figures on a single server, I'd like to have some confidence in the performance numbers I'm likely to see. I'd expect folk who are experienced with on-prem have a good intuition about this - after a decade of cloud-only work, I don't. Also, cloud networking offers a bunch of really nice primitives which I'm not clear how I'd replicate on-prem. I've estimated our IT workload would roughly double if we were to add physically racking machines, replacing failed disks, monitoring backups/SMART errors etc. That's... not cheap in staff time. Moving things on-prem starts making financial sense around the point your cloud bills hit the cost of one engineers salary.
- esseph 8mo ago
  > Also, cloud networking offers a bunch of really nice primitives which I'm not clear how I'd replicate on-prem. Like what?
  - danielheath 8mo ago
    The biggest one for me is the way AWS security groups & IAM work. In AWS, it's straightforward to say e.g. "permit traffic on port X from instances holding IAM role Y". You can easily e.g. get the firewall rules for all your ec2 instances in a structured format. I really would not look forward to building something even 1/10th as functional as that.
    - tempaccount5050 8mo ago
      And you think just anyone can set that up? No sys admin/infra guy needed? Seems pretty risky.

PunchyHamster 8mo ago

> The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers, or even the internal talent to know whether they are hiring a good infrastructure engineer or not during the interview process. That's partially true; managing cloud also takes skill, most people forget that with end result being "well we saved on hiring sysadmins, but had to have more devops guys". Hell I manage mostly physical infrastructure (few racks, few hundred VMs) and good 80% of my work is completely unrelated to that, it's just the devops gluing stuff together and helping developers to set their stuff up, which isn't all that different than it would be in cloud. > And remember, you want 24/7 monitoring, replication for disaster recovery, etc. And remember, you need that for cloud too. Plenty of cloud disaster stories to see where they copy pasted some tutorial thinking that's enough then surprise. There is also partial way of just getting some dedicated servers from say OVH and run infra on that, you cut out a bit of the hardware management from skillset and you don't have the CAPEX to deal with. But yes, if it is less than at least a rack, it's probably not worth looking for onprem unless you have really specific use case that is much cheaper there (I mean less than usual half)

dgxyz 8mo ago

This is not the case. We had to double staff count going from three cages to AWS. And AWS was a lot more expensive. And now we're stuck. On top of that no one really knows what the fuck they are doing in AWS anyway.

hattmall 8mo ago

To me this doesn't sound logical because you still have to hire someone to manage your cloud deployments which is an entire specialized discipline. Yeah you can get some leeway the job being fully remote I guess but ultimately you aren't reducing headcount as linearly as you seem to imply by going cloud vs on-prem.

briffle 8mo ago

While I agree with you, some solutions, such as Oxide Computing could come pretty close to having all the ease of cloud, one whole rack of computers at a time.

tiew9Vii 8mo ago

> The main cost with on-prem is not the price of the gear but the price of acquiring talent to manage the gear. Most companies simply don't have the skillset internally to properly manage these servers This comes up again and again. It was the original sales pitch from cloud vendors. Often the very same companies repeating this messaging are recruiting and paying large teams of platform developers to manage their cloud…and pay for them to be on call.

stego-tech 8mo ago

Except they do have IT infra as a skill - or did - and replaced it with higher-paid “cloud architects” who just manage VM fleets and DBaaS in the vendor they’re certified in, which could just as easily be done on-site nowadays. They’re lower-skilled overall but more specialized, and thus command higher pay. Generalists (it me) typically command lower market rates, remain far more flexible, and can replicate much of the experience on-prem while knowing when and why to put something in the public cloud. A few examples: * 24/7 Telemetry and Monitoring: if you have the talent to roll your own with OpenTelemetry, Grafana, Prometheus, and a database to store the telemetry involved, then great! If it’s me wrangling a hybrid environment though, I’m likely leaning on New Relic to save on headcount and deliver similar results. * DBaaS: this is increasingly just offered by hypervisor managers since it’s often just spooling up a container and pointing to storage. Not quite a “solved problem”, but enough of one that a single DBA can cover both estates if needed - or a moderately skilled generalist can at least secure it for internal use before offering it out to customers * Vulnerability Management: as much as I’d love to have an internal Red Team, it’s an order of magnitude cheaper to leverage Nessus, Wazuh, Wiz, or any of the other fleets of continuous scanners to identify vulnerabilities or misconfigurations That’s just me cherry-picking. The point is less “do everything possible on-prem”, and more “diversify workload placement depending on cost advantages relative to risk models”, and on-prem wins quite handily for a lot of LOB software that just quietly sits and does its thing with minimal fuss.