4 ms·

It's insane spends like that which are the main reason that reducing cloud costs is the main money maker for my consulting business at the moment. It's extreme

by vidarh 1y ago

It's insane spends like that which are the main reason that reducing cloud costs is the main money maker for my consulting business at the moment. It's extremely rare to come across people using cloud services where you can't cut their costs by at least 30%, often more than half, sometimes as much as 90%+, and usually it results in reduced devops spends as well. The interesting thing is that the hardest part of the sales process is to convince people that cloud isn't cheap, because there's a near religious belief that cloud providers must be cost effective in some companies.

handfuloflight 1y ago
What percentage of clients do you think don't even need cloud?
- vidarh 1y ago
  For the vast majority of them, rented servers at a managed hosting provider would beat cloud in price hands down, but for some the difference might be small enough once they've actually optimised their cloud setup that it's a judgement call. I usually recommend clients plan for optimizing their cloud setup first, both to get a fair comparison and also because a lot of the assessments you need to do of your cloud environment to do that will also make it easier to figure out what you'd pay for a managed hosting setup as well (e.g. understanding base load vs. spikes, durability requirements for different subsets of storage etc.) For some of them, a colo facility would be cheaper, but that's highly dependent on where you want to host it (e.g. I'm in London - putting things in a colo in London is really hard to make cost effective vs. renting servers somewhere with lower land costs; data centre operators are real-estate plays) However, you can usually make managed hosting/colo even cheaper by sprinkling some cloud in. E.g. a "trick" that can work amazingly well is to set up the bare minimum to let you spin up what you need to handle traffic spikes in a cloud environment, and then set up monitoring for your load balancer so that you start scaling into the cloud environment once load hits a certain level, but use only the managed hosting below that level. That way, you almost never end up actually spinning up cloud instances, but you gain the ability to run the managed hosting environment far closer to the wire than you could otherwise safely do, and drive your cost per request down accordingly.
gausswho 1y ago
We're discussing Figma's spend here which is a bit of a cherry pick. At their size the savings for on-prem can be substantial. For the vast majority of companies without such heavy bandwidth needs, the additional cost needs to be measured in terms of additional intenance and staffing.
- vidarh 1y ago
  Savings can be substantial at much smaller sizes and generally devops maintenance/staffing costs goes down if anything. This is literally my bread and butter, and we make more money off customers who insist on staying on cloud providers, because they consistently need more help.
  - gausswho 1y ago
    I'm curious what size and use case you're serving describing here. Up to a certain point, devops effort and cost is negligible when offloading to the cloud.
    - vidarh 1y ago
      "Negligible" varies greatly from organisation to organisation. People tend to come to my consultancy with needs that'll cost them anything from a few hours a month and up of access to a devops resource. That tends to involve far smaller setups for cloud users than for managed hosting. How large the setup is before that point varies a lot by organisational maturity and what exactly they do, but that can typically reflect hosting costs ranging from a couple of thousand a month and up. But the biggest cost differential is the hosting itself. It's not unusual for a managed setup to cost half or less what the equivalent cloud environment does, for similar levels of resiliency, including the fully loaded cost of devops resources.
icedchai 1y ago
Is this production only? Based on my personal experience, many orgs could save a decent chunk just be cleaning up or spinning down dev environments.
- vidarh 1y ago
  We'd typically look at everything, and you're absolutely right. Dev environments are often also poorly managed and you can find lots of resources nobody knows who are using.
  - noahjk 1y ago
    How do you recommend dev envs be treated when the company wants it to mimic prod but you still want to save money compared to prod?
    - vidarh 1y ago
      Usually this isn't a big issue if people are on managed hosting, because costs tends to be so much lower, and you can often overprovision more when you know it's a dev environment, while turning down number of instances etc. The most important thing tends to be to make it possible to run services in isolation so devs can test individual services on their own machines. Then you'll end up with much more time-limited need to spin up larger environments for integration tests etc. The most important part, though, tends to be to ensure you have a system to keep track of what is running and who is currently using it.
- samgranieri 1y ago
  This 100%. SRE at $dayjob periodically has to ask us to clean up dev and uat kubernetes deployments
mystified5016 1y ago
At the startup I work at, we have an AWS instance that only runs our GitLab server. The damn thing runs at 10-15% CPU nearly constantly (because GitLab's founding assumption is that everyone wants to scale to ten million global users) so our spend racks up to $70/month every months. And yes, that's absolute peanuts for any business. But in my view, spending almost $1k annually for a GitLab server for a team of ten is ridiculous. We could accomplish exactly the same thing on-prem with hardware we already own. It would take a couple of engineer-hours per year. As long as it's under 40 hours of maintenance per year, we come out day ahead. And over the last two years, I've had to spend a total of maybe 10 hours with hands on this box. I just don't get the thinking that leads businesses to put unnecessary crap in the cloud. Just save your money, on-prem is cheaper for pretty much everyone not doing a saas or less than a few hundred employees
- mdaniel 1y ago
  My mental model of startups is that they're free of the bureaucracy that would cause you to whine in comments versus Just Fucking Do It. So, what's the impediment to using your time to test your "10 hours" theory? Also, my experience with GitLab isn't that the thing is hurp-durping because of scaling to ten million, it's that ruby gonna ruby
- vidarh 1y ago
  I often starts that way, yes. And then you end up accreting more and more. I usually tell people that if they don't want something on-prem, then at least pick a cheaper provider. But as long as it's a VM and you don't build in reliance on other AWS services, at least you can move off it as you scale. The real problem comes when you start depending on a single cloud. Not only do you lose the ability to do an easy move, but you also lose almost all negotiating leverage as your bill increases.
- quickthrowman 1y ago
  Your estimate is off by an order of magnitude unless you value developer time at $21 an hour. I pay union electricians 6x more than that to work for me. > $70 a month times 12 is $840 > As long as it’s under 40 hours of maintenance (??)
PunchyHamster 1y ago
I've seen so many people being baited by whole "you will not only not need to buy expensive servers, but also not have expensive ops guys to manage it!" schtick. Meanwhile your devs put more man-hours than your actual hardware ops teams would ever do into wrangling cloud quirks and spend way more than getting a bunch of servers (even if you include costs of financing them in that) would ever cost.

by vidarh 1y ago

handfuloflight 1y ago

What percentage of clients do you think don't even need cloud?

vidarh 1y ago
For the vast majority of them, rented servers at a managed hosting provider would beat cloud in price hands down, but for some the difference might be small enough once they've actually optimised their cloud setup that it's a judgement call. I usually recommend clients plan for optimizing their cloud setup first, both to get a fair comparison and also because a lot of the assessments you need to do of your cloud environment to do that will also make it easier to figure out what you'd pay for a managed hosting setup as well (e.g. understanding base load vs. spikes, durability requirements for different subsets of storage etc.) For some of them, a colo facility would be cheaper, but that's highly dependent on where you want to host it (e.g. I'm in London - putting things in a colo in London is really hard to make cost effective vs. renting servers somewhere with lower land costs; data centre operators are real-estate plays) However, you can usually make managed hosting/colo even cheaper by sprinkling some cloud in. E.g. a "trick" that can work amazingly well is to set up the bare minimum to let you spin up what you need to handle traffic spikes in a cloud environment, and then set up monitoring for your load balancer so that you start scaling into the cloud environment once load hits a certain level, but use only the managed hosting below that level. That way, you almost never end up actually spinning up cloud instances, but you gain the ability to run the managed hosting environment far closer to the wire than you could otherwise safely do, and drive your cost per request down accordingly.

gausswho 1y ago

We're discussing Figma's spend here which is a bit of a cherry pick. At their size the savings for on-prem can be substantial. For the vast majority of companies without such heavy bandwidth needs, the additional cost needs to be measured in terms of additional intenance and staffing.

vidarh 1y ago
Savings can be substantial at much smaller sizes and generally devops maintenance/staffing costs goes down if anything. This is literally my bread and butter, and we make more money off customers who insist on staying on cloud providers, because they consistently need more help.
- gausswho 1y ago
  I'm curious what size and use case you're serving describing here. Up to a certain point, devops effort and cost is negligible when offloading to the cloud.
  - vidarh 1y ago
    "Negligible" varies greatly from organisation to organisation. People tend to come to my consultancy with needs that'll cost them anything from a few hours a month and up of access to a devops resource. That tends to involve far smaller setups for cloud users than for managed hosting. How large the setup is before that point varies a lot by organisational maturity and what exactly they do, but that can typically reflect hosting costs ranging from a couple of thousand a month and up. But the biggest cost differential is the hosting itself. It's not unusual for a managed setup to cost half or less what the equivalent cloud environment does, for similar levels of resiliency, including the fully loaded cost of devops resources.

icedchai 1y ago

Is this production only? Based on my personal experience, many orgs could save a decent chunk just be cleaning up or spinning down dev environments.

vidarh 1y ago
We'd typically look at everything, and you're absolutely right. Dev environments are often also poorly managed and you can find lots of resources nobody knows who are using.
- noahjk 1y ago
  How do you recommend dev envs be treated when the company wants it to mimic prod but you still want to save money compared to prod?
  - vidarh 1y ago
    Usually this isn't a big issue if people are on managed hosting, because costs tends to be so much lower, and you can often overprovision more when you know it's a dev environment, while turning down number of instances etc. The most important thing tends to be to make it possible to run services in isolation so devs can test individual services on their own machines. Then you'll end up with much more time-limited need to spin up larger environments for integration tests etc. The most important part, though, tends to be to ensure you have a system to keep track of what is running and who is currently using it.
samgranieri 1y ago
This 100%. SRE at $dayjob periodically has to ask us to clean up dev and uat kubernetes deployments

mystified5016 1y ago

At the startup I work at, we have an AWS instance that only runs our GitLab server. The damn thing runs at 10-15% CPU nearly constantly (because GitLab's founding assumption is that everyone wants to scale to ten million global users) so our spend racks up to $70/month every months. And yes, that's absolute peanuts for any business. But in my view, spending almost $1k annually for a GitLab server for a team of ten is ridiculous. We could accomplish exactly the same thing on-prem with hardware we already own. It would take a couple of engineer-hours per year. As long as it's under 40 hours of maintenance per year, we come out day ahead. And over the last two years, I've had to spend a total of maybe 10 hours with hands on this box. I just don't get the thinking that leads businesses to put unnecessary crap in the cloud. Just save your money, on-prem is cheaper for pretty much everyone not doing a saas or less than a few hundred employees

mdaniel 1y ago
My mental model of startups is that they're free of the bureaucracy that would cause you to whine in comments versus Just Fucking Do It. So, what's the impediment to using your time to test your "10 hours" theory? Also, my experience with GitLab isn't that the thing is hurp-durping because of scaling to ten million, it's that ruby gonna ruby
vidarh 1y ago
I often starts that way, yes. And then you end up accreting more and more. I usually tell people that if they don't want something on-prem, then at least pick a cheaper provider. But as long as it's a VM and you don't build in reliance on other AWS services, at least you can move off it as you scale. The real problem comes when you start depending on a single cloud. Not only do you lose the ability to do an easy move, but you also lose almost all negotiating leverage as your bill increases.
quickthrowman 1y ago
Your estimate is off by an order of magnitude unless you value developer time at $21 an hour. I pay union electricians 6x more than that to work for me. > $70 a month times 12 is $840 > As long as it’s under 40 hours of maintenance (??)

PunchyHamster 1y ago

I've seen so many people being baited by whole "you will not only not need to buy expensive servers, but also not have expensive ops guys to manage it!" schtick. Meanwhile your devs put more man-hours than your actual hardware ops teams would ever do into wrangling cloud quirks and spend way more than getting a bunch of servers (even if you include costs of financing them in that) would ever cost.