7 ms·

I recently heard from a senior IT person that the company for which he works could save many millions by moving a certain application from a hairball of micro-s

• mirochnik 4y ago
  This is a new type of conflict of interests. I assumed only good intentions from IT guys. But you're right that some of them may be more interested in complexity of used solutions, bigger budgets, and their personal job security than in company results and sustainability.
  • fidgewidge 4y ago
    It's not like they want complexity, it's the latter: the "nobody ever got fired for buying AWS" problem. They stated it explicitly. They're terrified of anything counter-cultural or that seems to be going against the tide because they want to be seen as in the future and not stuck in the past. Fashion, FOMO and weakness combine to yield a "cloud at any cost" mentality.
    • mirochnik 4y ago
      Wow! It sounds even worse.
• bombcar 4y ago
  A significant number of "cloud migrations" involve moving an application running on Linux on a server to running on Linux on a VPS for similar or higher cost. Not even using ANY of the "features" of the cloud, just treating it like a server. But then they can say "cloud" and everyone's happy.
  • hotpotamus 4y ago
    Ah yes, the "actual" way to use cloud is to undergo a major re-write of your application such that it is inextricably tied into the proprietary PaaS options of your cloud provider. Cloud is amazing for highly variable loads. I also see it as a bit of a luxury service for ops types (like me) - I don't have to go to a DC and manage hardware or deal with a DCops crew, so that's nice, but you probably wouldn't buy luxury cars if you were starting a courier business.
    • bombcar 4y ago
      What really annoys me is that few if any clouds actually allow "dynamic scaling" of a single instance running operating system where you can hot-add more RAM or more CPU, with or without a restart. Some can do it, some require you to basically image the entire machine onto a new one, some can't handle it at all.
      • amluto 4y ago
        I’ve never worked at a cloud company, but I know more than I’d really like about some of the stuff under the hood. First, changing out the CPU type to something that isn’t almost identical out from under a running VM is a mess and may require degrading the system by removing features from the starting CPU. Switching manufacturers at runtime (AMD vs Intel), while sort of possible in theory, is effectively a lost cause. To add CPUs, you have to deal with the architecture’s nasty hot-add CPU mechanism, and the guest OS needs to be expecting those CPUs starting from when it boots, and that expectation isn’t free. (The latter is Linux’s num_possible_cpus vs num_present_cpus.) Adding memory involves getting that memory into the kernel’s memory map in the right places. This is more complex than one would like. Removing memory is worse. And all this happens, in the cloud data center, on essentially normal hardware. If you want to add RAM, the CPUs need to be on a system with more RAM, preferably on the same NUMA node. And vice versa for adding CPUs. If the tenant is paying for local storage, that needs to move, too.
        • bombcar 4y ago
• gadders 4y ago
  I think a lot of companies are going to find that they have the same relationship with AWS as banks do with Bloomberg i.e. a massive spend that they constantly try to manage downwards but can never escape.
  • steve1977 4y ago
    I‘m in this picture twice and I don’t like it.
    • gadders 4y ago
      Same.
• bfeynman 4y ago
  From a biz perspective, that can be a bad move and signals that you're not in growth phase but in cutting costs to preserve profits. Using cloud derisks you for future costs because it's managed for you, and then you allocate capex towards making better features. Doing it locally means having a team of IT admins/professionals you normally only need a few of which also incurs a lot of cost and risk.
  • hardware2win 4y ago
    Lets replace 4 cheap admins with 3 expensive devops engs Yaay savings!
  • amluto 4y ago
    Siloing expenses into capex and opex gets a bit silly when the cost effectiveness of that opex becomes low enough. For some cloud services, one can replace it with on-prem or colo’d equipment and recover the capex in months. Similarly, “easy” scalability doesn’t necessarily make up for COGS when scaling up. If I can flip the autoscale switch while I grow to $1bn ARR and pay $750M/yr to the cloud, I’m not doing nearly as well as if I hire ten FTEs and pay $150M in combined capex and opex over two years. (And if my COGS is 75% of the list price, it’s hard to give discounts, pay for sales, etc)
• freitzkriesler2 4y ago
  Tale as old as time. Heard the same with exchange, MSP contracts, etc etc some suit somewhere always has some CYA reason no matter how silly or wrong it may me.
• warent 4y ago
  When consulting for a seed funded startup, I suggested they buy their own servers and colocate them. It could save them almost 100k a year. Everyone looked at me like an alien speaking a different language, lol. Then was politely dismissed, even though I'm experienced in running hardware. AWS, GCP, Azure, really managed to expertly pull off the greatest heist of all time. They're useful for sure, but somehow they've gotten everyone to drink this potent marketing kool-aid that if you're not using them then you're not serious about success. So cringe.
  • senttoschool 4y ago
    They'd have to keep you employed to manage the hardware. What happens if you get run over by a bus (which I never hope will happen)? What happens if you decide to move to another city? There are a lot more "what ifs" by employing you to build and manage the hardware. Saving $100k would not be worth the hassle/risk for most well funded startups.
    • warent 4y ago
      Again this is the kool-aid speaking. The co-location manages most of the complexity and the probability of failure is basically zero for the first 4 years. Hardware these days is really good. People have this illusion that servers are extremely hard to maintain because big companies constantly have to maintain their 1000s, and because clouds are incentivized to sell this lie. It would still be a huge cost saving and trivial for them to contract someone to maintain the servers for like 1 hour a month with emergency on-call. Of which there are numerous such people who are not me.
      • senttoschool 4y ago
        I used to build PCs. You can get the same hardware for nearly half the cost of pre-built PCs like HP or Dell. I did a lot of research on the best bang for the buck parts and the most reliable parts. If something broke, just pull it out and replace it. Easy. I worked in an IT department as an intern. I thought this company should build custom PCs for all their workers. You can get a much faster CPU or much faster GPU for the same price! It's a no brainer! Then I realized that if I wanted to build custom PCs for all the workers, I'd have to provide support for all of them. I'd have to be as good as CDW, who was supplying the company with PCs. This means next-day repairs and replacements. Drivers. Security. Firmware updates. For hundreds of PCs. Needless to say, a lot of people under appreciate all the things that vendors like AWS or CDW provide. They think they can do it because they know how it's done. But they're ignoring a lot of other factors that can come back to bite you. I'm sure you're very competent at managing server hardware. But if I'm a seed stage startup, I'd politely decline your offer.
• ejb999 4y ago
  But that doesn't really prove much does it? You are saying they could rearchitect/simplify the whole thing and move it on site to save money - but what about rearchitecting the whole thing and leaving it on AWS? Which part would actually saving them the money? - rearchitecting/simplifying, or changing where it runs? There would need to be an apples-to-apples comparison to be meaningful. That said, I don't believe everything needs to be in the cloud - but maintaining data-centers are also very expensive, and need to be accounted for very differently.
• sshine 4y ago
  I helped migrate a PHP app to AWS and K8s for a startup that got acquired a couple of years ago. Before it was running on a dedicated Xeon with the database available on a UNIX socket. After... where do I start. The app was very database heavy and experienced timeouts due to too many database calls (IOPS). Not a problem prior to migration, because you don’t pay separately for UNIX sockets. Just the IOPS ended up costing several hundred bucks a month. All environment variables were stored as secrets at $.40/mo. per variable. I know this is peanuts, but it is so illustrative of the pricing model. It feels like those candy bags where each tiny piece of candy is individually wrapped in plastic. I found $35k/mo I’d have spent differently.
  • oxfordmale 4y ago
    If you store secrets in the Parameter Store, there is no cost per month. In AWS you need to be careful what you use.
• oxfordmale 4y ago
  On-Premise Indirect (Hidden) Costs There are a variety of other expenses associated with an on-premise environment considered indirect expenses. These expenses are often referred to as “hidden” expenses due to how often they are overlooked rather than “hidden.” These include: The real estate of the storage space used for the servers Tools used for temperature control in the data center The cost of set up, configuration, and ongoing upgrades Staff salaries for administrators that maintain an on-premise data center Networking infrastructure set up and ongoing maintenance The cost of downtime while the team troubleshoots the issue Productivity lost when the system experiences downtime The cost of keeping the servers powered 24/7 Depreciation of the hardware and software Time spent on disaster recovery Administrative costs associated beyond IT staff such as HR, purchasing, financing, and other departments. https://pmsquare.com/analytics-blog/2022/6/9/calculating-on-prem-vs-cloud-costs https://pmsquare.com/analytics-blog/2022/6/9/calculating-on-...
  • fidgewidge 4y ago
    You speak like clouds never have downtime, or don't charge you for space and power. Look at the article - that one company would needed to have paid more than their entire revenue to AWS for worse hardware. They'd have spent $400M in 3 years on the cloud! There are no "hidden costs" that can even begin to approach a fraction of that. As for staff salaries, lol. It's not like AWS is self administering.
    • oxfordmale 4y ago
      In the cloud, costs are transparent, as you receive a single monthly bill. When you self-host, there are many hidden expenses you would need to track down (energy, admin, maintenance, etc.) The figures in the article are for illustration purposes only and should be taken with a large pinch of salt. The author doesn't detail what hardware they are running, what EC2 instances he has selected for comparison and how comparable the storage statistics are. I would also love to hear the Finance Department's version of his calculations. AWS is more expensive than self-hosting. However, it is not as skewed as the author claims. Otherwise, very few companies would be using the cloud.
• j45 4y ago
  Retail cloud setups provide standardization that hosting on bare metal may not. The irony of it is most of the cloud services are open-source technologies packaged up to be easy to use and administer from a web interface. Maybe if someone came up with a set of tooling in AWS to use many lightsail VMs to provide more of the basic AWS services, it could be a way to get the best of both worlds.
• Hnrobert42 4y ago
  What if they went from a hairball in AWS to a much simpler architecture in AWS. Wouldn’t that be a better comparison?