12 ms·

I'm always amazed at the complexity of AWS billing. "Why does this cost so much" turns out to be an incredibly hard problem to answer the moment you don't have

I'm always amazed at the complexity of AWS billing. "Why does this cost so much" turns out to be an incredibly hard problem to answer the moment you don't have 100% perfect discipline when it comes to resource tagging. The fact that there are consultants who specialize in figuring out AWS billing was, in retrospect, a warning sign.

doorstar 6y ago
I've been horrified at how bad this is on Azure, I'm sorry to hear that AWS has the same issue. I've signed in some mornings only to discover that I've been charged $20 for a MongoDB instance that's been sitting idle all night. Support can only come back with some hand-waving about RUs, a term that defies any clear definition.
- zaidf 6y ago
  This is a problem across enterprise SaaS companies. At my last company (small enterprise SaaS biz <$50M ARR), I got so tired of this consuming so much of my time. I literally hacked together some code so I could point at any line item from any bill generated by our "system" and it would try to answer "why did I get billed $x for this?" so a five year old could understand. I wish there were SaaS products that would effortlessly audit enterprise SaaS bills with support for more than just the top 1% SaaS products.
  - doorstar 6y ago
    I was poking around with their CLI to see if I could put together a cron job - this artifact now costs more than $X so delete it. Unfortunately if there's a 'get current charges' command I haven't found it. I have a ticket open with MS support. I suppose it's to their credit that they're listening to me but it's been a few weeks with only the aforementioned hand-waving. I started just sending them screenshots showing that their own dashboard showed no usage during the times I'd gotten charged.
- FridgeSeal 6y ago
  Dealing with Azure causes me physical pain. AWS you can poke around and figure it out, Azure is a Kafkaesque nightmare of infinitely confusing UI/UX, a permission system that is like trying to hit a moving target in a dark room, being moved by someone that hates you, doing anything seems needlessly complex, and trying to replicate/test anything locally is horrible (looking at you Azure functions).
  - doorstar 6y ago
    My sibling... I'm trapped in this mess because the nature of our Client is that they'd consider anything besides Azure as a slap in the face, but it's been absolutely the most miserable task I've taken on in years. As a bonus I'm getting side-eyed by teammates who have a naive faith in The Cloud and can't comprehend why this is taking so long and why I can't just give them a cost estimate.
    - qchris 6y ago
      As an aside, unless I'm reading this very wrong, this is the first time I've seen the use of "my sibling" as a gender-neutral replacement of "my brother/sister" to express empathy. I'm delighted by it.
  - lucianf 6y ago
    Semi-shameless plug (I work for Oracle) but have you tried OCI? You might find both the console UI/UX and the cloud calculator pretty decent. That's just my personal feel, coming over to work as an Oracle cloud consultant with previous AWS/Azure experience. I'd argue it's normal those two are where they are; AWS is now a spaghetti of products, while Microsoft are anti-KISS by DNA.
    - scarface74 6y ago
      Yes people who want to avoid lock in really want to get in bed with “call us for pricing” Oracle.
      - 6y ago
ken 6y ago
I'm as anti-AMZN/AWS as anyone these days, but to their credit, Amazon has their own teams of consultants who will come to your business for free to analyze your usage of AWS and tell you how to save money. They openly joked about how it was their job to get customers to spend less money with their company.
- skrebbel 6y ago
  That's not openly joking, it's sales. If you feel like you're way overpaying for AWS, you're much likelier to look out for alternatives. So if they help their biggest accounts save some money, it'll net them way more in the long run.
  - bt1a 6y ago
    Right - it gives the illusion that you're 'getting an amazing deal and wow these people are great' when it reality you're probably getting gouged without the consulting and have to opt in for fair prices.
    - skrebbel 6y ago
      Yep. To be frank I think that spinning it as a joke and not an honest retention tactic almost feels like lying.
    - Spivak 6y ago
      I mean it's a retention tactic at the company scale but for the individual consultant they're genuinely trying to help you out. I frequent a local family-owned Italian restaurant/pizza place and my usual order jalapeños, banana peppers, and roasted garlic so I do a build you own. Totally fine, happy to pay the price. But one day I come in and the woman working the take-out counter sees my order before it goes in and tells me that if I order a "meat lovers", remove two toppings and sub the rest it comes out way cheaper. So now I'm the vegetarian that always gets a meat lovers and she makes the joke every time I come in. I don't think it's weird or nefarious that a business doesn't have completely consistent pricing and holding it differently can save you money. When we spec out services on AWS we're using the official calculator when deciding whether it's "worth it" so to us at least it genuinely feels like we're saving money.
moduspol 6y ago
I usually try to cut them some slack. I mean, they're operating this stuff at Internet scale and have never (AFAIK) raised prices. They've basically got two options: * Price things in an intricate way sufficiently specific to actual costs of providing the service for many use cases * Price things more generally in a way that more or less probably results in about the same revenue The problem with option #2 is that it's dependent upon some customers implicitly subsidizing others (e.g. any "unlimited storage" backup solution). It can work a lot of times, but sometimes it doesn't, and at AWS's scale, that could change quickly. It'll upset a lot of customers if they then have to restructure pricing to account for a small subset of customers' use cases being too costly to provide.
- joosters 6y ago
  Doesn't that kind of problem get easier at large scale? It's like any time you have a limited resource to share out. e.g. if an ISP has 150mbits of connectivity, it would be difficult to provide a '100mbit' service to two homes - one busy user will clearly hit the other. But if you have 1500mbits to share out to 20 homes, the service will seem much better. As you scale up, it becomes easier to provision less per customer.
  - inlined 6y ago
    Economy of scale is different when you’re providing a cloud business because your costs (and hopefully revenue) are dominated by the head customers. Imagine a company whose business model isn’t profitable for AWS becomes the next Dropbox. Suddenly the tail can’t subsidize the head. And even if it could, the business is shot. You always need the head to subsidize the tail (e.g. free trials). You can’t let the head become unprofitable.
    - joosters 6y ago
      All these are problems, yes, but my point is they become more manageable when you have more customers. Having Dropbox as a client would be exceedingly difficult if you only have two other customers, but it gets easier as you add more customers and scale up.
outworlder 6y ago
> The fact that there are consultants who specialize in figuring out AWS billing was, in retrospect, a warning sign. Well, there are whole organizations in my company dedicated to installation, procurement, provisioning, ordering(and tracking of said orders), of datacenter resources and hardware. We are spoiled.
edaemon 6y ago
I always chuckled at the "AWS Simple Monthly Calculator": https://calculator.s3.amazonaws.com/index.html https://calculator.s3.amazonaws.com/index.html It isn't simple at all! Not only is it complex and fairly daunting, you also need pretty deep knowledge of AWS to fill it out correctly. They eventually introduced an improved calculator that dropped the "simple" branding: https://calculator.aws/ https://calculator.aws/
- takeda 6y ago
  The simple calculator also didn't calculate all the cost (didn't try the new one) maybe by "simple" they meant incomplete?
sharpy 6y ago
Billing is really hard. I've had projects that literally sat on the shelves for months while we finalized our billing model, after realizing it would significantly increase the cost for our largest customers.

doorstar 6y ago

I've been horrified at how bad this is on Azure, I'm sorry to hear that AWS has the same issue. I've signed in some mornings only to discover that I've been charged $20 for a MongoDB instance that's been sitting idle all night. Support can only come back with some hand-waving about RUs, a term that defies any clear definition.

zaidf 6y ago
This is a problem across enterprise SaaS companies. At my last company (small enterprise SaaS biz <$50M ARR), I got so tired of this consuming so much of my time. I literally hacked together some code so I could point at any line item from any bill generated by our "system" and it would try to answer "why did I get billed $x for this?" so a five year old could understand. I wish there were SaaS products that would effortlessly audit enterprise SaaS bills with support for more than just the top 1% SaaS products.
- doorstar 6y ago
  I was poking around with their CLI to see if I could put together a cron job - this artifact now costs more than $X so delete it. Unfortunately if there's a 'get current charges' command I haven't found it. I have a ticket open with MS support. I suppose it's to their credit that they're listening to me but it's been a few weeks with only the aforementioned hand-waving. I started just sending them screenshots showing that their own dashboard showed no usage during the times I'd gotten charged.
FridgeSeal 6y ago
Dealing with Azure causes me physical pain. AWS you can poke around and figure it out, Azure is a Kafkaesque nightmare of infinitely confusing UI/UX, a permission system that is like trying to hit a moving target in a dark room, being moved by someone that hates you, doing anything seems needlessly complex, and trying to replicate/test anything locally is horrible (looking at you Azure functions).
- doorstar 6y ago
  My sibling... I'm trapped in this mess because the nature of our Client is that they'd consider anything besides Azure as a slap in the face, but it's been absolutely the most miserable task I've taken on in years. As a bonus I'm getting side-eyed by teammates who have a naive faith in The Cloud and can't comprehend why this is taking so long and why I can't just give them a cost estimate.
  - qchris 6y ago
    As an aside, unless I'm reading this very wrong, this is the first time I've seen the use of "my sibling" as a gender-neutral replacement of "my brother/sister" to express empathy. I'm delighted by it.
- lucianf 6y ago
  Semi-shameless plug (I work for Oracle) but have you tried OCI? You might find both the console UI/UX and the cloud calculator pretty decent. That's just my personal feel, coming over to work as an Oracle cloud consultant with previous AWS/Azure experience. I'd argue it's normal those two are where they are; AWS is now a spaghetti of products, while Microsoft are anti-KISS by DNA.
  - scarface74 6y ago
    Yes people who want to avoid lock in really want to get in bed with “call us for pricing” Oracle.
    - 6y ago

ken 6y ago

I'm as anti-AMZN/AWS as anyone these days, but to their credit, Amazon has their own teams of consultants who will come to your business for free to analyze your usage of AWS and tell you how to save money. They openly joked about how it was their job to get customers to spend less money with their company.

skrebbel 6y ago
That's not openly joking, it's sales. If you feel like you're way overpaying for AWS, you're much likelier to look out for alternatives. So if they help their biggest accounts save some money, it'll net them way more in the long run.
- bt1a 6y ago
  Right - it gives the illusion that you're 'getting an amazing deal and wow these people are great' when it reality you're probably getting gouged without the consulting and have to opt in for fair prices.
  - skrebbel 6y ago
    Yep. To be frank I think that spinning it as a joke and not an honest retention tactic almost feels like lying.
  - Spivak 6y ago
    I mean it's a retention tactic at the company scale but for the individual consultant they're genuinely trying to help you out. I frequent a local family-owned Italian restaurant/pizza place and my usual order jalapeños, banana peppers, and roasted garlic so I do a build you own. Totally fine, happy to pay the price. But one day I come in and the woman working the take-out counter sees my order before it goes in and tells me that if I order a "meat lovers", remove two toppings and sub the rest it comes out way cheaper. So now I'm the vegetarian that always gets a meat lovers and she makes the joke every time I come in. I don't think it's weird or nefarious that a business doesn't have completely consistent pricing and holding it differently can save you money. When we spec out services on AWS we're using the official calculator when deciding whether it's "worth it" so to us at least it genuinely feels like we're saving money.

moduspol 6y ago

I usually try to cut them some slack. I mean, they're operating this stuff at Internet scale and have never (AFAIK) raised prices. They've basically got two options: * Price things in an intricate way sufficiently specific to actual costs of providing the service for many use cases * Price things more generally in a way that more or less probably results in about the same revenue The problem with option #2 is that it's dependent upon some customers implicitly subsidizing others (e.g. any "unlimited storage" backup solution). It can work a lot of times, but sometimes it doesn't, and at AWS's scale, that could change quickly. It'll upset a lot of customers if they then have to restructure pricing to account for a small subset of customers' use cases being too costly to provide.

joosters 6y ago
Doesn't that kind of problem get easier at large scale? It's like any time you have a limited resource to share out. e.g. if an ISP has 150mbits of connectivity, it would be difficult to provide a '100mbit' service to two homes - one busy user will clearly hit the other. But if you have 1500mbits to share out to 20 homes, the service will seem much better. As you scale up, it becomes easier to provision less per customer.
- inlined 6y ago
  Economy of scale is different when you’re providing a cloud business because your costs (and hopefully revenue) are dominated by the head customers. Imagine a company whose business model isn’t profitable for AWS becomes the next Dropbox. Suddenly the tail can’t subsidize the head. And even if it could, the business is shot. You always need the head to subsidize the tail (e.g. free trials). You can’t let the head become unprofitable.
  - joosters 6y ago
    All these are problems, yes, but my point is they become more manageable when you have more customers. Having Dropbox as a client would be exceedingly difficult if you only have two other customers, but it gets easier as you add more customers and scale up.

outworlder 6y ago

> The fact that there are consultants who specialize in figuring out AWS billing was, in retrospect, a warning sign. Well, there are whole organizations in my company dedicated to installation, procurement, provisioning, ordering(and tracking of said orders), of datacenter resources and hardware. We are spoiled.

edaemon 6y ago

I always chuckled at the "AWS Simple Monthly Calculator": https://calculator.s3.amazonaws.com/index.html https://calculator.s3.amazonaws.com/index.html It isn't simple at all! Not only is it complex and fairly daunting, you also need pretty deep knowledge of AWS to fill it out correctly. They eventually introduced an improved calculator that dropped the "simple" branding: https://calculator.aws/ https://calculator.aws/

takeda 6y ago
The simple calculator also didn't calculate all the cost (didn't try the new one) maybe by "simple" they meant incomplete?

sharpy 6y ago

Billing is really hard. I've had projects that literally sat on the shelves for months while we finalized our billing model, after realizing it would significantly increase the cost for our largest customers.