5 ms·

I’ve accidentally hit myself with a bigger than expected AWS bill (just $500 but as a student I didn’t really want to spend that much). So I get being annoyed w

I’ve accidentally hit myself with a bigger than expected AWS bill (just $500 but as a student I didn’t really want to spend that much). So I get being annoyed with the pricing model. But, I don’t think the idea of just stopping charging works. For example, I had some of their machine image thingies (AMI) on my account. They charged me less than a dollar a month, totally reasonable. The only reasonable interpretation of “emergency stop on all charges completely” would be to delete those images (as well as shutting down my $500 nodes). This would have been really annoying, I mean putting the images together took a couple hours. And that’s just for me. With accounts that have multiple users—do you really delete all the disk images on a business’s account, because one of their employees used compute to hit their spend limit? No, I think cloud billing is just inherently complicated.

falcor84 1y ago
> The only reasonable interpretation of “emergency stop on all charges completely” would be to delete those images I disagree; a reasonable but customer-friendly interpretation would be to move these into a read-only "recycle bin" storage for e.g. a month, and only afterwards delete them if you don't provide additional budget.
- the8472 1y ago
  Which they already do for account closure, supposedly, I have never tested it. https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-closing.html https://docs.aws.amazon.com/accounts/latest/reference/manage...
appreciatorBus 1y ago
Yeah I get it. It just irks that it's something I'd like to spend more time with and learn, but at every corner I feel like I'm exposing myself. For what I have done w/AWS & GCP so far with personal accounts, complete deletion of all resources & images would be annoying to be sure, but still preferable to unlimited liability. Ofc most companies using it won't be in that boat so IDK.
exe34 1y ago
> But, I don’t think the idea of just stopping charging works. I'm sorry but this is complete bullshit. they can set a default limit of 1 trillion dollars and give us the option to drop it to $5. there's a good reason they won't do it, but it's not this bullshit claim that's always bandied about.
- adhamsalama 1y ago
  I hate how every time this issue mentioned everyone's response is that it would hurt the companies. Literally just make it an option. It's not that difficult for some of the smartest engineers in the world to implement it.
- bee_rider 1y ago
  How would you resolve the situation where ongoing storage costs cause the limit (whatever it is) to be exceeded?
  - ipaddr 1y ago
    Like every other service. They warn, freeze access and give you a period of time to pay. What happens to your files when you refuse to pay the bill?
  - exe34 1y ago
    I wouldn't. I've explained what I want.
    - bee_rider 1y ago
      There isn’t an option to not resolve “you’ve reached your billing limit and now storage charges are exceeding it.” You can resolve it by unceremoniously dumping the user data. You can resolve it by… continuing to charge the user, and holding their files hostage until they pay the back storage charges, and then the egress fees (so, it isn’t really a limit at all). Or you can resolve it by just giving the user free storage by some other name. Just saying that there should be a limit is not an explanation.
mgkimsal 1y ago
> But, I don’t think the idea of just stopping charging works. You don't stop CHARGING. You stop providing the service that is accumulating charges in excess of what limit I set. And you give some short period of time to settle the bill, modify the service, etc. You can keep charging me, but provide a way to stop the unlimited accrual of charges beyond limits I want to set. > No, I think cloud billing is just inherently complicated. You're making it more complicated than it needs to be. > The only reasonable interpretation of “emergency stop on all charges completely” would be to delete those images. It's by far certainly not the 'only reasonable interpretation'. "Stop all charges" is a red herring. No one is asking for a stop on charges. They want an option to stop/limit/cap the stuff that causes the charges.
- bee_rider 1y ago
  So, are you looking for some “rate of charges” cap? Like, allow the charges to accumulate indefinitely, but keep track of how much $/sec is being accumulated, and don’t start up new services if it would cause the rate of charges to pass that threshold? Might work. I do think that part of the appeal of these types of services is that you might briefly want to have a very high $/sec. But the idea makes sense, at least.
  - mgkimsal 1y ago
    A theme of many of the horror stories is something like "I set up something personal, costing a few dollars a month, and I was DDOSed or (in earlier terms) slashdotted out of the blue, and I now have a bill for $17k accumulated over 4 hours". As someone else pointed out, some(?) services prevent unlimited autoscaling, but even without unlimited, you may still hit a much larger limit. Being able to say 'if my bill goes above $400, shut off all compute resources' or something like that. Account is still on, and you have X days (3? 1? 14?) to re-enable services, pay the bill, or proceed as you wish. Yes, you might still want some period of high $/sec, but nearly every horror story in this vein ends with an issue with the final bill. Whether I burn $300 in 5 minutes or 26 days, I want some assurance that the services that are contributing most to that - likely/often EC2 or lambda in the AWS world - will be paused to stop the bleeding. If you could pipe "billing notification" SNS message to something that could simply shut off public network access to certain resources, perhaps that would suffice. I imagine there's enough internal plumbing there to facilitate that, but even then, that's just AWS - how other cloud providers might handle that would be different. Having it be a core feature would be useful. I was on a team that had our github CI pipeline routinely shutdown multiple times over a few weeks because some rogue processes were eating up a lot of minutes. We may have typically used $50/$100 per month - suddenly it was $100 in a day. Then... $200. Github just stopped the ability to run, because the credits used were over the limits. They probably could run their business where they would have just moved to charging us hundreds per day, perhaps with an email to an admin, and then set the invoice at $4500 for the month. But they shut down functionality a bit after the credits were exhausted.
fredophile 1y ago
There is no reason that cloud providers shouldn't be able to set up the same kind of billing options that advertisers have had access to for years. In Google and Meta ads I can set up multiple campaigns and give each campaign a budget. When that budget gets hit, those ads stop showing. Why would it be unreasonable to expect the same from AWS?
- bee_rider 1y ago
  Cloud providers charge for holding data, for ingress/egress, and for compute (among other things). If I hit my budget by using too much compute, then keeping my data will cause the budget to be exceeded. The difference is that cloud providers charge you for the “at rest” configuration, doing nothing isn’t free.
  - pythonaut_16 1y ago
    Great so they can give you an option to kill all charges except basic storage. Or let you reserve part of your budget for storage. Or let you choose to have everything hard deleted. Surely these billion and trillion dollar companies can figure out something so basic.
    - 2muchcoffeeman 1y ago
      How many small charges get written off though? If you make a $20 mistake, maybe you let it go and just pay. Is that worth the support to refund the 10k and 100k charges? Maybe it is.
- easyThrowaway 1y ago
  Because usually marketing works next to administration and legal, while dev, devops or infra is 2-3 layers of management below.

falcor84 1y ago

> The only reasonable interpretation of “emergency stop on all charges completely” would be to delete those images I disagree; a reasonable but customer-friendly interpretation would be to move these into a read-only "recycle bin" storage for e.g. a month, and only afterwards delete them if you don't provide additional budget.

the8472 1y ago
Which they already do for account closure, supposedly, I have never tested it. https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-closing.html https://docs.aws.amazon.com/accounts/latest/reference/manage...

appreciatorBus 1y ago

Yeah I get it. It just irks that it's something I'd like to spend more time with and learn, but at every corner I feel like I'm exposing myself. For what I have done w/AWS & GCP so far with personal accounts, complete deletion of all resources & images would be annoying to be sure, but still preferable to unlimited liability. Ofc most companies using it won't be in that boat so IDK.

exe34 1y ago

> But, I don’t think the idea of just stopping charging works. I'm sorry but this is complete bullshit. they can set a default limit of 1 trillion dollars and give us the option to drop it to $5. there's a good reason they won't do it, but it's not this bullshit claim that's always bandied about.

adhamsalama 1y ago
I hate how every time this issue mentioned everyone's response is that it would hurt the companies. Literally just make it an option. It's not that difficult for some of the smartest engineers in the world to implement it.
bee_rider 1y ago
How would you resolve the situation where ongoing storage costs cause the limit (whatever it is) to be exceeded?
- ipaddr 1y ago
  Like every other service. They warn, freeze access and give you a period of time to pay. What happens to your files when you refuse to pay the bill?
- exe34 1y ago
  I wouldn't. I've explained what I want.
  - bee_rider 1y ago
    There isn’t an option to not resolve “you’ve reached your billing limit and now storage charges are exceeding it.” You can resolve it by unceremoniously dumping the user data. You can resolve it by… continuing to charge the user, and holding their files hostage until they pay the back storage charges, and then the egress fees (so, it isn’t really a limit at all). Or you can resolve it by just giving the user free storage by some other name. Just saying that there should be a limit is not an explanation.

mgkimsal 1y ago

> But, I don’t think the idea of just stopping charging works. You don't stop CHARGING. You stop providing the service that is accumulating charges in excess of what limit I set. And you give some short period of time to settle the bill, modify the service, etc. You can keep charging me, but provide a way to stop the unlimited accrual of charges beyond limits I want to set. > No, I think cloud billing is just inherently complicated. You're making it more complicated than it needs to be. > The only reasonable interpretation of “emergency stop on all charges completely” would be to delete those images. It's by far certainly not the 'only reasonable interpretation'. "Stop all charges" is a red herring. No one is asking for a stop on charges. They want an option to stop/limit/cap the stuff that causes the charges.

bee_rider 1y ago
So, are you looking for some “rate of charges” cap? Like, allow the charges to accumulate indefinitely, but keep track of how much $/sec is being accumulated, and don’t start up new services if it would cause the rate of charges to pass that threshold? Might work. I do think that part of the appeal of these types of services is that you might briefly want to have a very high $/sec. But the idea makes sense, at least.
- mgkimsal 1y ago
  A theme of many of the horror stories is something like "I set up something personal, costing a few dollars a month, and I was DDOSed or (in earlier terms) slashdotted out of the blue, and I now have a bill for $17k accumulated over 4 hours". As someone else pointed out, some(?) services prevent unlimited autoscaling, but even without unlimited, you may still hit a much larger limit. Being able to say 'if my bill goes above $400, shut off all compute resources' or something like that. Account is still on, and you have X days (3? 1? 14?) to re-enable services, pay the bill, or proceed as you wish. Yes, you might still want some period of high $/sec, but nearly every horror story in this vein ends with an issue with the final bill. Whether I burn $300 in 5 minutes or 26 days, I want some assurance that the services that are contributing most to that - likely/often EC2 or lambda in the AWS world - will be paused to stop the bleeding. If you could pipe "billing notification" SNS message to something that could simply shut off public network access to certain resources, perhaps that would suffice. I imagine there's enough internal plumbing there to facilitate that, but even then, that's just AWS - how other cloud providers might handle that would be different. Having it be a core feature would be useful. I was on a team that had our github CI pipeline routinely shutdown multiple times over a few weeks because some rogue processes were eating up a lot of minutes. We may have typically used $50/$100 per month - suddenly it was $100 in a day. Then... $200. Github just stopped the ability to run, because the credits used were over the limits. They probably could run their business where they would have just moved to charging us hundreds per day, perhaps with an email to an admin, and then set the invoice at $4500 for the month. But they shut down functionality a bit after the credits were exhausted.

fredophile 1y ago

There is no reason that cloud providers shouldn't be able to set up the same kind of billing options that advertisers have had access to for years. In Google and Meta ads I can set up multiple campaigns and give each campaign a budget. When that budget gets hit, those ads stop showing. Why would it be unreasonable to expect the same from AWS?

bee_rider 1y ago
Cloud providers charge for holding data, for ingress/egress, and for compute (among other things). If I hit my budget by using too much compute, then keeping my data will cause the budget to be exceeded. The difference is that cloud providers charge you for the “at rest” configuration, doing nothing isn’t free.
- pythonaut_16 1y ago
  Great so they can give you an option to kill all charges except basic storage. Or let you reserve part of your budget for storage. Or let you choose to have everything hard deleted. Surely these billion and trillion dollar companies can figure out something so basic.
  - 2muchcoffeeman 1y ago
    How many small charges get written off though? If you make a $20 mistake, maybe you let it go and just pay. Is that worth the support to refund the 10k and 100k charges? Maybe it is.
easyThrowaway 1y ago
Because usually marketing works next to administration and legal, while dev, devops or infra is 2-3 layers of management below.