5 ms·
How can it be that unlimited high costs can arise with AWS?
Hello all,
I would like your opinion on unlimited costs at AWS.
One of my customers, had, the perfect use case for the Step Functions I read about. I worked with it for a few days and was able to see the first results quite quickly. Shortly after that, I started the service that used the API gateway to accept data and process it through multiple processes using Step Functions. The cost was about 50$-100$ per day. Due to another feature request, from my customer, I extended the Step Function with a waiting loop, which looked ideal in my mind. Unfortunately, my idea turned out to be a big mistake. I built some kind of loop in one of the steps. At the beginning everything looked fine and my workflow worked exactly as it should. I watched the Step Machine for another day. Everything looked good and I took a few days off after a stressful time. However, in my absence the queue kept growing and suddenly I had 1000 tasks running in parallel and the costs shot up to $2000 a day. On the last day of my little break, I pulled up the AWS iOS app and frightened when I saw the "Estimated Costs" for this month. 33,000 $.
This project is my first contact with AWS and went completely wrong. I directly contacted the support and hoped that I can be helped. I hoped they could give me a refund.
The whole ticket was opened on 01.06.2021 and closed on 10.06.2021. 9 days I hoped that I could be helped. At the end they apologized for the waiting time, but they could not offer me a solution.
For me as a self-employed person it is a disaster, since it is my fault.
- What would have happened if the infinite loop, would cost $10,000 per day? When would AWS respond? For every credit, no matter how small, the credit rating is checked, why can I incur unlimited costs with AWS?
- What can I do better in the future and more importantly, how can I prevent that costs could rise unlimited?
- Is there a way to turn off all services at once?
- aynyc 5y agoBilling alerts. That should be the second thing you set up with AWS after MFA on the root. For managed services, never put them into production until you understand the cost structure. Unlimited scalability == unlimited cost.
- deleted 5y ago[deleted]
- depod 5y agoThank you for your opinion. The billing alerts I actually had set up, stupidly I set up a budget of 1000$ hoping the budget in that case means all services will be shut down when they reach that limit. I received an email after the first 500$. I never thought I could spend more than $100 a day, so I didn't check with the Budget Manager often enough. It's my own fault for working on that premise and not checking what budget means on AWS.
- moralestapia 5y agoI always had this doubt Do billing alerts work immediately? Or do they let you know like an hour, a day, or whatever, after the limit has been reached?
- deleted 5y ago[deleted]
- isbvhodnvemrwvn 5y agoNot immediately, especially with predicted costs. In some cases with new accounts and low usage it can take hours for anything to show up.
- mrfusion 5y agoIt’s a really dark pattern that you can’t set an account maximum and get cut off.
- aynyc 5y agoWhat does cut off means? If you add a lot of files to your S3, does it mean S3 should stop allowing more uploads or should AWS start deleting your files?
- Thetawaves 5y ago"Your storage device is full." Why should this be any different?
- aynyc 5y agoBut you still have to pay for long term storage, that means your bill is still going up.
- Spooky23 5y agoEver use Glacier? Obviously, the good analysts at AWS have the talent to cook up complex billing models.
- aynyc 5y agoDid AWS make Glacier free? At the end of the day, you are renting hardware, not buying it. I do think AWS should've done a better job at billing notification and documenting the potential disaster.
- quickthrower2 5y agoDefinitely stop more uploads. Deleting files is nasty but I don’t see an alternative. But there must be lots of alerts if they do this! Limit set separately for backups would make sense so you can recover. Even if your production limit starts chaos monkeying.
- andrewmcwatters 5y agoI avoid AWS like the plague because it's not small business friendly. It just seems foolish to me why any org that wasn't mid-sized or above would use it. There are plenty of cheaper vendors in the world! Tons of them! They can literally be 10x cheaper with more resources. Wild. If you actually have to use AWS though because you have RFP requirements to meet, then I would ask whether or not you can halt instances using CloudWatch. I don't know the answer to that, but Amazon makes it seem like you can. Although their documentation seems to suggest you can only do it for idle instances.[1] [1]: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/UsingAlarmActions.html https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...
- max_hammer 5y agoReminds me of warning message from Linux sys admins. ``` #3) With great power comes great responsibility. ``` I would suggest to write to AWS support again, explaining your financial situation and requesting some waiver on the total amount. In am aware of few cases, where AWS was supportive and reduced the bill. But it totally depends on your luck and person handling your ticket. Only way to close all services in one go is to close your account.
- Aperocky 5y agoApt comparison, AWS is basically sudo in terms of resource provisioning.
- aynyc 5y agoOnly way to close all services in one go is to close your account. AFAIK, this isn't completely true[1]. Closing your account might not automatically terminate all your active resources. You might continue to incur charges for some of your active resources even after you close your account. https://aws.amazon.com/premiumsupport/knowledge-center/terminate-resources-account-closure/ https://aws.amazon.com/premiumsupport/knowledge-center/termi...