8 ms·

It hasn't been implemented because it is nonsensical. So you hit your limit. Is all your storage deleted? You can't receive an alert because that costs somethi

It hasn't been implemented because it is nonsensical. So you hit your limit. Is all your storage deleted? You can't receive an alert because that costs something (even if it's a fraction of a cent) to send. Are your domains forfeit? Audit logs destroyed? There's no reasonable way to implement this. Billing alerts are the best you can do on this problem and I think it would be a good faith move for AWS to enable some by default but a limit just doesn't make sense on any level. EDIT: Lots of people proposing solutions that work for them. AWS has to think of everyone. And they did. That's why budget alerts exist and you can respond in the way you choose. Everyone's conflicting ideas for how to solve this can be implemented today on top of billing alerts/actions[1]. Case closed. [1] https://aws.amazon.com/blogs/aws-cost-management/get-started-with-aws-budgets-actions/ https://aws.amazon.com/blogs/aws-cost-management/get-started...

viraptor 6y ago
Resources can be divided into persistent and not. People could opt into "we'll cut off traffic and prevent spawning new resources" limit without affecting storage, domains, logs, etc. This also aligns well with the "planned vs unexpected" costs.
darkwizard42 6y ago
I feel like this response has to be said every time this sort of thread comes up when someone discusses their unexpectedly large bill. The worst case scenario is that the user configures their limits in a poor way then turns around and blames Amazon when something breaks in their project and Amazon points back at the limit. It would just be bad all around...so all in all, I think alerts are the best way to do this...
heipei 6y ago
At the very least allow users to roughly define what happens when the threshold is reached: S3 will still store data but maybe not serve anything, Lambdas will still be defined but not run, EC2 instances will keep running, etc. It's not supposed to be a hard cap as in "don't spend a penny more than my $5" but more as in "do everything possible and reasonable so the customer doesn't wake up to a sudden $20k overnight AWS bill".
- andrewguenther 6y ago
  You can do this by monitoring billing alerts and automatically shutting down non-critical infrastructure you have. "Do everything possible" is up to a customer to define because it depends on their infrastructure, AWS cannot define this in a way that would make anyone happy.
- sathyabhat 6y ago
  This sort of possible > AWS Budgets now allows you to configure actions — responses to cost and usage in your account or set of accounts— that will be applied automatically or via a workflow approval process once a budget target has been exceeded. There are three action types: Identity and Access Management (IAM) policies, Service Control policies (SCPs), or target running instances (EC2 or RDS). Actions can be configured for actual (after they’ve occurred) or for forecasted (before they occur) budgeted amounts. https://aws.amazon.com/blogs/aws-cost-management/get-started-with-aws-budgets-actions/ https://aws.amazon.com/blogs/aws-cost-management/get-started...
thaumasiotes 6y ago
Nonsense. If you're going to bill for the alert, send it when the charge is the difference between the current bill and the limit. Of course, it's no more realistic to charge people for the alert than it is to charge them for email receipts. Do you not get those from Amazon? You implement the feature cap by ceasing to provide the feature once the account is over the billing limit. It is always possible to project the cost of "freeze everything until the end of the month", because there is no unknown information in that scenario. Nothing you've listed is an actual problem.
foota 6y ago
These questions have reasonable answers, but it's admittedly a very hard problem. Anything that's a request should fail and likely everything else stored etc., should stay as is. It's not perfect, but a lot better.
neolog 6y ago
Having to make some design decisions doesn't mean it's impossible to design.
- andrewguenther 6y ago
  Budget alerts were the design decision made. You can set up your own infra to listen to budget alerts and cap your own costs. This is a feature that only makes sense to put in the customer's control because a general solution would work for no one. It's a great solution to the problem imo.
  - viraptor 6y ago
    This is way too complicated for people who use AWS on a small scale. Maybe if they provided some template to implement that system in the first place? Anyway, they should be able to protect people from an unexpected $1k charge. Once you deal with alerts triggering automatic infrastructure changes, $1k is likely a value you won't even notice.
    - andrewguenther 6y ago
      https://aws.amazon.com/blogs/aws-cost-management/get-started-with-aws-budgets-actions/ https://aws.amazon.com/blogs/aws-cost-management/get-started...
      - viraptor 6y ago
        This is still a bit low level. You need to think through the policies and they don't affect the traffic. I don't think that's enough for beginners.
  - detaro 6y ago
    Isn't the AWS Budget stuff recalculated only every few hours, or do the alerts somehow get quicker updates?
monsieurbanana 6y ago
> but a limit just doesn't make sense on any level. A limit very much makes sense of many levels, just not all of them at once. You're arguing against a hard limit of every single AWS service. Nobody wants their storage deleted once their budget hits a limit, what they want, and is reasonable to have, is a limit that prevents AWS from spinning additional EC instances. Or a limit that blocks your usage of Textractor. Or many other limits an user could individually set-up for the services where a limit makes sense.
nielsole 6y ago
The OPs request is for S3. Especially limiting egress fees and allowing some other fees. > There's no reasonable way to implement this [...] on any level That's not true (as OP proposes a reasonable way)
NoPicklez 6y ago
My feeling with this is potentially where small businesses or startups don't want something accidental to incur a stupidly high bill. But yes, you could set up alerts to monitor when this is likely to occur. But to do this across each service for each reason your bill might spike drastically might be a challenge. In the linked article the user was alerted, but they were asleep. One of the proposed solutions is to have usage limits which are calculated based on a maximum monthly cost and AWS will work itself within that.
atoav 6y ago
I used rented renderfarms a lot for VFX work. And many of those had a "if your workload stays like it is it will cost you roughly X after n frames rendered"-feature. Otheres made you transfer money first and you would get a warning before that money would run out. The problem is that you can run into unexpectedly high cost without warning or option to decide whether it is worth it and one of the solutions would be to just tell people when the rate at which they use your stuff increases or decreases by a certain threshold. And because they can bill you, they also know (roughly) what the things that you use cost. I never used AWS for serious things, but not being able to decide my spending will be a serious argument against it.
croes 6y ago
You do understand that this is not a binary feature? Not just on or off. The point is not that there should be no costs, but that they should be minimised. This means that data that has to be written, such as logs, should continue to be written and may also incur costs, but everything else should be switched off to prevent you from suddenly being stuck with thousands of dollars in debt.
capybara_2020 6y ago
As a small dev, I do not care if data is lost and they stop storing the data/turn off machines. I am more concerned about the cost. I have had two cases. 1. Misconfigured software that we accidentally pushed into production and by the time we noticed it like an hour later we were $100s out. Luckily it was not in the 1000s other that would have sunk the business. 2. An old account of ours got hacked and people spun up like 50-100 servers and it happened overnight and we had a bill of $50k-$70k. Luckily Amazon wrote that off for us. Otherwise I have no clue how we would have ever paid that. These are usually extraordinary cases, so even if there is a data loss or your servers are shut down, it is a better option than going down the hole with 10s of 1000s in charges.
preommr 6y ago
> You can't receive an alert because that costs something (even if it's a fraction of a cent) to send. Yea, what a crazy world that would be if AWS just gave each account free stuff like free data transfer for the first gb, or free s3 storage for the first ~100mb
raverbashing 6y ago
> AWS has to think of everyone. Sure. That's why you make it optional. And configurable in a way But at a very basic idea, blocking new services from being started and throttling of existing ones would go a long way in helping. A very long way.
- andrewguenther 6y ago
  You can implement this on top of budget actions today: https://aws.amazon.com/blogs/aws-cost-management/get-started-with-aws-budgets-actions/ https://aws.amazon.com/blogs/aws-cost-management/get-started...
  - jiggawatts 6y ago
    I have small and medium scale customers that have hundreds of distinct service types in Azure or AWS. Sometimes both. Several have had spending blowouts where a hard limit would have helped. None have the engineering capacity to figure out how to cap the spending on each one of the individual services, each with their unique and special API. You're arguing that Amazon can't afford the difficult engineering of spending caps, but the very customers that need spending caps because their budgets are so constrained are so well moneyed that they should all individually be able to invent a solution, engineer it, and manage it?
IanCal 6y ago
A limit to the cost to me. I don't care what the cost to Amazon is. > So you hit your limit. Is all your storage deleted? Easy. Cut access, don't delete the files, set a time limit for resolving the problem before they are deleted. > EDIT: Lots of people proposing solutions that work for them. AWS has to think of everyone. No they don't. A feature that solves the main problem for lots of users can be added and for the users that need a complex solution, well, they can justnot use the simple one.
mpmpmpmp 6y ago
Your comment makes it seem like it’s impossible for Amazon to determine you exceeded the threshold until after you exceeded it. Why couldn’t it work like rate limiting does but for resource consumption not just requests per X time?
Hermel 6y ago
That still does not explain why they refuse to answer the question.
johnwheeler 6y ago
What the hell good is a budget alert when you’ve launched 2000 high memory instances because your programmer screwed up?
- andrewguenther 6y ago
  Because those alerts are hookable and could shut down the instance.
  - matkoniecz 6y ago
    With delay measured in hours.
- whoknew1122 6y ago
  There's other things you can use for this, such as CloudWatch Alarms. And where are your code reviews?
Havoc 6y ago
>Everyone's conflicting ideas There is no reason why an absolute "don't ruin me financially" button can't be implemented in addition to everything you mention. >There's no reasonable way to implement this. Azure VS enterprise credits work this way. 150 hard cap with no credit card on the individual account.
anigbrowl 6y ago
This sort of dismissive attitude coupled with a circular justification is the epitome of user hostility, and an unattractive norm in the tech community.
ablekh 6y ago
> There's no reasonable way to implement this. Oh, really? Azure wants to have a word with you ... Yes, currently they don't enable this for subscriptions with commitment plans or with pay-as-you-go pricing, but it is not because it is not possible or feasible, as you argue - the technical infrastructure in the form of spending limits, spending budgets and quotas is there [1-3] and is available for select plans [4]. [1] https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/spending-limit https://docs.microsoft.com/en-us/azure/cost-management-billi... [2] https://docs.microsoft.com/en-us/partner-center/set-an-azure-spending-budget-for-your-customers https://docs.microsoft.com/en-us/partner-center/set-an-azure... [3] https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits https://docs.microsoft.com/en-us/azure/azure-resource-manage... [4] https://azure.microsoft.com/en-us/support/legal/offer-details https://azure.microsoft.com/en-us/support/legal/offer-detail...
- jiggawatts 6y ago
  Translation: "We have spending limits, but only for accounts where stopping the spending is important to us. For accounts where the spending is important to you, there cannot be a limit configured. Even though we wrote the code. In fact, we had to add extra code so that we could selectively remove this option."
  - ablekh 6y ago
    I like your sense of humor, even if it's sarcastic in this case. :-) However, the main point of my previous comment was to demonstrate that the above-cited statement by OP is false. As for selective - at this time - use of this option in Azure, I have mentioned it, but the big difference is that, should Microsoft decide to allow it for the larger subset of users (perhaps, including pay-as-you-go plans) eventually, the infrastructure is there, unlike in the AWS case (as I understand).
dataflow 6y ago
> It hasn't been implemented because it is nonsensical. If that's the case then why hasn't it been responded to? Is responding also nonsensical?

viraptor 6y ago

Resources can be divided into persistent and not. People could opt into "we'll cut off traffic and prevent spawning new resources" limit without affecting storage, domains, logs, etc. This also aligns well with the "planned vs unexpected" costs.

darkwizard42 6y ago

I feel like this response has to be said every time this sort of thread comes up when someone discusses their unexpectedly large bill. The worst case scenario is that the user configures their limits in a poor way then turns around and blames Amazon when something breaks in their project and Amazon points back at the limit. It would just be bad all around...so all in all, I think alerts are the best way to do this...

heipei 6y ago

At the very least allow users to roughly define what happens when the threshold is reached: S3 will still store data but maybe not serve anything, Lambdas will still be defined but not run, EC2 instances will keep running, etc. It's not supposed to be a hard cap as in "don't spend a penny more than my $5" but more as in "do everything possible and reasonable so the customer doesn't wake up to a sudden $20k overnight AWS bill".

andrewguenther 6y ago
You can do this by monitoring billing alerts and automatically shutting down non-critical infrastructure you have. "Do everything possible" is up to a customer to define because it depends on their infrastructure, AWS cannot define this in a way that would make anyone happy.
sathyabhat 6y ago
This sort of possible > AWS Budgets now allows you to configure actions — responses to cost and usage in your account or set of accounts— that will be applied automatically or via a workflow approval process once a budget target has been exceeded. There are three action types: Identity and Access Management (IAM) policies, Service Control policies (SCPs), or target running instances (EC2 or RDS). Actions can be configured for actual (after they’ve occurred) or for forecasted (before they occur) budgeted amounts. https://aws.amazon.com/blogs/aws-cost-management/get-started-with-aws-budgets-actions/ https://aws.amazon.com/blogs/aws-cost-management/get-started...

thaumasiotes 6y ago

Nonsense. If you're going to bill for the alert, send it when the charge is the difference between the current bill and the limit. Of course, it's no more realistic to charge people for the alert than it is to charge them for email receipts. Do you not get those from Amazon? You implement the feature cap by ceasing to provide the feature once the account is over the billing limit. It is always possible to project the cost of "freeze everything until the end of the month", because there is no unknown information in that scenario. Nothing you've listed is an actual problem.

foota 6y ago

These questions have reasonable answers, but it's admittedly a very hard problem. Anything that's a request should fail and likely everything else stored etc., should stay as is. It's not perfect, but a lot better.

neolog 6y ago

Having to make some design decisions doesn't mean it's impossible to design.

andrewguenther 6y ago
Budget alerts were the design decision made. You can set up your own infra to listen to budget alerts and cap your own costs. This is a feature that only makes sense to put in the customer's control because a general solution would work for no one. It's a great solution to the problem imo.
- viraptor 6y ago
  This is way too complicated for people who use AWS on a small scale. Maybe if they provided some template to implement that system in the first place? Anyway, they should be able to protect people from an unexpected $1k charge. Once you deal with alerts triggering automatic infrastructure changes, $1k is likely a value you won't even notice.
  - andrewguenther 6y ago
    https://aws.amazon.com/blogs/aws-cost-management/get-started-with-aws-budgets-actions/ https://aws.amazon.com/blogs/aws-cost-management/get-started...
    - viraptor 6y ago
      This is still a bit low level. You need to think through the policies and they don't affect the traffic. I don't think that's enough for beginners.
- detaro 6y ago
  Isn't the AWS Budget stuff recalculated only every few hours, or do the alerts somehow get quicker updates?

monsieurbanana 6y ago

> but a limit just doesn't make sense on any level. A limit very much makes sense of many levels, just not all of them at once. You're arguing against a hard limit of every single AWS service. Nobody wants their storage deleted once their budget hits a limit, what they want, and is reasonable to have, is a limit that prevents AWS from spinning additional EC instances. Or a limit that blocks your usage of Textractor. Or many other limits an user could individually set-up for the services where a limit makes sense.

nielsole 6y ago

The OPs request is for S3. Especially limiting egress fees and allowing some other fees. > There's no reasonable way to implement this [...] on any level That's not true (as OP proposes a reasonable way)

NoPicklez 6y ago

My feeling with this is potentially where small businesses or startups don't want something accidental to incur a stupidly high bill. But yes, you could set up alerts to monitor when this is likely to occur. But to do this across each service for each reason your bill might spike drastically might be a challenge. In the linked article the user was alerted, but they were asleep. One of the proposed solutions is to have usage limits which are calculated based on a maximum monthly cost and AWS will work itself within that.

atoav 6y ago

I used rented renderfarms a lot for VFX work. And many of those had a "if your workload stays like it is it will cost you roughly X after n frames rendered"-feature. Otheres made you transfer money first and you would get a warning before that money would run out. The problem is that you can run into unexpectedly high cost without warning or option to decide whether it is worth it and one of the solutions would be to just tell people when the rate at which they use your stuff increases or decreases by a certain threshold. And because they can bill you, they also know (roughly) what the things that you use cost. I never used AWS for serious things, but not being able to decide my spending will be a serious argument against it.

croes 6y ago

You do understand that this is not a binary feature? Not just on or off. The point is not that there should be no costs, but that they should be minimised. This means that data that has to be written, such as logs, should continue to be written and may also incur costs, but everything else should be switched off to prevent you from suddenly being stuck with thousands of dollars in debt.

capybara_2020 6y ago

As a small dev, I do not care if data is lost and they stop storing the data/turn off machines. I am more concerned about the cost. I have had two cases. 1. Misconfigured software that we accidentally pushed into production and by the time we noticed it like an hour later we were $100s out. Luckily it was not in the 1000s other that would have sunk the business. 2. An old account of ours got hacked and people spun up like 50-100 servers and it happened overnight and we had a bill of $50k-$70k. Luckily Amazon wrote that off for us. Otherwise I have no clue how we would have ever paid that. These are usually extraordinary cases, so even if there is a data loss or your servers are shut down, it is a better option than going down the hole with 10s of 1000s in charges.

preommr 6y ago

> You can't receive an alert because that costs something (even if it's a fraction of a cent) to send. Yea, what a crazy world that would be if AWS just gave each account free stuff like free data transfer for the first gb, or free s3 storage for the first ~100mb

raverbashing 6y ago

> AWS has to think of everyone. Sure. That's why you make it optional. And configurable in a way But at a very basic idea, blocking new services from being started and throttling of existing ones would go a long way in helping. A very long way.

andrewguenther 6y ago
You can implement this on top of budget actions today: https://aws.amazon.com/blogs/aws-cost-management/get-started-with-aws-budgets-actions/ https://aws.amazon.com/blogs/aws-cost-management/get-started...
- jiggawatts 6y ago
  I have small and medium scale customers that have hundreds of distinct service types in Azure or AWS. Sometimes both. Several have had spending blowouts where a hard limit would have helped. None have the engineering capacity to figure out how to cap the spending on each one of the individual services, each with their unique and special API. You're arguing that Amazon can't afford the difficult engineering of spending caps, but the very customers that need spending caps because their budgets are so constrained are so well moneyed that they should all individually be able to invent a solution, engineer it, and manage it?

IanCal 6y ago

A limit to the cost to me. I don't care what the cost to Amazon is. > So you hit your limit. Is all your storage deleted? Easy. Cut access, don't delete the files, set a time limit for resolving the problem before they are deleted. > EDIT: Lots of people proposing solutions that work for them. AWS has to think of everyone. No they don't. A feature that solves the main problem for lots of users can be added and for the users that need a complex solution, well, they can justnot use the simple one.

mpmpmpmp 6y ago

Your comment makes it seem like it’s impossible for Amazon to determine you exceeded the threshold until after you exceeded it. Why couldn’t it work like rate limiting does but for resource consumption not just requests per X time?

Hermel 6y ago

That still does not explain why they refuse to answer the question.

johnwheeler 6y ago

What the hell good is a budget alert when you’ve launched 2000 high memory instances because your programmer screwed up?

andrewguenther 6y ago
Because those alerts are hookable and could shut down the instance.
- matkoniecz 6y ago
  With delay measured in hours.
whoknew1122 6y ago
There's other things you can use for this, such as CloudWatch Alarms. And where are your code reviews?

Havoc 6y ago

>Everyone's conflicting ideas There is no reason why an absolute "don't ruin me financially" button can't be implemented in addition to everything you mention. >There's no reasonable way to implement this. Azure VS enterprise credits work this way. 150 hard cap with no credit card on the individual account.

anigbrowl 6y ago

This sort of dismissive attitude coupled with a circular justification is the epitome of user hostility, and an unattractive norm in the tech community.

ablekh 6y ago

> There's no reasonable way to implement this. Oh, really? Azure wants to have a word with you ... Yes, currently they don't enable this for subscriptions with commitment plans or with pay-as-you-go pricing, but it is not because it is not possible or feasible, as you argue - the technical infrastructure in the form of spending limits, spending budgets and quotas is there [1-3] and is available for select plans [4]. [1] https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/spending-limit https://docs.microsoft.com/en-us/azure/cost-management-billi... [2] https://docs.microsoft.com/en-us/partner-center/set-an-azure-spending-budget-for-your-customers https://docs.microsoft.com/en-us/partner-center/set-an-azure... [3] https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits https://docs.microsoft.com/en-us/azure/azure-resource-manage... [4] https://azure.microsoft.com/en-us/support/legal/offer-details https://azure.microsoft.com/en-us/support/legal/offer-detail...

jiggawatts 6y ago
Translation: "We have spending limits, but only for accounts where stopping the spending is important to us. For accounts where the spending is important to you, there cannot be a limit configured. Even though we wrote the code. In fact, we had to add extra code so that we could selectively remove this option."
- ablekh 6y ago
  I like your sense of humor, even if it's sarcastic in this case. :-) However, the main point of my previous comment was to demonstrate that the above-cited statement by OP is false. As for selective - at this time - use of this option in Azure, I have mentioned it, but the big difference is that, should Microsoft decide to allow it for the larger subset of users (perhaps, including pay-as-you-go plans) eventually, the infrastructure is there, unlike in the AWS case (as I understand).

dataflow 6y ago

> It hasn't been implemented because it is nonsensical. If that's the case then why hasn't it been responded to? Is responding also nonsensical?