5 ms·

That is an argument for why these overages occur. It isn't an argument for why customers should eat that cost rather than Amazon. In fact Amazon is in a much

by btilly 6y ago

That is an argument for why these overages occur. It isn't an argument for why customers should eat that cost rather than Amazon. In fact Amazon is in a much better position than customers to absorb those costs. Sure, they'd have to increase rates slightly to cover it. But it would give customers peace of mind. And if the costs become exorbitant, Amazon is in a better position to improve their own systems to reduce the amount of overages that people run into in practice. In theory, Amazon's first leadership principle is Customer Obsession. (See https://www.amazon.jobs/en/principles https://www.amazon.jobs/en/principles for the full list.) If they took that seriously, then setting this issue to rest for their customers would be a no-brainer.

_flux 6y ago
Should they simply automatically eat the costs then it would undoubtedly result in abuse. Just look at GitHub Actions. Oh I didn't think I was starting 1000 instances of miners in the EC2 GPU cloud, it certainly exceeded my configured budget, please give my money back..
- yorwba 6y ago
  That's where "if the costs become exorbitant, Amazon is in a better position to improve their own systems to reduce the amount of overages that people run into in practice" comes in. If starting 1000 instances exceeds your configured budget, they could simply not start them, and shut down whatever number of instances you managed to start as soon as their cost exceeds the budget.
  - _flux 6y ago
    I guess the question is how precise monitoring and reactionary system Amazon wants build for this, for an arguably marginal use case anyway; they do already provide Amazon Budgets for automatic actions when exceeding budgets, but it's quite not as real-time. And then making the niche cases favor the customer is an invitation to abuse. But least all Amazon accounts are tied to a credit card, so abusing in a scale similar to e.g. the GitHub case is not that easy.
- sneak 6y ago
  > Oh I didn't think I was starting 1000 instances of miners in the EC2 GPU cloud, it certainly exceeded my configured budget, please give my money back.. All AWS accounts start off without the ability to do this (via the quota system) and being able to start 1000 ec2 instances of any type is a setting that needs to be unlocked via a support request (which can never be done by that support person, but always needs to be escalated to the "service team" and takes about 1 business day).
- onion2k 6y ago
  It's trivial to add a clause that says "If you add a cap to the cost of your account you can only have 3 concurrent instances."
- mytherin 6y ago
  "You can only launch 1 concurrent EC2 instance per $100 on your max cap, if you want to launch 1000 instances of EC2 your max cap needs to be set to at least $100.000, or to $unlimited". There are many solutions to this problem to prevent abuse. All of which are way better for consumers than the status quo of everybody having an $unlimited spending limit.
- stefan_ 6y ago
  Don't give your customers unlimited margin and leverage then. See also: Archegos.
vineyardmike 6y ago
> In fact Amazon is in a much better position than customers to absorb those costs And if you call amazon support and talk to them, especially as an individual, you might get your bill cancelled.
- onion2k 6y ago
  you might get your bill cancelled The "might" in your post is doing a lot of work. FWIW I know of a startup whose video sharing app was used to reshare a pay per view football match and they incurred a $30k bandwidth bill that AWS did not cancel. That killed the startup. It was largely their own fault for not securing the platform well enough, or moderating popular streams, but being able to cap their AWS bill would have kept them in business..
- matkoniecz 6y ago
  I am not going to risk runaway costs in hope that Amazon "might" cancel it. Though, apparently population both caring about it and avoiding Amazon as result would pay less than cost of implementing it and not refunded income from catastrophic runaways.
jgalt212 6y ago
I would argue that AWS does not have a "Customer Obsession", and that's exactly why it's Amazon's most profitable business (by far) and the underwriter of all of Bezos's ambitions.
- whoknew1122 6y ago
  Disclosure: AWS employee. Support specifically. There are good things about my employer. There are bad things about my employer. AWS does indeed obsess about the customer. Every step along the chain there is someone there advocating for the customer. There are mechanisms to keep the customer in mind even for the developers who actually code the service and don't talk to customers on a daily basis. I've had many, many service team members shadow me as I worked their service's tickets. This is explicitly so they can see in real time customer pain-points. If a customer has a question about a unique use case, the service team will proactively reach out to support engineers to set up a call to discuss the use case further. There are monthly (or twice-monthly) meetings between support service owners (i.e. those people in support who 'own' a service) and service teams to identify the top issues customers are having with the service. AWS is constantly looking for ways to better assist customers, make support less difficult for customers, increase self-service options for customers, etc. I'm really, really curious where the basis behind your argument. Because from everything I've seen and been a part of, it's simply untrue.
  - jgalt212 6y ago
    $0 ingress charges and >> $0 egress charges make AWS roach motel.
    - whoknew1122 6y ago
      So basically you're complaining about pricing? Unlike other cloud providers, AWS has never had a price increase for a service. Just decreases. I guess 'customer obsession' would be giving away everything for free?
      - 6y ago

by btilly 6y ago

_flux 6y ago

Should they simply automatically eat the costs then it would undoubtedly result in abuse. Just look at GitHub Actions. Oh I didn't think I was starting 1000 instances of miners in the EC2 GPU cloud, it certainly exceeded my configured budget, please give my money back..

yorwba 6y ago
That's where "if the costs become exorbitant, Amazon is in a better position to improve their own systems to reduce the amount of overages that people run into in practice" comes in. If starting 1000 instances exceeds your configured budget, they could simply not start them, and shut down whatever number of instances you managed to start as soon as their cost exceeds the budget.
- _flux 6y ago
  I guess the question is how precise monitoring and reactionary system Amazon wants build for this, for an arguably marginal use case anyway; they do already provide Amazon Budgets for automatic actions when exceeding budgets, but it's quite not as real-time. And then making the niche cases favor the customer is an invitation to abuse. But least all Amazon accounts are tied to a credit card, so abusing in a scale similar to e.g. the GitHub case is not that easy.
sneak 6y ago
> Oh I didn't think I was starting 1000 instances of miners in the EC2 GPU cloud, it certainly exceeded my configured budget, please give my money back.. All AWS accounts start off without the ability to do this (via the quota system) and being able to start 1000 ec2 instances of any type is a setting that needs to be unlocked via a support request (which can never be done by that support person, but always needs to be escalated to the "service team" and takes about 1 business day).
onion2k 6y ago
It's trivial to add a clause that says "If you add a cap to the cost of your account you can only have 3 concurrent instances."
mytherin 6y ago
"You can only launch 1 concurrent EC2 instance per $100 on your max cap, if you want to launch 1000 instances of EC2 your max cap needs to be set to at least $100.000, or to $unlimited". There are many solutions to this problem to prevent abuse. All of which are way better for consumers than the status quo of everybody having an $unlimited spending limit.
stefan_ 6y ago
Don't give your customers unlimited margin and leverage then. See also: Archegos.

vineyardmike 6y ago

> In fact Amazon is in a much better position than customers to absorb those costs And if you call amazon support and talk to them, especially as an individual, you might get your bill cancelled.

onion2k 6y ago
you might get your bill cancelled The "might" in your post is doing a lot of work. FWIW I know of a startup whose video sharing app was used to reshare a pay per view football match and they incurred a $30k bandwidth bill that AWS did not cancel. That killed the startup. It was largely their own fault for not securing the platform well enough, or moderating popular streams, but being able to cap their AWS bill would have kept them in business..
matkoniecz 6y ago
I am not going to risk runaway costs in hope that Amazon "might" cancel it. Though, apparently population both caring about it and avoiding Amazon as result would pay less than cost of implementing it and not refunded income from catastrophic runaways.

jgalt212 6y ago

I would argue that AWS does not have a "Customer Obsession", and that's exactly why it's Amazon's most profitable business (by far) and the underwriter of all of Bezos's ambitions.

whoknew1122 6y ago
Disclosure: AWS employee. Support specifically. There are good things about my employer. There are bad things about my employer. AWS does indeed obsess about the customer. Every step along the chain there is someone there advocating for the customer. There are mechanisms to keep the customer in mind even for the developers who actually code the service and don't talk to customers on a daily basis. I've had many, many service team members shadow me as I worked their service's tickets. This is explicitly so they can see in real time customer pain-points. If a customer has a question about a unique use case, the service team will proactively reach out to support engineers to set up a call to discuss the use case further. There are monthly (or twice-monthly) meetings between support service owners (i.e. those people in support who 'own' a service) and service teams to identify the top issues customers are having with the service. AWS is constantly looking for ways to better assist customers, make support less difficult for customers, increase self-service options for customers, etc. I'm really, really curious where the basis behind your argument. Because from everything I've seen and been a part of, it's simply untrue.
- jgalt212 6y ago
  $0 ingress charges and >> $0 egress charges make AWS roach motel.
  - whoknew1122 6y ago
    So basically you're complaining about pricing? Unlike other cloud providers, AWS has never had a price increase for a service. Just decreases. I guess 'customer obsession' would be giving away everything for free?
    - 6y ago