3 ms·

I'm all up for hating corporations but how is this Amazon's problem/fault? You leaked a key, what do you expect Amazon to do? This is like software development

by atraac 5y ago

I'm all up for hating corporations but how is this Amazon's problem/fault? You leaked a key, what do you expect Amazon to do? This is like software development 101, protect any kind of secret/key, don't store/send it in plain text, don't make it public. He's using arguments like 'but as a solo founder...', what kind of excuse is this? I fully agree that AWS could improve detection of these kinds of things and that the guy needs actual support but this whole twitter thread sounds like he is blaming AWS for his own mistake. Just ask for help, don't blame them. No one is forcing anyone to use AWS, you know what you get into when you make an account.

wccrawford 5y ago
They expect Amazon to allow people to set maximize spending amounts on their accounts. They have steadfastly refused to implement this.
KronisLV 5y ago
In my eyes, the situation is simple: human beings are fallible. If there's a chance of your credentials leaking and being exploited that way, that will most certainly happen to someone sooner or later. I recall a VPS that i had years ago being hijacked due to Docker socket being exposed publicly (when i set one up for testing and was just learning about Docker), which was mining crypto throughout the night before i noticed it. Imagine what would happen if there had been some sort of auto scaling in place and i had used a managed Kubernetes solution (many of which didn't yet exist back then)... Is it Amazon's problem? Not really, apart from the bad PR that charging private individuals exorbitant sums (for them) might cause, so sometimes these bills are waived after people mess up. For the corporation at the size of Amazon, that's probably not too much in the grand scheme of things, but it's also understandable why people might be appropriately upset about not being able to set hard spending limits, merely alerts (which aren't enabled by default). That's just the reality that we live in for now, however one can also think about why a corporation that cares a lot about uptime might not necessarily want to implement functionality that'd let resources be easily cut off when spend limits are hit or something similar, since most businesses that utilize their services might just swallow the occasional expensive bill like that anyways.
deleted 5y ago
[deleted]
robinwarren 5y ago
It would also be software dev 101 for an organisation of AWS's size and clout to do some pretty basic things - If someone's costs increase drastically (ie spending in 24hrs what they spent in the last year or etc.) then maybe flag it up to them - If the above is combined with costs on services/regions they've not used, definitely maybe contact them? - If the above is combined with a well known exploit used on compromised accounts, again maybe contact them? Not saying the OP has no responsibility here. Just that I don't think it is that black and white. There is plenty AWS could do to avoid this sort of thing happening. Totally fair to call them out for that. Especially when the cost of phoning support is $2-3k specifically because of the new costs on your account!
jffry 5y ago
Blaming the victim for AWS having inadequate spending controls is kinda crummy. Absolute spending caps, or spending growth rate limits, would both be useful even outside of situations where a user's credential is compromised. One example: When teaching colleagues how to use AWS tooling, I gave them full access to a sandbox account where they can learn without disrupting anybody else's infra. It would be nice to be able to place a spending cap on that account. Another example: With things like autoscaling, it's very easy to accidentally misconfigure it to where it will happily start launching tons and tons of VMs. You might do this and not notice. If I had the ability to voluntarily opt into limits on how fast I can increase the aggregate $/second spend rate, then AWS could alert me in general when I'm doing something that exceeds that soft limit.
- tssva 5y ago
  Trying to turn this into him somehow being victimized by AWS and the original comment as being victim blaming in this regard is kind of crummy. The OP was a victim of the hacker and not AWS.
  - jffry 5y ago
    I did not say that this person was victimized by AWS, you are putting words in my mouth. We agree that this person has been victimized by a criminal. My point is that AWS failing to implement spending controls makes this type of attack significantly more damaging, and the slow turnaround on CloudWatch spending alarms means five or six-figure sums of money can be spent before customers are alerted. AWS can and should do more to give its customers additional control over spending as a defense-in-depth measure. This would be valuable both to limit unauthorized spending (this scenario, or a "rogue employee" scenario) and also unanticipated spending ("autoscaling gone wild" type scenarios). AWS is a big target, and the fact that credentials can be used to spend an unlimited quantity of money means they are a huge target for criminals. Defense in depth would be wise in this situation!
Tuna-Fish 5y ago
I expect them to allow me to set a limit on monthly spend, and cut my access to limit further costs when that is exceeded. Security needs to have depth. The one thing we have learned in these few decades of learning how to do online security is that any single measure can fail, and given enough scale, will fail. For anything to work, you need to be able to layer your security measures so that one failure, no matter how grave, will be fatal. AWS could add a very simple security measure here, making their service dramatically safer for their users, but they choose not to.
msh 5y ago
If I leave my credit card somewhere or the details gets stolen some how, so a thief gets it I do not have unlimited liability for the abuse of the card.
- PixelOfDeath 5y ago
  I'm very sorry Ms. or Mrs. msh. But the 3.4 trillion dollars have to be paid back by Monday! And take better care of your information next time. It is the duty of every customer to check cash dispenser for skimmers before using them! This is clearly written out in out general terms and conditions!

by atraac 5y ago

wccrawford 5y ago

They expect Amazon to allow people to set maximize spending amounts on their accounts. They have steadfastly refused to implement this.

KronisLV 5y ago

In my eyes, the situation is simple: human beings are fallible. If there's a chance of your credentials leaking and being exploited that way, that will most certainly happen to someone sooner or later. I recall a VPS that i had years ago being hijacked due to Docker socket being exposed publicly (when i set one up for testing and was just learning about Docker), which was mining crypto throughout the night before i noticed it. Imagine what would happen if there had been some sort of auto scaling in place and i had used a managed Kubernetes solution (many of which didn't yet exist back then)... Is it Amazon's problem? Not really, apart from the bad PR that charging private individuals exorbitant sums (for them) might cause, so sometimes these bills are waived after people mess up. For the corporation at the size of Amazon, that's probably not too much in the grand scheme of things, but it's also understandable why people might be appropriately upset about not being able to set hard spending limits, merely alerts (which aren't enabled by default). That's just the reality that we live in for now, however one can also think about why a corporation that cares a lot about uptime might not necessarily want to implement functionality that'd let resources be easily cut off when spend limits are hit or something similar, since most businesses that utilize their services might just swallow the occasional expensive bill like that anyways.

deleted 5y ago

[deleted]

robinwarren 5y ago

It would also be software dev 101 for an organisation of AWS's size and clout to do some pretty basic things - If someone's costs increase drastically (ie spending in 24hrs what they spent in the last year or etc.) then maybe flag it up to them - If the above is combined with costs on services/regions they've not used, definitely maybe contact them? - If the above is combined with a well known exploit used on compromised accounts, again maybe contact them? Not saying the OP has no responsibility here. Just that I don't think it is that black and white. There is plenty AWS could do to avoid this sort of thing happening. Totally fair to call them out for that. Especially when the cost of phoning support is $2-3k specifically because of the new costs on your account!

jffry 5y ago

Blaming the victim for AWS having inadequate spending controls is kinda crummy. Absolute spending caps, or spending growth rate limits, would both be useful even outside of situations where a user's credential is compromised. One example: When teaching colleagues how to use AWS tooling, I gave them full access to a sandbox account where they can learn without disrupting anybody else's infra. It would be nice to be able to place a spending cap on that account. Another example: With things like autoscaling, it's very easy to accidentally misconfigure it to where it will happily start launching tons and tons of VMs. You might do this and not notice. If I had the ability to voluntarily opt into limits on how fast I can increase the aggregate $/second spend rate, then AWS could alert me in general when I'm doing something that exceeds that soft limit.

tssva 5y ago
Trying to turn this into him somehow being victimized by AWS and the original comment as being victim blaming in this regard is kind of crummy. The OP was a victim of the hacker and not AWS.
- jffry 5y ago
  I did not say that this person was victimized by AWS, you are putting words in my mouth. We agree that this person has been victimized by a criminal. My point is that AWS failing to implement spending controls makes this type of attack significantly more damaging, and the slow turnaround on CloudWatch spending alarms means five or six-figure sums of money can be spent before customers are alerted. AWS can and should do more to give its customers additional control over spending as a defense-in-depth measure. This would be valuable both to limit unauthorized spending (this scenario, or a "rogue employee" scenario) and also unanticipated spending ("autoscaling gone wild" type scenarios). AWS is a big target, and the fact that credentials can be used to spend an unlimited quantity of money means they are a huge target for criminals. Defense in depth would be wise in this situation!

Tuna-Fish 5y ago

I expect them to allow me to set a limit on monthly spend, and cut my access to limit further costs when that is exceeded. Security needs to have depth. The one thing we have learned in these few decades of learning how to do online security is that any single measure can fail, and given enough scale, will fail. For anything to work, you need to be able to layer your security measures so that one failure, no matter how grave, will be fatal. AWS could add a very simple security measure here, making their service dramatically safer for their users, but they choose not to.

msh 5y ago

If I leave my credit card somewhere or the details gets stolen some how, so a thief gets it I do not have unlimited liability for the abuse of the card.

PixelOfDeath 5y ago
I'm very sorry Ms. or Mrs. msh. But the 3.4 trillion dollars have to be paid back by Monday! And take better care of your information next time. It is the duty of every customer to check cash dispenser for skimmers before using them! This is clearly written out in out general terms and conditions!