4 ms·

I posted to the AWS forum and accidentally copy/pasted my secret key. Within 24 hours $11k of charges. I called them, they wiped them. It's amazing how quick

by e40 11y ago

I posted to the AWS forum and accidentally copy/pasted my secret key. Within 24 hours $11k of charges. I called them, they wiped them. It's amazing how quickly people find and use these things. What kills me is there is no easy way to stop all the instances for an account, in a region. It took me hours to kill all the instances. They had maxed out the number of instances in every single region. Very, very annoying.

pests 11y ago
You can automate the stopping via the CLI interface. While I don't think there is a single command to stop all instances, you should be able to whip up a script to get all id's then call the stop-instances command on each of them.
agumonkey 11y ago
You must lose a fair amount of water when you're greeted with a surprise bill of $11k.
- e40 11y ago
  Yeah, it's a rare moment in life. Especially fun since it happened on a Saturday morning (the notification, that is).
copperx 11y ago
When people find the accounts, what do they use them for? Mining bitcoin?
mrestko 11y ago
Any idea what they were doing with them? Mining bitcoins? Hosting CnC for a botnet?
- fpgaminer 11y ago
  It's always Bitcoin mining. The attacker spins up a bunch of GPU EC2 instances and mines as long as they can. I don't think the profit ends up being very large (those GPU instances put out a puny amount of hashing power compared to modern mining ASICs)...
  - icelancer 11y ago
    ROI for BTC mining on standard AWS pricing is about -90%. So attackers get 10% free BTC on spend.
    - tajen 11y ago
      So, out of $11k, they make $1k. That's the kind of money they need to make between once and three times a month minimum depending on where they live. Doesn't Amazon notice patterns in terms of source, scripts which are uploaded and scaling profile: Who uses 2 medium instances for a year then spins off 2,000 in 20 minutes?
      - icelancer 11y ago
        Certainly there's a pattern there, but it's not THAT far away from people who intermittantly scale stuff for short bursts of huge processing. False positives in those cases for people that really intended to spend $10-50k in a short time might mean a HUGE loss of revenue and/or incurred customer support and service costs.
  - Taek 11y ago
    It's not that bad. The smart attackers don't do Bitcoin mining, they do altcoin mining, and they pick altcoins that don't have ASICs out. Then they are competing with other GPUs. I'm not sure what the profit is, but I'd guess it's between 30% and 70% of the bill incurred.

by e40 11y ago

pests 11y ago

You can automate the stopping via the CLI interface. While I don't think there is a single command to stop all instances, you should be able to whip up a script to get all id's then call the stop-instances command on each of them.

agumonkey 11y ago

You must lose a fair amount of water when you're greeted with a surprise bill of $11k.

e40 11y ago
Yeah, it's a rare moment in life. Especially fun since it happened on a Saturday morning (the notification, that is).

copperx 11y ago

When people find the accounts, what do they use them for? Mining bitcoin?

mrestko 11y ago

Any idea what they were doing with them? Mining bitcoins? Hosting CnC for a botnet?

fpgaminer 11y ago
It's always Bitcoin mining. The attacker spins up a bunch of GPU EC2 instances and mines as long as they can. I don't think the profit ends up being very large (those GPU instances put out a puny amount of hashing power compared to modern mining ASICs)...
- icelancer 11y ago
  ROI for BTC mining on standard AWS pricing is about -90%. So attackers get 10% free BTC on spend.
  - tajen 11y ago
    So, out of $11k, they make $1k. That's the kind of money they need to make between once and three times a month minimum depending on where they live. Doesn't Amazon notice patterns in terms of source, scripts which are uploaded and scaling profile: Who uses 2 medium instances for a year then spins off 2,000 in 20 minutes?
    - icelancer 11y ago
      Certainly there's a pattern there, but it's not THAT far away from people who intermittantly scale stuff for short bursts of huge processing. False positives in those cases for people that really intended to spend $10-50k in a short time might mean a HUGE loss of revenue and/or incurred customer support and service costs.
- Taek 11y ago
  It's not that bad. The smart attackers don't do Bitcoin mining, they do altcoin mining, and they pick altcoins that don't have ASICs out. Then they are competing with other GPUs. I'm not sure what the profit is, but I'd guess it's between 30% and 70% of the bill incurred.