12 ms·
My run-in with unauthorised Litecoin mining on AWS
- meritt 13y agoUgh, that sucks. Too late to help you now (but perhaps others) on your billing alerts points: check out http://cloudability.com http://cloudability.com -- alerts, analytics, prediction, suggestions, etc. Free for the most useful stuff.
- sillysaurus2 13y agoIs this illegal? Could he somehow go to some authority? EDIT: Why is it unlikely the FBI will successfully investigate?
- patio11 13y agoAsk a simple question, get a simple answer: it is obviously and unambiguously illegal. You can certainly refer this to the FBI computer crimes folks and your local law enforcement. It is unlikely they will successfully investigate.
- vertis 13y agoYeah, I'll discuss this with Amazon when they get back to me, but I don't hold much hope for a positive outcome.
- rplnt 13y agoWhat specifically is illegal about that? Someone gave you credentials to an online service. Now you use those credentials. What law did you brake? You probably broke the ToS/EULA/whatever, but that by itself is not illegal. (I'm not saying otherwise, I'm asking)
- computer 13y agoFor the US, pretty much anything is covered under https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act
- freehunter 13y agoJust because you have the means to do something does not mean you have the permission. If I drop my car key on the ground and you take it, it's still stealing. I gave you the means (accidentally), not the permission. Especially when using the ill-gotten item will cost me (gas, time, money). Reminds me of the old phrase: What's the difference between hacking and penetration testing? Permission.
- rplnt 13y agoYes, I see that difference. I also know there are laws protecting computer systems and such. But was wondering if it could be applied to an online service.
- stefan_kendall 13y agoPossibly because it would cost the FBI orders of magnitude more than the damages to investigate. The damages which amazon will likely refund. The FBI does not target petty thieves.
- richo 13y agoAmusingly, I went to a meetup with some feds the other day. Their statutory minimum to investigate is 5k, but they suggested it's worth getting in touch with them either way. If nothing else, it'll give them an excuse to say "cyber" a few more times.
- fsckin 13y agoThis. I was defrauded to the tune of $3,500 (run of the mill stolen Paypal account bought a top of the line iMac from me on eBay) and after discovering to my horror that both eBay and Paypal sidestep all fraud for protection... I called the FBI, police, etc. They listened politely until I said how much it was worth -- one of my favorite quotes was "If you added a zero to the end, we'd knock down their door at 6AM tomorrow. For $3,500 it's simply not worth our time."
- jellicle 13y agoOh, they certainly do, if powerful enough people complain. http://en.wikipedia.org/wiki/Aaron_Swartz http://en.wikipedia.org/wiki/Aaron_Swartz The question is whether the poster is such a person.
- negamax 13y agoI am not sure but FBI can only be involved on financial frauds over 1 million (I vaguely remember this and could be wrong)
- tptacek 13y agoPretty sure you're wrong about that.
- billclerico 13y agostatutory minimum is 5k. we work with federal law enforcement somewhat frequently at wepay to deter fraud, and it appears that the informal threshold to get their interest is about 100k of damages
- meowface 13y agoIllegal? Yes. Convictable? Yes. But will any law enforcement department or agency actually look into it? Highly unlikely. There's so much major scale white collar crime/cybercrime going on daily that it's generally too hard to dedicate resources to smaller cases like this. Now, if they got evidence the perpetrator was doing this with dozens or hundreds of AWS accounts, that would be another story.
- dhughes 13y agoNow I'm curious, how many litecoins would it have generated in two days?
- vertis 13y agoWish I knew.
- dwiel 13y agoLess than the cost of the ec2 instances, though that is still a wide window
- deleted 13y ago[deleted]
- gigq 13y agoOne cc2.8xlarge instance will give you 85 kH/s so 20 instances would give 1700 kH/s which only nets about $18 at current market prices / difficulty. So over 2 days he would have made off with a whopping $36.
- rkuykendall-com 13y agoWriting a script that generated $38 every time somebody accidentally commits a AWS key sounds like an amazing source of extra cash, depending on your current income.
- nisa 13y agoIf he would have used an altcoin that is limited by CPU like Primecoin he could have earned around 8000$ in 2days. Not sure if my calculation is legit through. It's based on the chains/day from the list at http://anty.info/primecoin-calculator/ http://anty.info/primecoin-calculator/ Scary. I'm sure a lot of universities and servers will see an influx of hacks for coin-mining.
- delinka 13y agoor "...with unauthorized account usage on AWS." I get that the unauthorized use was mining, but the mining operation itself isn't unauthorized by Amazon nor by the creator of the currency.
- vertis 13y agoSure guess that could have been clearer. Unauthorized account usage used for mining litecoins.
- trapexit 13y agoYou can (and should) set up an AWS CloudWatch alert on your account that will send you an email or SMS notification when your monthly bill exceeds a set threshold.
- vertis 13y agoYeah, I've done that now, but it's 'after the horse has bolted'
- zwass 13y agoI was surprised how incredibly difficult that is to set up. Eventually I dead-ended following the instructions when CloudWatch told me there were 0 metrics to choose from for monitoring...
- sharpy 13y agoHi, There are a couple of things that are easy to overlook when using billing metrics. 1. All billing metrics are stored in us-east-1 even for usages in other regions. 2. If you are using consolidated billing, billing metrics will be published under the linked account, and will only be visible to that account. Hope that helps.
- chimeracoder 13y ago> 1. All billing metrics are stored in us-east-1 even for usages in other regions. ...what? Is this true? If so, can someone explain the logic behind this?
- debaserab2 13y agoI wonder if the author is going to be on the hook for the bill for this. If he originally received this note from amazon, it makes me also wonder if amazon knew about the fraud while it was happening. I sense that they probably monitor the launch of many of the XXL servers more closely than others.
- vertis 13y agoI have a support ticket open with Amazon about this. I'm hoping not, but I have to accept a certain amount of responsibility for leaking the key in the first place.
- res0nat0r 13y agoI highly doubt you will have to pay this large bill, they most likely will reverse the charges and you should be free and clear, especially if you detail what you found and that you also have an email from them that your key was exposed. I for sure would go to your billing report on the AWS console and setup alerts. You can have AWS notify you automatically if your bill goes above any threshold you want to define.
- snowpalmer 13y agoI had the exact same thing happen to me. 2 days of 20 instances running and then amazon called me. I shut it down and revoked permissions immediately. They did refund the money.
- vertis 13y agoYeah, I'd say that anyone on Github that has somehow leaked keys is in for a nasty surprise (or has already received one)
- icpmacdo 13y agoI just checked my EC2 page and have found $45 worth of charges from instances that I shut down weeks ago. Would it have automatically re ordered some if the bidding price dropped below a threshold? When I looked at my management console there were no EC2 instances running. This has turned out to be a very expensive experiment in LTC mining.
- umairsiddique 13y agoExactly same thing happened to me. 20 x xlarge instances raking up a total bill of $1800. I've opened a support case with them.
- sheetjs 13y ago> Audit code before open sourcing It's important to remember that open-sourcing is generally one-way: once it's out there, it's impossible to completely eliminate all traces. Always audit code, and if there's even a remote possibility that you'll regret it you should check again
- lancefisher 13y agoI'd recommend blowing away your repository's history before open sourcing as well.
- DanWaterworth 13y agoIf you decide to do this, using git replace so that you and other trusted people can see the history, but others can't is a particularly good idea.
- acangiano 13y agoCPU mining of scrypt-based cryptocurrency is highly inefficient. Let's do some math. A cc2.8xlarge is reported to mine at 85 kh/s, so 20 of them would give you 1700 kh/s. That's roughly equivalent to a couple of high-end AMD GPUs (say a couple of overclocked 290x). This hashing power gives you a little over 0.5 LTC per day. It mined for two days, so it gained a little over 1 LTC. Let's call it $40. That's right, the idiot behind this cost the OP $3000+ for $40 profit. A smarter criminal would have spawn GPU instances on EC2.
- vertis 13y agosigh I was afraid it was THAT stupid.
- fat0wl 13y agoit almost always is, with the crypto-currency crew. unbridled greed is the driving force of the movement now
- gigq 13y agoMakes you wonder why the criminal didn't launch g2.2xlarge instances which would get 185 kH/s per instance. In fact for awhile there litecoin mining was profitable at spot prices. http://www.completefusion.com/profitable-litecoin-mining-on-ec2/ http://www.completefusion.com/profitable-litecoin-mining-on-...
- acangiano 13y agoMy guess would be ignorance. I don't think the criminal behind this knows what he is doing. A smarter criminal would have opted for g2.2xlarge instances as well as mining for a currently more profitable coin. Granted he'd need to be careful not to leave a trail, he could essentially trade these coins for LTC and still obtain more litecoins.
- gigq 13y agoAn even smarter criminal would have "mined" the Ripple give away using EC2, there are still people paying for those instances as we speak so surely they would be more profitable than litecoin mining (assuming you sell them right away). https://www.computingforgood.org/ https://www.computingforgood.org/ https://ripple.com/forum/viewtopic.php?f=18&t=4382 https://ripple.com/forum/viewtopic.php?f=18&t=4382
- billjive 13y agoHow did Amazon detect your key in the wild? Or did they notice based on usage patterns/activity in your instances?
- earless1 13y agoAll AWS keys I've seen start with 'AKIA'. I am assuming that they have bots that search Github and other search engines for access keys. At that point it is easy for them to tie them back to an account and notify the user.
- vertis 13y agoThey must only have started doing that recently. This project has been out in the wild for at least a year.
- enko 13y agoWell, kudos to them for doing that, at least. Of course it's awful that you could be out ~$3k, but imagine how bad it could have been if they hadn't been so proactive.
- devonbleak 13y agoFYI all AWS keys start with AKIA - makes it easy to search for 'em.
- awhitty 13y agoShoot, as someone who made the same mistake of leaving my AWS keys in an open source project, I think I narrowly dodged a bullet. I didn't realize this risk was so high. Thanks for this post!
- earless1 13y agoI think smarter usage of IAM roles would have also helped here. Keys created strictly for S3 access should not have the ability to launch new instances and so on. Limiting keys to their specific purpose is a good security practice even for dev environments.
- sounds 13y agoRole-based access control. :) Though Amazon still has services that "don't support IAM" ... I'm looking squarely at Payments. That particular master key always makes me nervous.
- miles932 13y agoAWS IAM supports the ability to permit or prevent specific types of instances from being started by a given key; if folks are worried about a key being used to start G2 or CG1 or any other specific instance, take a look at the instructions here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-policies-for-amazon-ec2.html http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-polic...
- colbyaley 13y agoI suggest the OP check out Cloudability[1], which provides realtime cost management for AWS and other cloud providers. We help over 10,000 customers make sure this doesn't happen to them. (disclosure: I work there) [1]: https://cloudability.com/ https://cloudability.com/
- lambda 13y agoAnother good habit to be in is never checking any kind of credentials into source control; even if it's some private personal project, just don't be tempted to check in your credentials to source control, because at some point you may find some portion of that that's useful that you import into a public project, accidentally preserving full history. Sorry to the OP, hope that Amazon reverses those charges once you tell them what happened.
- xur17 13y agoWhat's the best way to handle api keys without storing them in a repository? A separate configuration file? I've saved api keys to a repository, but only ones with no payment information attached and read only access. Mainly because I couldn't think of a better way at the time.
- grannyg00se 13y agoUsally in environment variables rather than a config file. for example https://devcenter.heroku.com/articles/config-vars https://devcenter.heroku.com/articles/config-vars
- chaz 13y agoThis is very convenient on Heroku, but on my own servers, I have to use a post-deploy hook to copy a file that has all of my API keys to the server. That works great, except I now have a separate file to manage outside of source control, it's not under its own source control, and it's difficult to share amongst my team. Anyone have any suggestions?
- ketralnis 13y agoI've had some luck with having a separate repo. I'll have a "public" (even if it's not public) with all of the usual source code, and a "private" repo, with passwords, fabric files, run scripts, etc. Works well on teams, where the subset of developers that don't need to do actual deployments don't need the private repo anyway.
- badmadrad 13y ago"Having a poke around confirmed what I had already guessed. The unauthorised user had been mining litecoin with the mining pool pool-x.eu." Hmmm..you already guessed someone hacked your account to mine litecoin? Astroturfing much? That's the last thing I would have guessed. I would have thought someone was using it as some crazy web server or mail server to generate spam or phony websites for bogus ad clicks.
- vertis 13y agoshrug maybe I was wrong to make that kind of assumption. It was probably the type and quantity of instance that tipped me off a little, having read about people trying to mine with EC2 again.
- omarchowdhury 13y agoSo are you liable?
- tomphoolery 13y agoI know there are a few code-quality bots on Github, but is there any service that you can install as a webhook which automatically checks for things like Amazon key pairs (which, IIRC, always start with "AKIA", at least the API keys anyway)?
- tlrobinson 13y agoI'm also curious if there's some utility or at least a list of regular expressions that can be used to scan for a number of credentials, not just AWS.
- mnml_ 13y agoAmazon will refund you if you explain your situation.
- davidjgraph 13y agoThis is rough luck, but getting specific servers hacked is more commonplace. In the AWS billing console [0] there is an "alert" option. It walks you through setting up the various types of alarms. If you're hacked the most likely problem you'll get is a spike in data transfer costs. You can up the alarms to, for example, email you if the bandwidth usage goes above x (cost) over y time period. I had a perl DOS bot get into a server, took about 2 hours to trigger the alarm. Shame I was fast asleep at the time, but the idea was there... [0] https://console.aws.amazon.com/billing/home https://console.aws.amazon.com/billing/home
- judah 13y agoThe author suggested enabling billing alerts. For those running on Azure, billing alerts are currently in preview mode, and can be enabled via https://account.windowsazure.com/PreviewFeatures https://account.windowsazure.com/PreviewFeatures
- smiro2000 13y agosorry dude :(