7 ms·

Reading this response it seems that crypto-mining is not allowed on digital ocean as they have checks against it. The TOS doesn't say so explicitly but does not

Reading this response it seems that crypto-mining is not allowed on digital ocean as they have checks against it. The TOS doesn't say so explicitly but does note that: >violation of any of these Terms of Service or any law, or if you misuse system resources, such as, by employing programs that consume excessive network capacity, CPU cycles, or disk IO By my reading that seems to mean that you're not allowed to use your VMs to their full capacity due to them being over-provisioned. This is in contrast to AWS who are more explicit on which instances (T instances) are over-provisioned and exactly how they're throttled.

xvf22 7y ago
I don't think that it isn't allowed but that they have seen that fraudulently acquired resources are generally used for crypto-mining so they felt that this was a good signal to look for.
- duskwuff 7y ago
  Right. Reading between the lines a bit, it's not the activity itself that DO is worried about, but a pattern of usage that suggests that the account may have been created fraudulently or compromised.
  - bcooks 7y ago
    This is correct. This was the primary thing we were attempting to solve for in this case and the bug in the algorithm started the chain of events documented in the postmortem.
jimmaswell 7y ago
Anecdotally, I had a python script unintentionally using 100% for a month or two and never heard about it or noticed until I happened to look at top.
- xnxn 7y ago
  Counter-anecdote slash venting: I recently logged into my long-dormant DO account to kick the tires on their now-GA managed Kubernetes service and to contribute some CPU cycles to a distributed computing project I'm interested in (not cryptocurrency, I swear). I first requested and was approved for a droplet limit increase (to 25). I started 20 nodes, deployed my very CPU intensive workload, and 12 hours later my account was locked and my nodes all went NotReady. I immediately replied to the abuse ticket explaining my usage. 4 days later, my account was unlocked and received the "allow high CPU" flag... but they billed me for the nodes as if they had been running that whole time. I asked for a credit (5 days ago) and they haven't replied yet. Probably a little busy over there right now. So... I'm not too thrilled with DO. I get that cryptocurrency ruined everything, but this has been a frustrating experience and I'm glad I was only using it for a pet project.
  - zacharybk 7y ago
    Hey xnxn - Thanks for raising this issue. I'm Zach, Director of Support at DO. Please send me an email with your account detalis, first name at, and I'll take care of this.
    - xnxn 7y ago
      Thanks Zach, will do.
      - zacharybk 7y ago
        Email received, followed up on, and account credited. -Zach
        chris_wot 7y ago
        Really? What is so bad about your support that you needed to hear about this from HN before you did anything?
        7y ago
bcooks 7y ago
If you want to do cryptocurrency mining on DO that is actually okay with us. Some of the other respondents are correct the behavior we were looking for was really around fraudulent accounts being created and performing cryptocurrency mining. This is why the trigger that flagged this account was using payment history as a key factor in the triggering.
- chris_wot 7y ago
  Your post-mortem implies this is not allowed at all.
  - rajaganesh87 7y ago
    Blog post did mention that accounts with high CPU usage and payment history won't be flagged.
  - Johnny555 7y ago
    Your post-mortem implies this is not allowed at all. Not sure why you were downvoted, I had the same impression, after reading: ...an automated service that monitors for cryptocurrency mining activity (Droplet CPU loads and Droplet create behaviors). These signals, coupled with a number of account-level signals (including payment history and current run rate compared to total payments) are used to determine if automated action is warranted to minimize the impact of potential fraudulent high-cpu-loads on other customers This sounds like they don't permit extended high CPU loads due to the impact it can have on other customers.
    - fhars 7y ago
      No, that quite clearly states that they treat high CPU loads as suspect on accounts without an established good payment history or if it significantly deviates from previous usage patterns.
    - bgirard 7y ago
      The keyword here is 'fraudulent'. High-cpu-loads is allowed, but an automated service monitors for fraudulent activity.
Shoop 7y ago
I believe the purpose of these checks is for when people's outdated wordpress install or whatever gets compromised by script kiddies. Generally, the scripts install crypto miners to mine for the hacker until your account gets shut down, running you up a huge bill in the process.
- treis 7y ago
  I think it's moreso people continually creating new accounts to get free credits. Then bitcoin mining turns the credits into actual money.
ac29 7y ago
Here was my key takeaway: "Cryptocurrency mining mitigation detects suspicious behavior, including very high CPU utilization on an account with no payment history, which results in an account lock" Lots might have been done wrong here, but it sounds like they had an account with trial or promotional credits - I can see how this could easily be abused.
- phire 7y ago
  Sounds like it's designed to counter stolen credit cards. An attacker might load a stolen credit card number into an account and only use enough resources to generate a few dollars worth of billing. The owner of the credit card might not notice the small charge. Then after a few months of low billing (to bypass a previous heuristic), they ramp up the utilization, mine a bunch of coins then the holder gets a massive bill. The holder does a charge back and DO is left holding the bill.
  - kijin 7y ago
    It's also designed to keep everyone relatively happy in a shared-CPU environment. "Standard" droplets share CPU with others on the same node, so one droplet pegging the CPU 24/7 can be problematic. AWS doesn't have this problem because either your instance is allowed to use all the CPU that's allicated to it, or else (t2 & t3) the platform will automatically limit your CPU usage. You don't have to care about how your usage affects other people. It's one more thing that AWS abstracts away. DO's abstraction, on the other hand, is rather leaky in this area. That's a problem in and of itself, in addition to the matter of credit card fraud that every company has to deal with.
- mdip 7y ago
  Completely. At the same time, those promotional credits are going to be used by guys like me who will have to decide if their services are worth having to spend an extended period of time explaining why "we're not recommending AWS/Azure/Goober" An account shutdown, or enough complaints from verifiable sources, and I'm not going to the trouble. Not to pull out an old trope "Nobody got fired for picking IBM". But that's the case: pulls a move like this and the entirety of the customer, who likely came in with AWS in mind (in some cases, was advised against it and insisted on it) is going to shrug their shoulders. Pick a provider that the customer hasn't heard of and I'm going to get a phone call that goes something like "You're the one that said we should use that basement-operation!" with raised voices. Heck, the last time there was an Azure outage, we didn't hear from most of our customers. It was so impacting that even customers well outside of software development/technology read news articles and connected the dots. I had one customer tell me he thought it was just their corporate internet connection; they assumed it was working[1]. [0] Plus, too lazy to put in the research; sorry. [1] They were a customer who insisted on doing the app monitoring, themselves -- that guy was getting the alerts and similarly assumed it was the network since that happened regularly with another application they developed -- the monitoring server was on-prem.
Twirrim 7y ago
Cryptocurrency mining is a favourite of fraudsters. Get a fake or stolen credit card number / identity, sign up for an account on a cloud provider, and spin up instances to do your mining. Depending on how quickly the provider reacts to the pattern of behaviour, or identifies the account as fraudulent etc, you can have earned a reasonable pay-off. If the provider isn't paying attention and/or doesn't have good fraud detection in place, it may be a few months before your account and resources are terminated (assuming the payments eventually bounce, and the provider gives you a chance to fix it)

xvf22 7y ago

I don't think that it isn't allowed but that they have seen that fraudulently acquired resources are generally used for crypto-mining so they felt that this was a good signal to look for.

duskwuff 7y ago
Right. Reading between the lines a bit, it's not the activity itself that DO is worried about, but a pattern of usage that suggests that the account may have been created fraudulently or compromised.
- bcooks 7y ago
  This is correct. This was the primary thing we were attempting to solve for in this case and the bug in the algorithm started the chain of events documented in the postmortem.

jimmaswell 7y ago

Anecdotally, I had a python script unintentionally using 100% for a month or two and never heard about it or noticed until I happened to look at top.

xnxn 7y ago
Counter-anecdote slash venting: I recently logged into my long-dormant DO account to kick the tires on their now-GA managed Kubernetes service and to contribute some CPU cycles to a distributed computing project I'm interested in (not cryptocurrency, I swear). I first requested and was approved for a droplet limit increase (to 25). I started 20 nodes, deployed my very CPU intensive workload, and 12 hours later my account was locked and my nodes all went NotReady. I immediately replied to the abuse ticket explaining my usage. 4 days later, my account was unlocked and received the "allow high CPU" flag... but they billed me for the nodes as if they had been running that whole time. I asked for a credit (5 days ago) and they haven't replied yet. Probably a little busy over there right now. So... I'm not too thrilled with DO. I get that cryptocurrency ruined everything, but this has been a frustrating experience and I'm glad I was only using it for a pet project.
- zacharybk 7y ago
  Hey xnxn - Thanks for raising this issue. I'm Zach, Director of Support at DO. Please send me an email with your account detalis, first name at, and I'll take care of this.
  - xnxn 7y ago
    Thanks Zach, will do.
    - zacharybk 7y ago
      Email received, followed up on, and account credited. -Zach
      - chris_wot 7y ago
        Really? What is so bad about your support that you needed to hear about this from HN before you did anything?
        7y ago

bcooks 7y ago

If you want to do cryptocurrency mining on DO that is actually okay with us. Some of the other respondents are correct the behavior we were looking for was really around fraudulent accounts being created and performing cryptocurrency mining. This is why the trigger that flagged this account was using payment history as a key factor in the triggering.

chris_wot 7y ago
Your post-mortem implies this is not allowed at all.
- rajaganesh87 7y ago
  Blog post did mention that accounts with high CPU usage and payment history won't be flagged.
- Johnny555 7y ago
  Your post-mortem implies this is not allowed at all. Not sure why you were downvoted, I had the same impression, after reading: ...an automated service that monitors for cryptocurrency mining activity (Droplet CPU loads and Droplet create behaviors). These signals, coupled with a number of account-level signals (including payment history and current run rate compared to total payments) are used to determine if automated action is warranted to minimize the impact of potential fraudulent high-cpu-loads on other customers This sounds like they don't permit extended high CPU loads due to the impact it can have on other customers.
  - fhars 7y ago
    No, that quite clearly states that they treat high CPU loads as suspect on accounts without an established good payment history or if it significantly deviates from previous usage patterns.
  - bgirard 7y ago
    The keyword here is 'fraudulent'. High-cpu-loads is allowed, but an automated service monitors for fraudulent activity.

Shoop 7y ago

I believe the purpose of these checks is for when people's outdated wordpress install or whatever gets compromised by script kiddies. Generally, the scripts install crypto miners to mine for the hacker until your account gets shut down, running you up a huge bill in the process.

treis 7y ago
I think it's moreso people continually creating new accounts to get free credits. Then bitcoin mining turns the credits into actual money.

ac29 7y ago

Here was my key takeaway: "Cryptocurrency mining mitigation detects suspicious behavior, including very high CPU utilization on an account with no payment history, which results in an account lock" Lots might have been done wrong here, but it sounds like they had an account with trial or promotional credits - I can see how this could easily be abused.

phire 7y ago
Sounds like it's designed to counter stolen credit cards. An attacker might load a stolen credit card number into an account and only use enough resources to generate a few dollars worth of billing. The owner of the credit card might not notice the small charge. Then after a few months of low billing (to bypass a previous heuristic), they ramp up the utilization, mine a bunch of coins then the holder gets a massive bill. The holder does a charge back and DO is left holding the bill.
- kijin 7y ago
  It's also designed to keep everyone relatively happy in a shared-CPU environment. "Standard" droplets share CPU with others on the same node, so one droplet pegging the CPU 24/7 can be problematic. AWS doesn't have this problem because either your instance is allowed to use all the CPU that's allicated to it, or else (t2 & t3) the platform will automatically limit your CPU usage. You don't have to care about how your usage affects other people. It's one more thing that AWS abstracts away. DO's abstraction, on the other hand, is rather leaky in this area. That's a problem in and of itself, in addition to the matter of credit card fraud that every company has to deal with.
mdip 7y ago
Completely. At the same time, those promotional credits are going to be used by guys like me who will have to decide if their services are worth having to spend an extended period of time explaining why "we're not recommending AWS/Azure/Goober" An account shutdown, or enough complaints from verifiable sources, and I'm not going to the trouble. Not to pull out an old trope "Nobody got fired for picking IBM". But that's the case: pulls a move like this and the entirety of the customer, who likely came in with AWS in mind (in some cases, was advised against it and insisted on it) is going to shrug their shoulders. Pick a provider that the customer hasn't heard of and I'm going to get a phone call that goes something like "You're the one that said we should use that basement-operation!" with raised voices. Heck, the last time there was an Azure outage, we didn't hear from most of our customers. It was so impacting that even customers well outside of software development/technology read news articles and connected the dots. I had one customer tell me he thought it was just their corporate internet connection; they assumed it was working[1]. [0] Plus, too lazy to put in the research; sorry. [1] They were a customer who insisted on doing the app monitoring, themselves -- that guy was getting the alerts and similarly assumed it was the network since that happened regularly with another application they developed -- the monitoring server was on-prem.

Twirrim 7y ago

Cryptocurrency mining is a favourite of fraudsters. Get a fake or stolen credit card number / identity, sign up for an account on a cloud provider, and spin up instances to do your mining. Depending on how quickly the provider reacts to the pattern of behaviour, or identifies the account as fraudulent etc, you can have earned a reasonable pay-off. If the provider isn't paying attention and/or doesn't have good fraud detection in place, it may be a few months before your account and resources are terminated (assuming the payments eventually bounce, and the provider gives you a chance to fix it)