3 ms·

Digital Ocean – Shut Down – Zero Notice Without Reason

11 points by allendking 5y ago

allendking 5y ago
Last night Digital Ocean (https://www.digitalocean.com https://www.digitalocean.com) completely SHUT DOWN our account asking for verification (automated email) with ZERO warning. They wrote that they wanted to understand our usage and asked to explain our workload (automated email), but WHY shut down without notice? We are live with users and paying customers! How can DigitalOcean.com justify shutting down our servers WITHOUT ANY notice? Ethical? We think not! We explained that we have a SAAS platform and company based in California that we have built and are ready to launch our new version. After providing all the documentation, we got an email from an outsourced low-level "specialist" basically screwing us over even further. Sorry, we can't give any REASON but go host somewhere else (to paraphrase). Oh, and by the way, our team has ZERO ACCESS to our database and servers. How do you propose we move Mr. Specialist? STARTUP Founders - may we suggest you AVOID company Digital Ocean! We signed up through the Stripe Atlas program and applied Hatch at the beginning of the month. We were stoked to have customers and launch quickly. They completely screwed us, and it's unclear if we'll ever get our database back. We asked them to escalate the matter to a supervisor, pleading they give us a few days to move over to another company and retrieve our database. Still no word. Luckily, we have a backup from a week ago, but we'll still lose hundreds of transactions and thousands dollars if they don't give us back access. Our company will be reporting DigitalOcean to the BBB and potentially filing lawsuits for damages because this type of behavior is unethical and unprofessional (from our point of view). This action from Digital-Ocean is costing us money and time during a critical launch period. Fellow start-up founders AVOID the headache. On the flip side, we've added "Disaster Relief Plan" into a future sprint, so once we figure out how to recover from this, we'll be better prepared for situations with unscrupulous hosting companies that don't value their customers.
- akhmatova 5y ago
  We asked them to escalate the matter to a supervisor, pleading they give us a few days to move over to another company and retrieve our database. Still no word. Unannounced shutdowns are one thing, but then ghosting the customer and simply refusing to answer their inquiries is another. I'd like to know if Digital Ocean has another side to this story, but I don't have high hopes that they will. Best of luck, and thanks for sharing your experience. Luckily, we have a backup from a week ago, but we'll still lose hundreds of transactions and thousands dollars if they don't give us back access. It isn't clear here if the loss of transactions is from the time window between the last backup and the servers being shut down, or from loss of business after the shutdown. If it's the former, then ... at least we know that isn't Digital Ocean's fault.
  - allendking 5y ago
    We'd love for Digital Ocean to have an excellent customer service response. Rebuilding our hosting infrastructure on a new provider may take a few days. Tomorrow, we'll have our Lead Developer start moving onto a new hosting platform while our full-time team of nine (9) developers enjoys a miny unplanned paid vacation. We signed up for the digital backup service that Digital Ocean offers, but it does no good if you can't access those backups. Luckily, we have a manual backup from a week ago. Unfortunately, we'll need to manually figure out what is missing, which will require directly calling customers via the Stripe receipts. Dear customers, what did you sign up for again? We're sorry, digitalocean.com has... and we need to match your payments receipts and manually create new records for profiles, credits, and reservations (only a few hundred phone calls). Moreover, payments, bookings, and reservations have completely been halted due to the downtime imposed by Digital Ocean. Existing and potential customers are unable to process transactions. Maybe new customers will wait, or perhaps they'll Google another provider that can make a reservation now (who likes waiting).
- db48x 5y ago
  The BBB is toothless; don’t bother. File a lawsuit.
  - allendking 5y ago
    Thanks for the recommendation, and that does make sense. We don't wish to litigate this matter because launching a start-up takes a lot of time and resources. Unfortunately, if the financial losses continue to accrue, we'll need to pursue legal options for equitable remuneration. For now, we remain optimistic that a supervisor will quickly see their mistake and fix the situation.
    - db48x 5y ago
      That is true, but if nobody ever files a lawsuit then companies will just continue to run roughshod over everybody knowing that there will be no consequences.
    - thfuran 5y ago
      It costs very little to get a lawyer to draft an angry letter and it certainly won't slow down the escalation process.
allendking 5y ago
Account Reactivated - DAMAGE ALREADY DONE The trigger happy outsourced customer service representative decided to reactivate our account instead of escalating the matter (per our request) to a supervisor. We will ask again for the issue to be escalated to a supervisor so that Digital Ocean can fix this from happening to future customers. We are currently planning on moving to a new hosting company and began the process yesterday. Excellent customer service and professionalism are vital for the stability of our SAAS as we move to scale. Unfortunately, we can't recommend digitalocean.com based on this experience. In the comments below are the emails from DigitalOcean (Caveat Emptor). I appreciate the support from this community. Hopefully, this information will be helpful to future start-up founders.
- allendking 5y ago
  Hi there, We're excited to have you as a customer of DigitalOcean and we would like to verify your account. Because your recent account activity is atypical, we’d like to better understand your product needs. To help us in restoring access to the account, please respond to this email message with a description of the workload you’ve deployed on your Droplets and how you plan to use your account going forward. We appreciate your understanding and look forward to better understanding how DigitalOcean can support your needs! Regards, DigitalOcean Security ref:_00Df218t5m._5004P1qDp3y:ref
  - deleted 5y ago
    [deleted]
- allendking 5y ago
  Hello there, Thank you so much for your patience, as we investigated this incident and reviewed the information you have shared with us so far, we have come to the conclusion that your deployment and actions were in violation of our Terms of Service Agreement [1] and Acceptable Use Policy [2] and as a result, we are unable to remove the lock from your account. I sincerely apologize for any inconvenience caused by this. Additionally, please note that we're unable to share details regarding what factors we considered that led to this decision or how we came to this conclusion as this is critical to maintaining the integrity of our platform security operations. You can review our Terms of Service agreement and Acceptable Usage policy here [1] Terms of Agreement: Legal - Terms of Service Agreement (digitalocean.com) [2] Acceptable Usage Policy: Legal - Acceptable Use Policy (digitalocean.com) We recommend you find a new service provider that better suits your needs as our security tooling would consider these actions when you sign up again resulting in account locks and termination of linked accounts during manual platform-wide reviews. If you still feel this is in error we request you to share detailed information about you, your business, and your deployment with us so that we can take your explanation into further consideration and contextually analyze it with information we have on file with us to review this decision. Regards, Nouroze Developer Experience Specialist ref:_00Df218t5m._5004P1qDp3y:ref

11 points by allendking 5y ago

Last night Digital Ocean (https://www.digitalocean.com https://www.digitalocean.com) completely SHUT DOWN our account asking for verification (automated email) with ZERO warning. They wrote that they wanted to understand our usage and asked to explain our workload (automated email), but WHY shut down without notice? We are live with users and paying customers! How can DigitalOcean.com justify shutting down our servers WITHOUT ANY notice? Ethical? We think not! We explained that we have a SAAS platform and company based in California that we have built and are ready to launch our new version. After providing all the documentation, we got an email from an outsourced low-level "specialist" basically screwing us over even further. Sorry, we can't give any REASON but go host somewhere else (to paraphrase). Oh, and by the way, our team has ZERO ACCESS to our database and servers. How do you propose we move Mr. Specialist? STARTUP Founders - may we suggest you AVOID company Digital Ocean! We signed up through the Stripe Atlas program and applied Hatch at the beginning of the month. We were stoked to have customers and launch quickly. They completely screwed us, and it's unclear if we'll ever get our database back. We asked them to escalate the matter to a supervisor, pleading they give us a few days to move over to another company and retrieve our database. Still no word. Luckily, we have a backup from a week ago, but we'll still lose hundreds of transactions and thousands dollars if they don't give us back access. Our company will be reporting DigitalOcean to the BBB and potentially filing lawsuits for damages because this type of behavior is unethical and unprofessional (from our point of view). This action from Digital-Ocean is costing us money and time during a critical launch period. Fellow start-up founders AVOID the headache. On the flip side, we've added "Disaster Relief Plan" into a future sprint, so once we figure out how to recover from this, we'll be better prepared for situations with unscrupulous hosting companies that don't value their customers.

akhmatova 5y ago
We asked them to escalate the matter to a supervisor, pleading they give us a few days to move over to another company and retrieve our database. Still no word. Unannounced shutdowns are one thing, but then ghosting the customer and simply refusing to answer their inquiries is another. I'd like to know if Digital Ocean has another side to this story, but I don't have high hopes that they will. Best of luck, and thanks for sharing your experience. Luckily, we have a backup from a week ago, but we'll still lose hundreds of transactions and thousands dollars if they don't give us back access. It isn't clear here if the loss of transactions is from the time window between the last backup and the servers being shut down, or from loss of business after the shutdown. If it's the former, then ... at least we know that isn't Digital Ocean's fault.
- allendking 5y ago
  We'd love for Digital Ocean to have an excellent customer service response. Rebuilding our hosting infrastructure on a new provider may take a few days. Tomorrow, we'll have our Lead Developer start moving onto a new hosting platform while our full-time team of nine (9) developers enjoys a miny unplanned paid vacation. We signed up for the digital backup service that Digital Ocean offers, but it does no good if you can't access those backups. Luckily, we have a manual backup from a week ago. Unfortunately, we'll need to manually figure out what is missing, which will require directly calling customers via the Stripe receipts. Dear customers, what did you sign up for again? We're sorry, digitalocean.com has... and we need to match your payments receipts and manually create new records for profiles, credits, and reservations (only a few hundred phone calls). Moreover, payments, bookings, and reservations have completely been halted due to the downtime imposed by Digital Ocean. Existing and potential customers are unable to process transactions. Maybe new customers will wait, or perhaps they'll Google another provider that can make a reservation now (who likes waiting).
db48x 5y ago
The BBB is toothless; don’t bother. File a lawsuit.
- allendking 5y ago
  Thanks for the recommendation, and that does make sense. We don't wish to litigate this matter because launching a start-up takes a lot of time and resources. Unfortunately, if the financial losses continue to accrue, we'll need to pursue legal options for equitable remuneration. For now, we remain optimistic that a supervisor will quickly see their mistake and fix the situation.
  - db48x 5y ago
    That is true, but if nobody ever files a lawsuit then companies will just continue to run roughshod over everybody knowing that there will be no consequences.
  - thfuran 5y ago
    It costs very little to get a lawyer to draft an angry letter and it certainly won't slow down the escalation process.

allendking 5y ago

Account Reactivated - DAMAGE ALREADY DONE The trigger happy outsourced customer service representative decided to reactivate our account instead of escalating the matter (per our request) to a supervisor. We will ask again for the issue to be escalated to a supervisor so that Digital Ocean can fix this from happening to future customers. We are currently planning on moving to a new hosting company and began the process yesterday. Excellent customer service and professionalism are vital for the stability of our SAAS as we move to scale. Unfortunately, we can't recommend digitalocean.com based on this experience. In the comments below are the emails from DigitalOcean (Caveat Emptor). I appreciate the support from this community. Hopefully, this information will be helpful to future start-up founders.

allendking 5y ago
Hi there, We're excited to have you as a customer of DigitalOcean and we would like to verify your account. Because your recent account activity is atypical, we’d like to better understand your product needs. To help us in restoring access to the account, please respond to this email message with a description of the workload you’ve deployed on your Droplets and how you plan to use your account going forward. We appreciate your understanding and look forward to better understanding how DigitalOcean can support your needs! Regards, DigitalOcean Security ref:_00Df218t5m._5004P1qDp3y:ref
- deleted 5y ago
  [deleted]
allendking 5y ago
Hello there, Thank you so much for your patience, as we investigated this incident and reviewed the information you have shared with us so far, we have come to the conclusion that your deployment and actions were in violation of our Terms of Service Agreement [1] and Acceptable Use Policy [2] and as a result, we are unable to remove the lock from your account. I sincerely apologize for any inconvenience caused by this. Additionally, please note that we're unable to share details regarding what factors we considered that led to this decision or how we came to this conclusion as this is critical to maintaining the integrity of our platform security operations. You can review our Terms of Service agreement and Acceptable Usage policy here [1] Terms of Agreement: Legal - Terms of Service Agreement (digitalocean.com) [2] Acceptable Usage Policy: Legal - Acceptable Use Policy (digitalocean.com) We recommend you find a new service provider that better suits your needs as our security tooling would consider these actions when you sign up again resulting in account locks and termination of linked accounts during manual platform-wide reviews. If you still feel this is in error we request you to share detailed information about you, your business, and your deployment with us so that we can take your explanation into further consideration and contextually analyze it with information we have on file with us to review this decision. Regards, Nouroze Developer Experience Specialist ref:_00Df218t5m._5004P1qDp3y:ref