4 ms·
I went down a bit of a rabbit hole of Digital Ocean and their "security" for production workloads. > Show me any other vps provider that silently provides acce
by jiggawatts 6y ago
I went down a bit of a rabbit hole of Digital Ocean and their "security" for production workloads.
> Show me any other vps provider that silently provides access to customer A's data to customer B after receiving commands from customer A to destroy their instance and then I'll believe you guys aren't at the very bottom of the "takes security seriously" list.
From: https://github.com/fog/fog/issues/2525#issuecomment-31337481 https://github.com/fog/fog/issues/2525#issuecomment-31337481
YC News Discussion: https://news.ycombinator.com/item?id=6983097 https://news.ycombinator.com/item?id=6983097
> You do not need to scrub or write anything to not provide user A’s data to user B in a multi-tenant environment. Sparse allocation can easily return nulls to a reader even while the underlying block storage still contains the old data. ... On top of all of that, when I pointed out that what they were doing was absolute amateur hour clownshoes, they oscillated between telling me it was a design decision working as intended (and that it was fine for me to publicize it), and that I was an irresponsible discloser by sharing a vulnerability.
From: https://news.ycombinator.com/item?id=20091026 https://news.ycombinator.com/item?id=20091026
> You've got an additional problem though, which is that this tells us you have two support channels: one that doesn't work (i.e. yours, the one you built), and one that does (Twitter-shaming). The first channel represents how you act when no one's watching; the second, how you act when they are. Most people prefer to deal with people for whom those two are the same.
From: https://news.ycombinator.com/item?id=20064169 https://news.ycombinator.com/item?id=20064169
Speaking of randomly locking accounts, the post-mortem kills me:
> The initial account lock and resource power down resulted from an automated service that monitors for cryptocurrency mining activity (Droplet CPU loads and Droplet create behaviors). These signals, coupled with a number of account-level signals (including payment history and current run rate compared to total payments) are used to determine if automated action is warranted to minimize the impact of potential fraudulent high-cpu-loads on other customers.
From: https://www.digitalocean.com/blog/an-update-on-last-weeks-customer-shutdown-incident/ https://www.digitalocean.com/blog/an-update-on-last-weeks-cu...?
In other other words, DO will kill your account with a curt email staring simply: "We have reviewed your account and have declined to activate it. No further information or action is required from you." for simply using "too much CPU"! https://pbs.twimg.com/media/D76ocofXoAY_xB5.png https://pbs.twimg.com/media/D76ocofXoAY_xB5.png
- mitjafelicijan 6y agoI would pin this post to the top if I could. Thanks for this info!
- marcinzm 6y agoIs there anything commercial that is both cheap and popular that doesn't cut corners to get there? edit: Point being, I don't see how DO could get their act together as their business model essentially requires them to act like this. If they don't cut corners in one area then they will somewhere else.
- jiggawatts 6y agoThey don't have to treat the customers as the enemy. They do so by choice. The root cause of the "DO killed our business" situation was that DO accepts credit cards for payment but provides "cash equivalent" as the product: Rented CPUs can be used crypto mining. But from the "outside" it's practically impossible to distinguish between crypto mining and legitimate high CPU usage, they "look" the same. Hence, DO felt that they were "forced" to kill everyone that signs up with a credit card and starts their CPU at 100% load, because a significant percentage of the time that's some hacker using a stolen credit card number to mine Bitcoins. DigitalOcean feels forced to treat customers as the enemy, because a significant percentage is the enemy. However, this business model of "we give you cash for credit cards" is their choice! Nobody forced them to do this. Credit cards aren't legally mandated tender. Not every shopping centre till provides a cash out service. Some take on this risk. Some don't. I've noticed that less well-to-do areas of my city don't offer this service, but the more posh areas with lower risk do offer cash out. Does the shopping centre get burnt by this sometimes? Yes, but that's their risk to bear. Digital Ocean is like a shopping centre in a low-rent district full of drug addicts that offers cash out. That's their choice to accept that much higher level of risk. What they're doing however is passing on the consequences to their customers. They're not bearing the risk. It's your risk to bear that other customers mine crypto on DO droplets. So what alternative does DO have, you ask? They could just ask customers to transfer money up-front, much like pre-paid SIM cards. Many other companies operating in high-risk customer markets do this kind of thing. Heck, Azure and AWS do exactly this: They offer enormous discounts if you pre-pay. Absolutely huge, like up to 50% or even 80% off! See: https://azure.microsoft.com/en-us/pricing/reserved-vm-instances/ https://azure.microsoft.com/en-us/pricing/reserved-vm-instan... So this is what DigitalOcean should do: When you sign up, on the next page after you've created your account it should give a full-page choice screen with two columns: Column A: "Prepay for savings and guaranteed service levels" -- blah blah blah, we won't randomly kill your account, and you get a 20% discount! Column B: "Credit card sign up" -- blah blah, ideal for dev/test, ideal for non-critical workloads, we may suspend your account if the usage exceeds certain thresholds, etc...