6 ms·

I'm really happy to see Ben and Moisey get this far. They are both extremely competent sysadmins and system architects too. They retained a lot of equity as w

I'm really happy to see Ben and Moisey get this far. They are both extremely competent sysadmins and system architects too. They retained a lot of equity as well - good to see.

deleted 6y ago
[deleted]
TechBro8615 6y ago
Between the two of them, they own less than 20% of the company. Is this a lot? Not in my opinion. They sold 80% of their company.
- Tostino 6y ago
  It's a pretty big pie that got baked, that <20% is quite a lot in the scheme of things.
- jkaplowitz 6y ago
  Compared to how much most VC-fueled founders would own at IPO stage, sounds like a lot to me.
- tgtweak 6y ago
  They are also in the holding co
- ZephyrBlu 6y ago
  Yes it is. Usually founders would only have a few percentage points at the tail end of things.
sneak 6y ago
My experience is that not only did they start out as incompetent, they actively downplayed early security issues and outright lied about certain aspects of their products. I found major, glaring security/architecture issues with their main product early on, and was told by DO staff to go full-disclosure because it was working as designed. When I did so, DO lied about the impact on their blog. Those lies are still up: https://www.digitalocean.com/blog/transparency-regarding-data-security/ https://www.digitalocean.com/blog/transparency-regarding-dat... > At no time was customer data "leaked" between accounts. (I have screenshots of other DO customers' data.) https://news.ycombinator.com/item?id=6983097 https://news.ycombinator.com/item?id=6983097 Even their outage blog post from a few months ago is now deleted. My experience is that DO is a shady company, and I'll not do any serious/critical business with them or ever use them to store secret information. As for the lying, I'd personally never do business with any of the early DO management/founders. There's no amount of money that can be used to buy a reputation for integrity.
- yte15 6y ago
  Your issue was exactly, and I mean exactly, what I had in mind when I said “learning experiences” upthread. I followed it closely. Linode painfully learned the scrubbing lesson as well - and DO was in a worse position to learn it at the time, being forced to throw the scrub write load at SSDs that don’t like that type of load. That’s the real issue, honestly, that they probably started noticing SMART warnings from scrubbing by default on SSDs (particularly for cheap instances that turn over often), and took the venture capital approach of taping over it to make the next round by making it opt in. It’s a tough problem to deal with and not something you’re likely to think of designing such a product from first principles. That’s not an indictment, it’s just fundamental to experience gained doing this stuff (and it’s perilous to get wrong). Filtering unsolicited ARP to prevent domUs from hijacking default gateways was another lesson in blood, and one of the first things we tried within five minutes when we did competitive on DO around launch time (they hadn’t thought of it; it worked).
  - sneak 6y ago
    It's not tough to solve. The whole scrub thing was a red herring: you don't need to scrub anything to not leak data, you just thin provision. It's (mostly) fine if customer data stays on your disks after they delete. It's not fine to give it to the next customer because you don't know how disk abstraction works. My issue is mainly how they coped with it, which has nothing to do with their (at the time) technical incompetence: they simply lied about the effects. Competent or not, lying on your corporate blog isn't a good choice. It doesn't take any special skills or training to be honest on your journey from incompetence to competence, all it takes is integrity.
    - yte15 6y ago
      Yeah. Agreed. I unfairly walked back solve to “deal with” on you simply from knowing the considerations that go into bursts of write load every time a customer clicks “delete”. At the time Linode had a host-side scrub daemon that simply ate LVs as customers deleted. It’s technically simple, but drive longevity and capacity on the host (the user is probably recreating their VM, for example) is where it gets tricky. External considerations. In the end scrubbing is basically killing a few dozen inodes of user data, but the architecture in VPS usually requires you to nuke the whole image, including what is ultimately the host’s 59th copy of Ubuntu. Just managing the iops without annoying neighboring customers is a challenge. To my cloning point I made at the top of the thread, Linode was certainly no stranger to the reality distortion field when it suited, and I think that’s yet another thing DO copied. That vertical is closer to B2C since you’re usually dealing with individuals, not sales teams, and glossing over stuff is a bit easier than when your counterparty has a better engineering team than you.

I'm really happy to see Ben and Moisey get this far. They are both extremely competent sysadmins and system architects too. They retained a lot of equity as well - good to see.

deleted 6y ago

[deleted]

TechBro8615 6y ago

Between the two of them, they own less than 20% of the company. Is this a lot? Not in my opinion. They sold 80% of their company.

Tostino 6y ago
It's a pretty big pie that got baked, that <20% is quite a lot in the scheme of things.
jkaplowitz 6y ago
Compared to how much most VC-fueled founders would own at IPO stage, sounds like a lot to me.
tgtweak 6y ago
They are also in the holding co
ZephyrBlu 6y ago
Yes it is. Usually founders would only have a few percentage points at the tail end of things.

sneak 6y ago

My experience is that not only did they start out as incompetent, they actively downplayed early security issues and outright lied about certain aspects of their products. I found major, glaring security/architecture issues with their main product early on, and was told by DO staff to go full-disclosure because it was working as designed. When I did so, DO lied about the impact on their blog. Those lies are still up: https://www.digitalocean.com/blog/transparency-regarding-data-security/ https://www.digitalocean.com/blog/transparency-regarding-dat... > At no time was customer data "leaked" between accounts. (I have screenshots of other DO customers' data.) https://news.ycombinator.com/item?id=6983097 https://news.ycombinator.com/item?id=6983097 Even their outage blog post from a few months ago is now deleted. My experience is that DO is a shady company, and I'll not do any serious/critical business with them or ever use them to store secret information. As for the lying, I'd personally never do business with any of the early DO management/founders. There's no amount of money that can be used to buy a reputation for integrity.

yte15 6y ago
Your issue was exactly, and I mean exactly, what I had in mind when I said “learning experiences” upthread. I followed it closely. Linode painfully learned the scrubbing lesson as well - and DO was in a worse position to learn it at the time, being forced to throw the scrub write load at SSDs that don’t like that type of load. That’s the real issue, honestly, that they probably started noticing SMART warnings from scrubbing by default on SSDs (particularly for cheap instances that turn over often), and took the venture capital approach of taping over it to make the next round by making it opt in. It’s a tough problem to deal with and not something you’re likely to think of designing such a product from first principles. That’s not an indictment, it’s just fundamental to experience gained doing this stuff (and it’s perilous to get wrong). Filtering unsolicited ARP to prevent domUs from hijacking default gateways was another lesson in blood, and one of the first things we tried within five minutes when we did competitive on DO around launch time (they hadn’t thought of it; it worked).
- sneak 6y ago
  It's not tough to solve. The whole scrub thing was a red herring: you don't need to scrub anything to not leak data, you just thin provision. It's (mostly) fine if customer data stays on your disks after they delete. It's not fine to give it to the next customer because you don't know how disk abstraction works. My issue is mainly how they coped with it, which has nothing to do with their (at the time) technical incompetence: they simply lied about the effects. Competent or not, lying on your corporate blog isn't a good choice. It doesn't take any special skills or training to be honest on your journey from incompetence to competence, all it takes is integrity.
  - yte15 6y ago
    Yeah. Agreed. I unfairly walked back solve to “deal with” on you simply from knowing the considerations that go into bursts of write load every time a customer clicks “delete”. At the time Linode had a host-side scrub daemon that simply ate LVs as customers deleted. It’s technically simple, but drive longevity and capacity on the host (the user is probably recreating their VM, for example) is where it gets tricky. External considerations. In the end scrubbing is basically killing a few dozen inodes of user data, but the architecture in VPS usually requires you to nuke the whole image, including what is ultimately the host’s 59th copy of Ubuntu. Just managing the iops without annoying neighboring customers is a challenge. To my cloning point I made at the top of the thread, Linode was certainly no stranger to the reality distortion field when it suited, and I think that’s yet another thing DO copied. That vertical is closer to B2C since you’re usually dealing with individuals, not sales teams, and glossing over stuff is a bit easier than when your counterparty has a better engineering team than you.