4 ms·

Wow, for a company that boasts "no bullshit", only offering a month after destroying data and backups seems a little tone deaf Edit: in fairness, I'm not sure

by petee 7y ago

Wow, for a company that boasts "no bullshit", only offering a month after destroying data and backups seems a little tone deaf Edit: in fairness, I'm not sure how exactly you would quantify such a loss anyway...

Shalle135 7y ago
It sounds like they didn’t have any backups at all but rather relied on a active-active replication link to a secondary storage. Edit: who knows it may be related to the HPE issue. https://www.bleepingcomputer.com/news/hardware/hp-warns-that-some-ssd-drives-will-fail-at-32-768-hours-of-use/ https://www.bleepingcomputer.com/news/hardware/hp-warns-that...
- jnwatson 7y ago
  In other words, RAID is not backup.
- mschuster91 7y ago
  What baffles me is that there seems to be no way for either the customer or a data-recovery company to flash a new firmware onto the drive after it has failed. Someone there wanted to spare the few millicents of copper trace for a JTAG port?!
  - marcinzm 7y ago
    Probably to prevent supply chain firmware changes for hacking, espionage, etc.
- deelowe 7y ago
  Hmm... I wonder what the "incident" was. If it involved something akin to an "rm -rf," then of course their replication link didn't protect them.
- fencepost 7y ago
  Perhaps they were depending on snapshotting and were not prepared for some kind of hardware failure taking out the entire storage system.
joepie91_ 7y ago
Reputable hosting providers typically don't try to quantify such a loss, but rather outright offer a credit/compensation that is very obviously generous (say, a year or even two of free service). Especially when a small set of your customerbase is affected, it won't cost you that much, and "overcompensating" like that means that virtually noone is going to criticize you for quantifying it wrong; instead, the public narrative will be centered around "well, shit happens, they did their best and generously compensated".

by petee 7y ago

Shalle135 7y ago

It sounds like they didn’t have any backups at all but rather relied on a active-active replication link to a secondary storage. Edit: who knows it may be related to the HPE issue. https://www.bleepingcomputer.com/news/hardware/hp-warns-that-some-ssd-drives-will-fail-at-32-768-hours-of-use/ https://www.bleepingcomputer.com/news/hardware/hp-warns-that...

jnwatson 7y ago
In other words, RAID is not backup.
mschuster91 7y ago
What baffles me is that there seems to be no way for either the customer or a data-recovery company to flash a new firmware onto the drive after it has failed. Someone there wanted to spare the few millicents of copper trace for a JTAG port?!
- marcinzm 7y ago
  Probably to prevent supply chain firmware changes for hacking, espionage, etc.
deelowe 7y ago
Hmm... I wonder what the "incident" was. If it involved something akin to an "rm -rf," then of course their replication link didn't protect them.
fencepost 7y ago
Perhaps they were depending on snapshotting and were not prepared for some kind of hardware failure taking out the entire storage system.

joepie91_ 7y ago

Reputable hosting providers typically don't try to quantify such a loss, but rather outright offer a credit/compensation that is very obviously generous (say, a year or even two of free service). Especially when a small set of your customerbase is affected, it won't cost you that much, and "overcompensating" like that means that virtually noone is going to criticize you for quantifying it wrong; instead, the public narrative will be centered around "well, shit happens, they did their best and generously compensated".