4 ms·

Sounds like they built a dead-man's switch and then broke the process through which the man and the switch communicate.

• dtech 8y ago
  Would be a pretty ineffective dead mans switch if a backup from 5 minutes ago is available
  • spydum 8y ago
    The keys didn’t get deleted. Only the encrypted data. If keys are gone - they are GONE. AFAIK you cannot restore the keys
  • eridius 8y ago
    It's presumably intended as way to reclaim space now taken up by effectively-random garbage rather than a security measure.
• ajross 8y ago
  It was garbage collection. They were deleting data that couldn't be accessed (because no key existed in the system to decrypt it), but the DNS failure fooled the detection into thinking that a failed lookup meant "no key exists". Yikes. To be clear: it was ultimately only a 5 minute loss (and the fact that the DNS outage was simultaneous probably meant there wasn't much data being stored anyway) because they had a regular snapshot facility. So defense in depth saved them. Still, yikes. That's a pretty disastrous bug.
  • tybit 8y ago
    I’m really curious as to whether this would be because of REST APIs. Conflating ‘I can’t find that endpoint’ with ‘the endpoint says that resource has been deleted’ via a 404.