4 ms·
This raises a relevant concern that's been on my mind: what's the best way to back up cloud services? Given that services like S3 and Google Drive have many mor
by danepowell 11y ago
This raises a relevant concern that's been on my mind: what's the best way to back up cloud services? Given that services like S3 and Google Drive have many more nines of durability than any local storage system I could devise, are backups even worth the trouble?
There are a lot of cloud-to-cloud backup services out there, but to me that seems like the blind leading the blind, especially with regards to malicious data destruction. For instance, I've recently been experimenting with Cloudally to automatically back up Google Drive, which seems like a good solution at first- until you think about the fact that Cloudally uses Google accounts for authentication (and doesn't use 2FA for native authentication). In other words, an attacker with access to my primary data (Google Drive) would also have access to my backups. Worse than that, Cloudally actually increases the attack surface, since its lack of 2FA presumably makes it easier to crack than my Google account.
Similarly, I'm guessing a lot of cloud backup services share data centers with the services they are backing up.
- njharman 11y agodurability is not same as "protection". It doesn't include items like government seizure, hacking. Your own drives, in your physical possession fill in that gap (somewhat).
- nemo1618 11y agoIf you really care about durability, your best best is erasure-coding + a wide geographic distribution of shards. For example, you could encode 1 TB of data into four shards, each shard containing 500 GB. You distribute these to servers in SF, NYC, Berlin, and Sydney. The key here is that you only need two shards to recover your 1 TB of data, and they can be any two shards. So if lightning strikes Berlin, and the Big One hits SF, your data is still safe. And thanks to erasure-coding, you can achieve this with only 2x redundancy (instead of 4x).
- mirimir 11y agoIt's been a while since I followed this. I see over 100 Reed-Solomon erasure-coding projects on GitHub. Which would you recommend?
- nemo1618 11y agoWell, I have to shamelessly plug my startup, www.siacoin.com, which implements the scheme I described above using a peer-to-peer network, with payments made on a blockchain. We are using Klaus Post's excellent pure-Go implementation (https://github.com/klauspost/reedsolomon https://github.com/klauspost/reedsolomon) which exceeds 1GB/s throughput.
- mirimir 11y agoThank you. It's very cool. But can Sia work with hosts that are reachable only as Tor hidden services?