5 ms·

what's the optimal setup for long-term, large-scale (personal) data storage? I want to build one big Backup. Some initial research has pointed me to something

what's the optimal setup for long-term, large-scale (personal) data storage? I want to build one big Backup. Some initial research has pointed me to something like Bacula to manage the data backup process from a machine. With the 3-2-1 rule, I know I also need my Backup itself to have at least 3 copies, in at least 2 different forms (cloud/hard disk), at least one of which is off-site from me. As an individual, do you or anybody else know the best way to implement such a system? Should I buy one giant hard drive, use many hard drives to create a RAID array, something else?

kortex 7y ago
Oooh. I've been wrestling with this problem for a while now. Basically I'm working on a tiered system. Files/dirs are categorized by size (<10MB, <25GB, >25GB) , and by sensitivity (public, confidential, secure. And importance is usually proportional to security). I have fortunately found that security is usually inverse to size. Github/lab anything which makes sense. Confidential small stuff (sans keys) is just stored in gmail/drive. Big, boring stuff (music, ebooks) is just kept on external hard drives. Secure, ultra-important stuff, I don't really have a system for. The system I'm leaning towards is just encrypt archives and store the key/password securely, and store it like you would any boring data, with a local NAS and a cloud backup service of some sort, or just stored on drives offsite.
- silicon2401 7y ago
  Do you feel comfortable using cloud storage for so much of your content? My ideal is to be entirely self-backed-up. I want a personal git server, photo archive, etc. With bandwidth, service costs, vendor issues (dealing with google seems like a nightmare from reading online). How did you construct your NAS? Is it a single system, or multiple hard drives/storage solutions connected to your network?
  - kortex 7y ago
    It depends. Github is not going down. Gmail is not going down. If they do, it's Bug-out-bag time, and I am working on curating what information subset I need for that. Ideally though yes I would have my own entire backup system but I frankly don't trust myself enough to do it right, so hence some redundancy in the cloud. The NAS I am still designing actually :p

kortex 7y ago

Oooh. I've been wrestling with this problem for a while now. Basically I'm working on a tiered system. Files/dirs are categorized by size (<10MB, <25GB, >25GB) , and by sensitivity (public, confidential, secure. And importance is usually proportional to security). I have fortunately found that security is usually inverse to size. Github/lab anything which makes sense. Confidential small stuff (sans keys) is just stored in gmail/drive. Big, boring stuff (music, ebooks) is just kept on external hard drives. Secure, ultra-important stuff, I don't really have a system for. The system I'm leaning towards is just encrypt archives and store the key/password securely, and store it like you would any boring data, with a local NAS and a cloud backup service of some sort, or just stored on drives offsite.

silicon2401 7y ago
Do you feel comfortable using cloud storage for so much of your content? My ideal is to be entirely self-backed-up. I want a personal git server, photo archive, etc. With bandwidth, service costs, vendor issues (dealing with google seems like a nightmare from reading online). How did you construct your NAS? Is it a single system, or multiple hard drives/storage solutions connected to your network?
- kortex 7y ago
  It depends. Github is not going down. Gmail is not going down. If they do, it's Bug-out-bag time, and I am working on curating what information subset I need for that. Ideally though yes I would have my own entire backup system but I frankly don't trust myself enough to do it right, so hence some redundancy in the cloud. The NAS I am still designing actually :p