4 ms·
If it is backup, it doesn't need to be super safe. The risk that you lose both your primary data, (optionally your local backup) and your online backup at the s
by cm2187 9y ago
If it is backup, it doesn't need to be super safe. The risk that you lose both your primary data, (optionally your local backup) and your online backup at the same time is pretty insignificant, given that the online backup will be uncorrelated as in a different physical location (primary and local backup very correlated I agree).
- himlion 9y agoThe problem is sometimes you only find out your backup is corrupted when you need it.
- cm2187 9y agoYeah, agree. And I am not talking about enterprise level of reliability. More like personnal / small office. But for anything above a few TB, running your own hardware will be way cheaper if you take say a 5y horizon. Disks are so large today that you don't need a big config. In fact you might even keep two copies to reduce the risk. Of course there is the occasional trip to the datacentre. I made that mistake. I pay more in uber than I saved by picking a datacentre far away.
- w458cmau 9y agoDo you need co-location though? I dropped an HP Microserver at a friend's place, which backs up snapshots of my main backup server at home daily. Seems to work well so far, at quite limited costs. Since my main backup server also backs up his NAS we both win. Since all backups are encrypted there is also little privacy risk from theft (or snooping on one another).
- cm2187 9y agoWith 1Gbps symetrical connections slowly becoming more frequent this would be a much cheaper alternative (assuming you only run it for backups, I also run a mail server and some websites). But in London it is not very practical. I know very few people who have really fast broadband.
- qeternity 9y agoIf you don't test your backups, then they're not backups.
- jjn2009 9y agoyup, the assertion that both failing is highly unlikely is under the assumption that you are checking the status of each copy regularly.
- cm2187 9y agoIn fact for any disk, it is important to have a script that reads all the data at least every couple of months. It will force the bad sectors to be identified or to be notified you have a bad disk before things get worse.
- mongol 9y agoI would argue the opposite. When you reach for your backup, you are in a dire situation. If it fails you then, things will be very gloom.
- klodolph 9y ago> The risk that you lose both your primary data, (optionally your local backup) and your online backup at the same time is pretty insignificant, given that the online backup will be uncorrelated as in a different physical location (primary and local backup very correlated I agree). Actually, that reasoning is exactly what I wanted to talk about. When the primary is lost, there is a much higher chance than you'd expect that a secondary contains unrecoverable errors. This is why RAID 5 arrays fail to rebuild after a disk failure so often—they're supposed to be able to tolerate a single disk failing, but they can't tolerate a disk failing and any other IO error at the same time. Part of this is due to how short the timeout is for failed reads in RAID setups, but I've still seen a lot of RAID 5 arrays fail, and I've seen a few RAID 6 arrays fail too. On top of that, there's the high chance of configuration errors in DIY systems.
- cm2187 9y agoAgree. Having a script doing a full read of all the data every couple of months and sending a report by email (that you would notice if not sent) is a sort of must have.
- w458cmau 9y agoI am still looking for a good way of doing this. Weekly integrity check of all backed up data. E-Mail which informs of result. Web interface which shows overview of results of historical checks. External service which sends E-Mail if integrity check failed to run (e.g. https://deadmanssnitch.com/ https://deadmanssnitch.com/).
- cm2187 9y agoIntegrity is harder because you have to sort of maintain a signature of every file on the side. What I do is to just have a script that reads all data. If some data is unreadable an exception will be thrown and a text message will be sent to me. And a confirmation email is sent otherwise. In .net it's only a few lines of codes. Haven't thought of deadmansnitch but it's a good idea. Would just take one more line of code.
- 9y ago
- jorangreef 9y agoIf it is backup, then it needs to be scrubbed regularly to detect and repair bit rot.
- jo909 9y agoMy backup contains files and old versions of files I no longer have on the primary storage. It is much more than a 1:1 copy, and can not be recreated. I need to feel safe that I can reasonably undo my changes on the primary storage, even if it takes me years to realise what I did. This is why I keep two backups. Until now, the secondary offsite copy was on ACD...
- dsr_ 9y agoThat's not a backup. That's an archive. You need to backup archives the same way you do any other data you care about.