3 ms·

Don't consider it a backup until you've successfully restored the data from it. The first thing I do after setting up a new data backup is test a restore of th

Don't consider it a backup until you've successfully restored the data from it. The first thing I do after setting up a new data backup is test a restore of the data. Only after that will I feel confident that the backup procedure works right. In the article author's case, an attempt to restore would have caught the problem of the missing .env files and the large movie files. As for the Ctrl-C on both the backup and restore, you should check your I/O (network and disk) before terminating a process. Doing that would have confirmed that the process was still going, and indicate the rate at which the process was going.

spsesk117 5y ago
+1. `strace` can be very helpful here, to see if a process is stuck waiting on something or whether it's just zooming along with no output.
- dspillett 5y ago
  `progress` (https://github.com/Xfennec/progress https://github.com/Xfennec/progress) and similar can be very helpful too depending on the backup utilities being used (in my case often involving rsync) even if the processes normally have everything set to quiet so no progress information is automatically forthcoming.
  - spsesk117 5y ago
    Wow this looks great, didn't know about this tool. Thanks for the recommendation!
geoduck14 5y ago
> Don't consider it a backup until you've successfully restored the data from it. Ouch! I back up every 2 hours - should I REALLY restore from each of those?
- xgbi 5y ago
  No, not ALL backups, just the first backup you do: you test that you can restore the data from it. Then you can be confident that following backups will be restorable. Corrolary: If you introduce a new folder for another service in your backup, make sure you can restore it too.
  - foobarian 5y ago
    :itsatrap: It needs to be done periodically.
- AnimalMuppet 5y ago
  No, not if they're written by the same hardware using the same process. You need to restore data from one of them, and then another one every so often. Not every two hours, though.
- dspillett 5y ago
  Maybe, it depends on your data-loss and time to restore flexibility. Test regularly enough that you are confident it works. If you don't test at all you can't be confident that it works at all. For instance, if you have a full/diff/log backup cycle for a database for instance, perhaps test restore each full backup. Just be aware that if a full backup happens daily your comfort zone for data loss in the case of a disaster needs to be at least a day worth of work. Also, if you backup a range of data and your method allows for partial restores, you might do partial restores of key information far more often than you test everything. You could spread the testing load temporaly: check the really import parts every time and cycle through the other bits less often. Also if your concern is man-time, automate the process as much as possible. My mail server (running Zimbra) has a small replica in a VM that restores itself from the latest backup once a day and sends me a message to say what the last message in its queue was. This way if I don't get the message, or it says the last message was too far ago, I know something has gone wrong and the backup or the restore failed. I manually log in regularly to inspect a little deeper too, this is slow as the VM has far lower resources than the main box. If properly separated from other resources (so there are no single points of failure that can take it out along with everything else - mine isn't currently that well arranged) then this copy becomes an extra secondary single-snapshot backup itself.
AnimalMuppet 5y ago
I'd even go a step further. It's not a backup until you've restored it using different hardware. You really want to know that your tape (or whatever) can be read by a different tape reader than the one that wrote it.

spsesk117 5y ago

+1. `strace` can be very helpful here, to see if a process is stuck waiting on something or whether it's just zooming along with no output.

dspillett 5y ago
`progress` (https://github.com/Xfennec/progress https://github.com/Xfennec/progress) and similar can be very helpful too depending on the backup utilities being used (in my case often involving rsync) even if the processes normally have everything set to quiet so no progress information is automatically forthcoming.
- spsesk117 5y ago
  Wow this looks great, didn't know about this tool. Thanks for the recommendation!

geoduck14 5y ago

> Don't consider it a backup until you've successfully restored the data from it. Ouch! I back up every 2 hours - should I REALLY restore from each of those?

xgbi 5y ago
No, not ALL backups, just the first backup you do: you test that you can restore the data from it. Then you can be confident that following backups will be restorable. Corrolary: If you introduce a new folder for another service in your backup, make sure you can restore it too.
- foobarian 5y ago
  :itsatrap: It needs to be done periodically.
AnimalMuppet 5y ago
No, not if they're written by the same hardware using the same process. You need to restore data from one of them, and then another one every so often. Not every two hours, though.
dspillett 5y ago
Maybe, it depends on your data-loss and time to restore flexibility. Test regularly enough that you are confident it works. If you don't test at all you can't be confident that it works at all. For instance, if you have a full/diff/log backup cycle for a database for instance, perhaps test restore each full backup. Just be aware that if a full backup happens daily your comfort zone for data loss in the case of a disaster needs to be at least a day worth of work. Also, if you backup a range of data and your method allows for partial restores, you might do partial restores of key information far more often than you test everything. You could spread the testing load temporaly: check the really import parts every time and cycle through the other bits less often. Also if your concern is man-time, automate the process as much as possible. My mail server (running Zimbra) has a small replica in a VM that restores itself from the latest backup once a day and sends me a message to say what the last message in its queue was. This way if I don't get the message, or it says the last message was too far ago, I know something has gone wrong and the backup or the restore failed. I manually log in regularly to inspect a little deeper too, this is slow as the VM has far lower resources than the main box. If properly separated from other resources (so there are no single points of failure that can take it out along with everything else - mine isn't currently that well arranged) then this copy becomes an extra secondary single-snapshot backup itself.

AnimalMuppet 5y ago

I'd even go a step further. It's not a backup until you've restored it using different hardware. You really want to know that your tape (or whatever) can be read by a different tape reader than the one that wrote it.