5 ms·

They key is usually something that is pretty current but can't be corrupted. A drive in a closet doesn't get you there. The nigeria office offline due to power

They key is usually something that is pretty current but can't be corrupted. A drive in a closet doesn't get you there. The nigeria office offline due to power does. I feel good with S3 with an object lock and retention policy for backups.

rectang 5y ago
Pretty current, can't be corrupted, and complete. Unless you're practicing continuous restoration to ensure that your backups are actually backing up what you need, the odds that you got everything critical are not good.
- tempnow987 5y ago
  The idea that throwing a USB in a drawer does better is absolutely ridiculous. I think this illustrates where folks can go wrong. Many folks and business TODAY are backing up to NAS / RAID arrays etc. Many are easily corruptible by ransomware. We see this REPEATEDLY with many infections of large orgs. The basic story here is that something not editable has great value even if not perfectly current. If you do a diff every 4 hours you have covered a LOT of ground already. I've sat through continuous DR architecturing discussions - the massive cost / complexity / scale can just be out of control for many users. In many use cases Active Directory for example behind by 4 hours is not game ending. With notice to users that recent edits may not be captured you may be able to get back up and running pretty quickly.
  - rectang 5y ago
    I'm not singing the praises of "a USB in a drawer" — the medium doesn't matter, the number of copies doesn't matter. The only criteria I am considering is whether the backups are tested to prove that they actually work as intended. There are shades of gray, here: continuous restoration is a best practice, but backups which are tested even once, and perhaps incompletely, are better than backups which are never tested at all.
    - tempnow987 5y ago
      Many continuous restoration / HA systems are surprisingly vulnerable to hacks because the level of management / coordination is pretty high. They work well if datacenter 1 goes down and you want to get to datacenter 2. They work less well (in my view) if someone gets root. A fair number of public victims you read about actually DID have continuous backups / dr setups.
      - rectang 5y ago
        OK, but how about at least testing that that it's actually feasible to restore from that S3 bucket of yours? Verifying that restoration is possible once is better than not testing restoration at all. As for verifying restoration on an ongoing basis, if the attacker has root I suppose we can argue about the cost/benefit ratio but continuous backup validation doesn't hurt.
        tempnow987 5y ago
        My comment was responding to USB in a closet. Where did I say that I don't test restores? I think you are misunderstanding the S3 object lock feature. With S3 object lock, you write your backup, then can read to verify or do a test recovery, but you cannot modify or delete for the retention period. In the package I use it's called Veeam Ready-Object with Immutability and there are now a fair number of solutions that meet that target.
        rectang 5y ago
        This has been a pretty unsatisfying exchange. :( It's a good thing we don't work together, we seemingly cannot get on the same page. From the start, I've been trying to add the qualifier that regardless of whether you throw your backup into a USB drive or an S3 bucket, if you don't test restoration, your backups are probably incomplete.
        tempnow987 5y ago
        "Unless you're practicing continuous restoration to ensure that your backups are actually backing up what you need, the odds that you got everything critical are not good." "OK, but how about at least testing that that it's actually feasible to restore from that S3 bucket of yours?" These comments have been both off topic and unhelpful. I gave an example of S3 with object lock (or veam equivalents) that I liked - to be helpful to others exploring this question. Continuous type services are what this story is about - how things that are disconnected / not continuous may fare better in a hack / ransomware situation. Lecturing me that I need "continuous restoration" and or making random claims that I don't test restoration (huh?) is weird. If I was to give some feedback -> focus on one thing. If you don't like S3 with object lock, say that. If you want to add some comment about restorations from S3 with object lock being tricky for some reason, say it and explain it. If you have something about "continuous restoration" helping in malware hacks explain why. In many cases this can pass corrupted data over to your backup site if you are doing HA/DR work where continuous stuff is most common.

rectang 5y ago

Pretty current, can't be corrupted, and complete. Unless you're practicing continuous restoration to ensure that your backups are actually backing up what you need, the odds that you got everything critical are not good.

tempnow987 5y ago
The idea that throwing a USB in a drawer does better is absolutely ridiculous. I think this illustrates where folks can go wrong. Many folks and business TODAY are backing up to NAS / RAID arrays etc. Many are easily corruptible by ransomware. We see this REPEATEDLY with many infections of large orgs. The basic story here is that something not editable has great value even if not perfectly current. If you do a diff every 4 hours you have covered a LOT of ground already. I've sat through continuous DR architecturing discussions - the massive cost / complexity / scale can just be out of control for many users. In many use cases Active Directory for example behind by 4 hours is not game ending. With notice to users that recent edits may not be captured you may be able to get back up and running pretty quickly.
- rectang 5y ago
  I'm not singing the praises of "a USB in a drawer" — the medium doesn't matter, the number of copies doesn't matter. The only criteria I am considering is whether the backups are tested to prove that they actually work as intended. There are shades of gray, here: continuous restoration is a best practice, but backups which are tested even once, and perhaps incompletely, are better than backups which are never tested at all.
  - tempnow987 5y ago
    Many continuous restoration / HA systems are surprisingly vulnerable to hacks because the level of management / coordination is pretty high. They work well if datacenter 1 goes down and you want to get to datacenter 2. They work less well (in my view) if someone gets root. A fair number of public victims you read about actually DID have continuous backups / dr setups.
    - rectang 5y ago
      OK, but how about at least testing that that it's actually feasible to restore from that S3 bucket of yours? Verifying that restoration is possible once is better than not testing restoration at all. As for verifying restoration on an ongoing basis, if the attacker has root I suppose we can argue about the cost/benefit ratio but continuous backup validation doesn't hurt.
      - tempnow987 5y ago
        My comment was responding to USB in a closet. Where did I say that I don't test restores? I think you are misunderstanding the S3 object lock feature. With S3 object lock, you write your backup, then can read to verify or do a test recovery, but you cannot modify or delete for the retention period. In the package I use it's called Veeam Ready-Object with Immutability and there are now a fair number of solutions that meet that target.
        rectang 5y ago
        This has been a pretty unsatisfying exchange. :( It's a good thing we don't work together, we seemingly cannot get on the same page. From the start, I've been trying to add the qualifier that regardless of whether you throw your backup into a USB drive or an S3 bucket, if you don't test restoration, your backups are probably incomplete.
        tempnow987 5y ago
        "Unless you're practicing continuous restoration to ensure that your backups are actually backing up what you need, the odds that you got everything critical are not good." "OK, but how about at least testing that that it's actually feasible to restore from that S3 bucket of yours?" These comments have been both off topic and unhelpful. I gave an example of S3 with object lock (or veam equivalents) that I liked - to be helpful to others exploring this question. Continuous type services are what this story is about - how things that are disconnected / not continuous may fare better in a hack / ransomware situation. Lecturing me that I need "continuous restoration" and or making random claims that I don't test restoration (huh?) is weird. If I was to give some feedback -> focus on one thing. If you don't like S3 with object lock, say that. If you want to add some comment about restorations from S3 with object lock being tricky for some reason, say it and explain it. If you have something about "continuous restoration" helping in malware hacks explain why. In many cases this can pass corrupted data over to your backup site if you are doing HA/DR work where continuous stuff is most common.