6 ms·

What’s the difference between replication and backups? Is the distinction that backups must be stored on separate infrastructure, whereas replication might stil

What’s the difference between replication and backups? Is the distinction that backups must be stored on separate infrastructure, whereas replication might still be 1 or 2 points of failure?

lukevp 6y ago
Replication is about having data level redundancy to protect from drive failure. Backups are about having point in time snapshots of the system state, and about having them tiered from a location perspective. The 3-2-1[1] principle says to have 3 total copies, 2 of which are local but on different devices, and 1 of which is offsite. This gives you tiers of recoverability. It’s important from a backup perspective that it’s point in time as well, otherwise as soon as you get ransomware that encrypts your file you now have replicated those changes everywhere. [1] https://www.backblaze.com/blog/the-3-2-1-backup-strategy/ https://www.backblaze.com/blog/the-3-2-1-backup-strategy/
- mumblemumble 6y ago
  I would add to that, that the offsite one should also be offline.
  - ashtonkem 6y ago
    Or at least in an append only data store.
    - mumblemumble 6y ago
      Though, tread carefully there. If the thing that makes it append only is that you're just not using its destructive update features, then the store isn't really append only. Just because everyone agreed to limit the ways they interact with the data store doesn't mean you can trust buggy code, sloppy programming, and attackers to honor that agreement.
  - skoskie 6y ago
    That’s a lot harder to pull off. What methods do you use to accomplish this?
    - randoramax 6y ago
      S3 bucket with object lock is as offline and convenient as it gets
      - reaperducer 6y ago
        I don't know much about S3, but isn't S3, by definition, online, not offline?
        zxcvbn4038 6y ago
        I’ve always wondered if Amazon backs up S3. I don’t think they explicitly say but I get the impression that it is the user’s responsibility to replicate to a second region to guard against data loss so I am guessing not. Object Lock wouldn’t protect against an S3 failure.
        count 6y ago
        They say it's designed to provide significant durability : https://docs.aws.amazon.com/AmazonS3/latest/dev/DataDurability.html https://docs.aws.amazon.com/AmazonS3/latest/dev/DataDurabili... (99.999999999% durable over a year per object, and able to sustain data loss in 2 facilities). Given the peeks that amazon has provided into the scale of S3, I don't know if you CAN 'back it up'.
        danielheath 6y ago
        As far as I can tell it’s replicated (by default - see “reduced redundancy storage”), not backed up.
        csydas 6y ago
        AWS S3 (and a few compliant providers) offer immutable options, both in governance/compliance mode. Allegedly, compliance mode is unalterable by any account period, I guess the equivalent of the immutable attribute without an overriding account. I'm not immediately familiar with any literature on attacks on this feature, but I've also not searched hard; however, I know from my clients that it's an accepted form of WORM, and that cloud storages like S3 are considered in the same vein as tape when immutability is in play. I suppose it will be a case in the future that proves the efficacy of AWS/Azure/s3 providers, but for now, a lot of regulatory policies for 3-2-1 allow for such storage to fulfill the "2" part of the 3-2-1 rule.
deleted 6y ago
[deleted]
geerlingguy 6y ago
Classically, backups are taken then stored offline (e.g. on archive tape, on optical media, or on drives that are disconnected and put in storage between backups). Otherwise, if they are not 'cold' backups, they are still disconnected so if the main storage blows up, it won't impact the data in the backups. Replication usually involves storage that is powered up and connected somehow to the same systems as the main storage; and any data that's corrupted on the main storage would propagate to the replicated storage.
kgermino 6y ago
I think the biggest factor is timing (although backups should also be on a second system). In short, if anything that happens to the files is immediately copied to the "backup" then you don't actually have a chance to recover from any software problems. Whereas, if you make a copy of the data every night and keep the last 30 copies you can find an issue like this and go back in time to retrieve the files from before it started.
ResidentSleeper 6y ago
Backups protect against a wider class of problems. For example, a software vulnerability/bug or a human error could result in the deletion of an object in all replicas (because deletions are replicated too), but you'd usually need another incident to occur for you to lose the backup as well.
throwaway9d0291 6y ago
Replication is intended to create a 1:1 copy of the data. If data is removed from the primary system, replication will ensure it's removed from the replica. A backup can be a 1:1 copy but it should be set up such that something going wrong with the primary (e.g. cryptolocker malware), can't affect it (since "something went wrong with the primary" is what the backup is intended to resolve). To do that you could take the backup offline or use features like filesystem snapshots to ensure that changes can be rolled back.
SAI_Peregrinus 6y ago
If your drive fails (or a single computer fails) replication saves your data, and keeps things running seamlessly. If you `sudo rm -rf / --no-preserve-root` your drive, replication deletes everything, while backups let you restore to the last time you took a backup.
gameswithgo 6y ago
A backup would be resistant to someone accidentally deleting all the files, for instance. Replication would not.
jrochkind1 6y ago
The problem is that most replication setups will replicate deletes (and other modifications) as well. So if misbehaving software deletes (or corrupts) things, those deletes (or corruptions) will be replicated, and the replication does not give you a way to actually recover from a mistaken delete.
njharman 6y ago
Replication includes replicating deletes (and overwrites, and subtilier changes). Replication is READ / WRITE. Backups are WRITE ONCE. They persist if the original is deleted or modified. Separate infrastructure and geographical distribution are orthogonal.
dec0dedab0de 6y ago
I see a bunch of explanations, but I don't think any of them really drove home the reason for why you usually need both. Imagine if you had a document that stored useful information. You had this document automatically replicated to another system in a different time zone every time there was a change. You think you're doing great, if there is an outage in the one system you just get your important document from the other system. Then one day someone accidentally copies and pastes the wrong data into the file. Now what do you do? If you goto your replicated copy it also has the bad data. The answer is you should have had backups too. so you could go back an hour,a day or a week or even much longer. Depending on how much data you have, how often it changes, how important it is, and how quickly someone would notice bad data.

What’s the difference between replication and backups? Is the distinction that backups must be stored on separate infrastructure, whereas replication might still be 1 or 2 points of failure?

lukevp 6y ago

Replication is about having data level redundancy to protect from drive failure. Backups are about having point in time snapshots of the system state, and about having them tiered from a location perspective. The 3-2-1[1] principle says to have 3 total copies, 2 of which are local but on different devices, and 1 of which is offsite. This gives you tiers of recoverability. It’s important from a backup perspective that it’s point in time as well, otherwise as soon as you get ransomware that encrypts your file you now have replicated those changes everywhere. [1] https://www.backblaze.com/blog/the-3-2-1-backup-strategy/ https://www.backblaze.com/blog/the-3-2-1-backup-strategy/

mumblemumble 6y ago
I would add to that, that the offsite one should also be offline.
- ashtonkem 6y ago
  Or at least in an append only data store.
  - mumblemumble 6y ago
    Though, tread carefully there. If the thing that makes it append only is that you're just not using its destructive update features, then the store isn't really append only. Just because everyone agreed to limit the ways they interact with the data store doesn't mean you can trust buggy code, sloppy programming, and attackers to honor that agreement.
- skoskie 6y ago
  That’s a lot harder to pull off. What methods do you use to accomplish this?
  - randoramax 6y ago
    S3 bucket with object lock is as offline and convenient as it gets
    - reaperducer 6y ago
      I don't know much about S3, but isn't S3, by definition, online, not offline?
      - zxcvbn4038 6y ago
        I’ve always wondered if Amazon backs up S3. I don’t think they explicitly say but I get the impression that it is the user’s responsibility to replicate to a second region to guard against data loss so I am guessing not. Object Lock wouldn’t protect against an S3 failure.
        count 6y ago
        They say it's designed to provide significant durability : https://docs.aws.amazon.com/AmazonS3/latest/dev/DataDurability.html https://docs.aws.amazon.com/AmazonS3/latest/dev/DataDurabili... (99.999999999% durable over a year per object, and able to sustain data loss in 2 facilities). Given the peeks that amazon has provided into the scale of S3, I don't know if you CAN 'back it up'.
        danielheath 6y ago
        As far as I can tell it’s replicated (by default - see “reduced redundancy storage”), not backed up.
      - csydas 6y ago
        AWS S3 (and a few compliant providers) offer immutable options, both in governance/compliance mode. Allegedly, compliance mode is unalterable by any account period, I guess the equivalent of the immutable attribute without an overriding account. I'm not immediately familiar with any literature on attacks on this feature, but I've also not searched hard; however, I know from my clients that it's an accepted form of WORM, and that cloud storages like S3 are considered in the same vein as tape when immutability is in play. I suppose it will be a case in the future that proves the efficacy of AWS/Azure/s3 providers, but for now, a lot of regulatory policies for 3-2-1 allow for such storage to fulfill the "2" part of the 3-2-1 rule.

deleted 6y ago

[deleted]

geerlingguy 6y ago

Classically, backups are taken then stored offline (e.g. on archive tape, on optical media, or on drives that are disconnected and put in storage between backups). Otherwise, if they are not 'cold' backups, they are still disconnected so if the main storage blows up, it won't impact the data in the backups. Replication usually involves storage that is powered up and connected somehow to the same systems as the main storage; and any data that's corrupted on the main storage would propagate to the replicated storage.

kgermino 6y ago

I think the biggest factor is timing (although backups should also be on a second system). In short, if anything that happens to the files is immediately copied to the "backup" then you don't actually have a chance to recover from any software problems. Whereas, if you make a copy of the data every night and keep the last 30 copies you can find an issue like this and go back in time to retrieve the files from before it started.

ResidentSleeper 6y ago

Backups protect against a wider class of problems. For example, a software vulnerability/bug or a human error could result in the deletion of an object in all replicas (because deletions are replicated too), but you'd usually need another incident to occur for you to lose the backup as well.

throwaway9d0291 6y ago

Replication is intended to create a 1:1 copy of the data. If data is removed from the primary system, replication will ensure it's removed from the replica. A backup can be a 1:1 copy but it should be set up such that something going wrong with the primary (e.g. cryptolocker malware), can't affect it (since "something went wrong with the primary" is what the backup is intended to resolve). To do that you could take the backup offline or use features like filesystem snapshots to ensure that changes can be rolled back.

SAI_Peregrinus 6y ago

If your drive fails (or a single computer fails) replication saves your data, and keeps things running seamlessly. If you `sudo rm -rf / --no-preserve-root` your drive, replication deletes everything, while backups let you restore to the last time you took a backup.

gameswithgo 6y ago

A backup would be resistant to someone accidentally deleting all the files, for instance. Replication would not.

jrochkind1 6y ago

The problem is that most replication setups will replicate deletes (and other modifications) as well. So if misbehaving software deletes (or corrupts) things, those deletes (or corruptions) will be replicated, and the replication does not give you a way to actually recover from a mistaken delete.

njharman 6y ago

Replication includes replicating deletes (and overwrites, and subtilier changes). Replication is READ / WRITE. Backups are WRITE ONCE. They persist if the original is deleted or modified. Separate infrastructure and geographical distribution are orthogonal.

dec0dedab0de 6y ago

I see a bunch of explanations, but I don't think any of them really drove home the reason for why you usually need both. Imagine if you had a document that stored useful information. You had this document automatically replicated to another system in a different time zone every time there was a change. You think you're doing great, if there is an outage in the one system you just get your important document from the other system. Then one day someone accidentally copies and pastes the wrong data into the file. Now what do you do? If you goto your replicated copy it also has the bad data. The answer is you should have had backups too. so you could go back an hour,a day or a week or even much longer. Depending on how much data you have, how often it changes, how important it is, and how quickly someone would notice bad data.