4 ms·

"borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. I'm surprised it's not more wel

"borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. I'm surprised it's not more well known.

locknitpicker 9mo ago
> "borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. Deduplication helps minimize space, but isn't it a major liability in backups? I mean, what happens when you try to restore your backups but a lone sector holding a file from way back in the past happens to not be recoverable? Doesn't it mean that no matter how frequent your backups are, your data is lost?
- NSUserDefaults 9mo ago
  In that case you are supposed to use your /other/ backup. Which you have.
  - DaSHacka 9mo ago
    Defeating the point of deduplication...
    - homebrewer 9mo ago
      You don't have backups if you only have one "backup". Look up sysadmin's 3-2-1.
    - NSUserDefaults 9mo ago
      On a single drive, deduplicate all you want. But you need to have two drives, at least, anyway.
- haradion 9mo ago
  It doesn't really hurt in practice because it's only one part of the full backup procedure. Deduplicate to save space; re-duplicate the (smaller) backups to separate media for redundancy; scrub regularly for bit rot and rotate your media. A proper backup system requires all but the first of those anyway.
- stuxnet79 9mo ago
  If you are storing your data in a filesystem like ZFS then the risk is lower since you'll know you have a bad sector long before you attempt to deduplicate. But I otherwise share your same concerns and I'm eager to hear from others how to mitigate this. I've lost a volume due to poor practice while encrypting so I'm understandably reluctant to overcomplicate my backup strategies in the name of space compression or privacy.
- LeoPanthera 9mo ago
  This is bordering on paranoia. If the bad sector contains a critical part of the filesystem, you're going to lose everything anyway. Do modern disks even have physical "sectors" anymore? Isn't it all virtual? Without dedup you're just going to backup less stuff, which is far worse.
  - locknitpicker 9mo ago
    > This is bordering on paranoia. What? You actually think that the mere idea of a backup getting corrupted is something that is "bordering on paranoia"? Have you ever heard of redundancy or even RAID? > If the bad sector contains a critical part of the filesystem, you're going to lose everything anyway. Do you honestly failed to understand that the problem is actually the "lose everything" part?
- dmd 9mo ago
  This is why Proxmox Backup Server has scheduled "verify" tasks which look at the sha256 of every stored hunk to make sure it hasn't rotted on disk. (If it has, it makes sure to re-acquire that data on the next backup - hopefully it's still available!)
- DrBurrito 9mo ago
  the backups in borg and restic are not classical copies, each snapshot is not independent, it is a collection of hashed blocks. In a sense, time machine and hard links do the same: they reference the same bits across snapshots, saving space. for the bit rot / faulty sector or failed drive, you use redundancy with RAID or similar in your file system. if all else fails and your backup system burns in a fire, then you reach for your other backup. deduplication and redundancy solve different problems.
IshKebab 9mo ago
I used to use Borg, but Restic (and it's Rust clone Rustic) are better.
- setopt 9mo ago
  Why? I’m currently trying to decide between Borg and Restic myself. Borg has worked very well for me in the past, and I’m a bit excited by the new Vorta GUI frontend. But I already have a BackBlaze B2 setup for backing up my Mac, and Restic would let me continue to use that as a backup store. I’m interested in hearing about differentiating points between them.
  - electroly 9mo ago
    Primarily it's the reason you already know: restic and borg are the same model, but restic doesn't need it to be an ssh-accessible filesystem on the remote end. Restic can send backups almost anywhere, including object storage like your Backblaze B2 (that's what I use with restic, too). I agree with OP: restic is strictly better. There's no reason to use borg today; restic is a superset of its functionality.
    - stogot 9mo ago
      Does restic work well with truenas?
      - electroly 9mo ago
        I don't know specifically, but it's a self-contained single file Go executable. It doesn't need much from a Linux system beyond its kernel. Chances are good that it'll work.
    - setopt 9mo ago
      Thanks! Then I’ll look more at Restic :)

"borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. I'm surprised it's not more well known.

locknitpicker 9mo ago

> "borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. Deduplication helps minimize space, but isn't it a major liability in backups? I mean, what happens when you try to restore your backups but a lone sector holding a file from way back in the past happens to not be recoverable? Doesn't it mean that no matter how frequent your backups are, your data is lost?

NSUserDefaults 9mo ago
In that case you are supposed to use your /other/ backup. Which you have.
- DaSHacka 9mo ago
  Defeating the point of deduplication...
  - homebrewer 9mo ago
    You don't have backups if you only have one "backup". Look up sysadmin's 3-2-1.
  - NSUserDefaults 9mo ago
    On a single drive, deduplicate all you want. But you need to have two drives, at least, anyway.
haradion 9mo ago
It doesn't really hurt in practice because it's only one part of the full backup procedure. Deduplicate to save space; re-duplicate the (smaller) backups to separate media for redundancy; scrub regularly for bit rot and rotate your media. A proper backup system requires all but the first of those anyway.
stuxnet79 9mo ago
If you are storing your data in a filesystem like ZFS then the risk is lower since you'll know you have a bad sector long before you attempt to deduplicate. But I otherwise share your same concerns and I'm eager to hear from others how to mitigate this. I've lost a volume due to poor practice while encrypting so I'm understandably reluctant to overcomplicate my backup strategies in the name of space compression or privacy.
LeoPanthera 9mo ago
This is bordering on paranoia. If the bad sector contains a critical part of the filesystem, you're going to lose everything anyway. Do modern disks even have physical "sectors" anymore? Isn't it all virtual? Without dedup you're just going to backup less stuff, which is far worse.
- locknitpicker 9mo ago
  > This is bordering on paranoia. What? You actually think that the mere idea of a backup getting corrupted is something that is "bordering on paranoia"? Have you ever heard of redundancy or even RAID? > If the bad sector contains a critical part of the filesystem, you're going to lose everything anyway. Do you honestly failed to understand that the problem is actually the "lose everything" part?
dmd 9mo ago
This is why Proxmox Backup Server has scheduled "verify" tasks which look at the sha256 of every stored hunk to make sure it hasn't rotted on disk. (If it has, it makes sure to re-acquire that data on the next backup - hopefully it's still available!)
DrBurrito 9mo ago
the backups in borg and restic are not classical copies, each snapshot is not independent, it is a collection of hashed blocks. In a sense, time machine and hard links do the same: they reference the same bits across snapshots, saving space. for the bit rot / faulty sector or failed drive, you use redundancy with RAID or similar in your file system. if all else fails and your backup system burns in a fire, then you reach for your other backup. deduplication and redundancy solve different problems.

IshKebab 9mo ago

I used to use Borg, but Restic (and it's Rust clone Rustic) are better.

setopt 9mo ago
Why? I’m currently trying to decide between Borg and Restic myself. Borg has worked very well for me in the past, and I’m a bit excited by the new Vorta GUI frontend. But I already have a BackBlaze B2 setup for backing up my Mac, and Restic would let me continue to use that as a backup store. I’m interested in hearing about differentiating points between them.
- electroly 9mo ago
  Primarily it's the reason you already know: restic and borg are the same model, but restic doesn't need it to be an ssh-accessible filesystem on the remote end. Restic can send backups almost anywhere, including object storage like your Backblaze B2 (that's what I use with restic, too). I agree with OP: restic is strictly better. There's no reason to use borg today; restic is a superset of its functionality.
  - stogot 9mo ago
    Does restic work well with truenas?
    - electroly 9mo ago
      I don't know specifically, but it's a self-contained single file Go executable. It doesn't need much from a Linux system beyond its kernel. Chances are good that it'll work.
  - setopt 9mo ago
    Thanks! Then I’ll look more at Restic :)