5 ms·

I love Proxmox but the absence of incremental backups is very annoying. If you have a large vm whose contents don't change much your options are limited as stor

by icefo 6y ago

I love Proxmox but the absence of incremental backups is very annoying. If you have a large vm whose contents don't change much your options are limited as storage (edited: and bandwidth) is not infinite. I also don't want to use aryufan patches to enable incremental backup. I feel like they could probably partner with duplicacy to ship that feature. Duplicacy already released a paid version specifically designed to backup VMware hosts so it should be possible to adapt it to backup Proxmox hosts. (Thank you all for your comments ! They gave me some ideas)

JustFinishedBSG 6y ago
Use ZFS as backend and tada your problem stops existing
- icefo 6y ago
  I could only use snapshots and regularly send the pool somewhere but it feel a bit hacky
  - jblwps 6y ago
    You don't send the whole pool--you send the datasets (ZFS filesystems) that you're concerned with.
iggldiggl 6y ago
I've been wondering about the very same problem (and stumbled across aryufan's patches as well), and discovered that they're now actually working on an expanded backup solution of their own: https://git.proxmox.com/?p=proxmox-backup.git;a=tree https://git.proxmox.com/?p=proxmox-backup.git;a=tree https://forum.proxmox.com/threads/proxmox-backup-server-and-client.68781/ https://forum.proxmox.com/threads/proxmox-backup-server-and-...
beagle3 6y ago
Use a CoW/snapshotting backing file system for your backup (btrfs/zfs), or one of the modern de-duplicating backup systems (bup/borg/restic), and the space taken will be proportional to the changed data, while still providing random-access to any snapshot -- Not much more you can expect in terms of space usage. snapshotting filesystems will be much more efficient in terms of time/effort compared to de-duplicating later, but other than that, all of these solutions are mostly equivalent.
- icefo 6y ago
  I don't argue that it would work but when you do that you're still sending all the data over some data link. For example if I have 1.5TB of VM and only 10-20Gb changed today I'm still sending everything to a remote storage. Snapshotting locally and sending the zfs volume seems to be a more efficient solution
  - beagle3 6y ago
    That's not at all true for bup and borg, almost certainly not true for restic (Haven't used it myself and can't tell for sure). bup and borg (and likely restic) keep local hash summaries that are synchronized with the repository if it is remote. They would still have to _read_ the entire 1.5TB VMS if every vm file has changed (there are no facilities on a regular file system that they can use to figure out which parts changed), but they WILL use those local hashes to find out parts actually changed, and only send those changes to the remote storage. The local hashes typically occupy 0.5% of the data size (for bup, by default: 20 byte SHA1 per 8192 bytes on average; You can change the chunking parameters for bup/borg/restic for a tradeoff between cache size and granularity of change detected; for VMs it might make more sense to e.g. have 256KB chunks in which case you'll have 0.01% local storage overhead, and likely still negligible network transfer overhead. > I don't argue that it would work but when you do that you're still sending all the data over some data link. That's only true if you include e.g. the SATA link in that list; To avoid that, you need a filesystem that effectively tracks change ("damage") regions, or VM storage that uses lots of underlying files so you can track it on a regular file system (parallels on the mac used to do that so time machine backups are efficient). no way around it.
    - toomuchtodo 6y ago
      > there are no facilities on a regular file system that they can use to figure out which parts changed Doesn't ZFS do block level checksums that could be used for this, and backup tooling could use block pointer metadata for this purpopse?
      - 6y ago
blue1 6y ago
pve-zsync ?

by icefo 6y ago

JustFinishedBSG 6y ago

Use ZFS as backend and tada your problem stops existing

icefo 6y ago
I could only use snapshots and regularly send the pool somewhere but it feel a bit hacky
- jblwps 6y ago
  You don't send the whole pool--you send the datasets (ZFS filesystems) that you're concerned with.

iggldiggl 6y ago

I've been wondering about the very same problem (and stumbled across aryufan's patches as well), and discovered that they're now actually working on an expanded backup solution of their own: https://git.proxmox.com/?p=proxmox-backup.git;a=tree https://git.proxmox.com/?p=proxmox-backup.git;a=tree https://forum.proxmox.com/threads/proxmox-backup-server-and-client.68781/ https://forum.proxmox.com/threads/proxmox-backup-server-and-...

beagle3 6y ago

Use a CoW/snapshotting backing file system for your backup (btrfs/zfs), or one of the modern de-duplicating backup systems (bup/borg/restic), and the space taken will be proportional to the changed data, while still providing random-access to any snapshot -- Not much more you can expect in terms of space usage. snapshotting filesystems will be much more efficient in terms of time/effort compared to de-duplicating later, but other than that, all of these solutions are mostly equivalent.

icefo 6y ago
I don't argue that it would work but when you do that you're still sending all the data over some data link. For example if I have 1.5TB of VM and only 10-20Gb changed today I'm still sending everything to a remote storage. Snapshotting locally and sending the zfs volume seems to be a more efficient solution
- beagle3 6y ago
  That's not at all true for bup and borg, almost certainly not true for restic (Haven't used it myself and can't tell for sure). bup and borg (and likely restic) keep local hash summaries that are synchronized with the repository if it is remote. They would still have to _read_ the entire 1.5TB VMS if every vm file has changed (there are no facilities on a regular file system that they can use to figure out which parts changed), but they WILL use those local hashes to find out parts actually changed, and only send those changes to the remote storage. The local hashes typically occupy 0.5% of the data size (for bup, by default: 20 byte SHA1 per 8192 bytes on average; You can change the chunking parameters for bup/borg/restic for a tradeoff between cache size and granularity of change detected; for VMs it might make more sense to e.g. have 256KB chunks in which case you'll have 0.01% local storage overhead, and likely still negligible network transfer overhead. > I don't argue that it would work but when you do that you're still sending all the data over some data link. That's only true if you include e.g. the SATA link in that list; To avoid that, you need a filesystem that effectively tracks change ("damage") regions, or VM storage that uses lots of underlying files so you can track it on a regular file system (parallels on the mac used to do that so time machine backups are efficient). no way around it.
  - toomuchtodo 6y ago
    > there are no facilities on a regular file system that they can use to figure out which parts changed Doesn't ZFS do block level checksums that could be used for this, and backup tooling could use block pointer metadata for this purpopse?
    - 6y ago

blue1 6y ago

pve-zsync ?