7 ms·

so why exactly does somebody backup into a image with variable size, when they could just dd together a fixed size image that makes these free space calculation

so why exactly does somebody backup into a image with variable size, when they could just dd together a fixed size image that makes these free space calculations unnecessary?

tqkxzugoaupvwqr 9y ago
I use a sparse bundle disk image, i.e. a disk image with variable size and consisting of multiple files under the hood, because it is more efficient to back up over a network. Instead of uploading a 50 GB file to a cloud storage on every backup, only a fraction of data has to be uploaded (the sparse bundle’s changed files) which makes backing up the 50 GB file very fast if only a few megabytes were added. If anyone is curious: I use restic[1] as backup client and Backblaze B2[2] as backup storage. Works well with sparse bundles. [1] https://restic.net https://restic.net [2] https://www.backblaze.com/b2/cloud-storage.html https://www.backblaze.com/b2/cloud-storage.html
- woodfall 9y ago
  Wait, why would the 50 GB need to be transferred fully every time if it was a fixed size image?
  - tqkxzugoaupvwqr 9y ago
    The very first backup of a fixed size image will be the full size of the image, e.g. 50 GB, no matter which backup software you use. Even if inside the image no files exist. The very first backup of a sparse bundle disk image will be ~100 MB (the initial size of that image). On repeated backups, some backup softwares operate on file level and upload the whole file if it changed. So if you have a fixed size 50 GB image, mount it, add a file, unmount it, it changed, and the whole 50 GB image file has to be uploaded (with some backup softwares).
    - Nullabillity 9y ago
      Sure, but any halfway competent backup program would split it into chunks before deduplication anyway, rendering the exercise pointless.
pfranz 9y ago
I doubt I'd hit the bug described, but when I do a "clean install" I often create an encrypted sparse disk image onto a network volume and a copy on the Desktop of my new install. Unrelated, I do my bootable backup as a "smart backup" (only copy changes) to a usb-disk. I can see myself doing that to a sparse image on the network. As I set up my new computer I'll move stuff or delete things out of the sparse disk image from my Desktop, then periodically reclaim space. I only do this every 5+ years or so. I haven't done this with APFS, yet either. But those are a few of my use cases that dd wouldn't cover.
praseodym 9y ago
Variable size (sparse) disk images make it possible to backup e.g. two 1TB volumes that are on average <50% full to a single 1TB disk. Very useful to have for networked backups (Time Machine) of multiple machines.

so why exactly does somebody backup into a image with variable size, when they could just dd together a fixed size image that makes these free space calculations unnecessary?

tqkxzugoaupvwqr 9y ago

I use a sparse bundle disk image, i.e. a disk image with variable size and consisting of multiple files under the hood, because it is more efficient to back up over a network. Instead of uploading a 50 GB file to a cloud storage on every backup, only a fraction of data has to be uploaded (the sparse bundle’s changed files) which makes backing up the 50 GB file very fast if only a few megabytes were added. If anyone is curious: I use restic[1] as backup client and Backblaze B2[2] as backup storage. Works well with sparse bundles. [1] https://restic.net https://restic.net [2] https://www.backblaze.com/b2/cloud-storage.html https://www.backblaze.com/b2/cloud-storage.html

woodfall 9y ago
Wait, why would the 50 GB need to be transferred fully every time if it was a fixed size image?
- tqkxzugoaupvwqr 9y ago
  The very first backup of a fixed size image will be the full size of the image, e.g. 50 GB, no matter which backup software you use. Even if inside the image no files exist. The very first backup of a sparse bundle disk image will be ~100 MB (the initial size of that image). On repeated backups, some backup softwares operate on file level and upload the whole file if it changed. So if you have a fixed size 50 GB image, mount it, add a file, unmount it, it changed, and the whole 50 GB image file has to be uploaded (with some backup softwares).
  - Nullabillity 9y ago
    Sure, but any halfway competent backup program would split it into chunks before deduplication anyway, rendering the exercise pointless.

pfranz 9y ago

I doubt I'd hit the bug described, but when I do a "clean install" I often create an encrypted sparse disk image onto a network volume and a copy on the Desktop of my new install. Unrelated, I do my bootable backup as a "smart backup" (only copy changes) to a usb-disk. I can see myself doing that to a sparse image on the network. As I set up my new computer I'll move stuff or delete things out of the sparse disk image from my Desktop, then periodically reclaim space. I only do this every 5+ years or so. I haven't done this with APFS, yet either. But those are a few of my use cases that dd wouldn't cover.

praseodym 9y ago

Variable size (sparse) disk images make it possible to backup e.g. two 1TB volumes that are on average <50% full to a single 1TB disk. Very useful to have for networked backups (Time Machine) of multiple machines.