4 ms·

I have been using ZFS on my daily-driver Ubuntu laptop for six months or so. Also been using ZFS as the backing store for a heavy update postgres database for a

I have been using ZFS on my daily-driver Ubuntu laptop for six months or so. Also been using ZFS as the backing store for a heavy update postgres database for a few years before that. Haven't had any issues with it at all, generally it seems like a very solid bit of software. I'm not familiar with Deja dup, but the usual method of backing up a ZFS filesystem is to snapshot it and then stream the snapshot (just a file, can be compressed) into the remote store of your choice.

Octabrain 4y ago
Thank you for answering! Deja dup it's just a GUI over duplicity (https://en.wikipedia.org/wiki/Duplicity_(software) https://en.wikipedia.org/wiki/Duplicity_(software)). It allows me to have encryption, differential backups and restoring files in a granular way (even from Gnome context menu) with no effort. Very pleasant experience similar to Time machine in MacOS. Anyways, I'll definitely give it a go to ZFS.
- bo0tzz 4y ago
  ZFS can do all those same things!
  - Octabrain 4y ago
    I'll give it a go for sure. It's gonna be an exciting evening!
aborsy 4y ago
Streaming ZFS snapshots to a file and storing that in a remote of your choice is not the usual way of backing up ZFS data at all (it’s actually recommended against). The usual way is ZFS send and receive to another ZFS server. But almost no cloud storage provider supports that, other than rsync.net (and zfs.rent), which starts with $60/month. So you have to set up and manage another ZFS server with secure remote access, PIA if you ask me.
- gnfargbl 4y ago
  Interesting. Who is making that recommendation, and why? For a while now, my backup process has been essentially to run zfs send -L -c -P "${snap}" | zstd - > "${out}" gsutil -o GSUtil:parallel_composite_upload_threshold=150M cp "${out}" "${gcs_path}/" It seems to work well. Am I carrying some unrecognised risk that I wouldn't be carrying if I were backing up into another ZFS filesystem? How does that risk balance against the risk of data loss on the backup server? i.e. I am pretty sure GCS won't lose my data, but not quite so sure about the other services you mention.
  - 7e 4y ago
    If your monolithic backup suffers single a bit error, it's unrecoverable. If you zfs recv, though, you can recover all pristine files. And if you zfs recv to a redundant filesystem (mirror, RAID-Z), you can periodically zfs scrub and recover from those errors completely.
    - gnfargbl 4y ago
      Hmmn, I wasn't aware of that behaviour. I think this is the correct answer: thank you.
      - radiowave 4y ago
        I've been running ZFS continually since about 2009, currently using its replication as a major part of the disaster recovery strategy for my employer, and in the intervening time having been through a number of disk failures, controller failures, etc. Just chiming in here to say that I endorse every aspect of the answer that 7e has given you.
  - aborsy 4y ago
    ZFS receive checks the stream to ensure correctness. The stream can be easily corrupted, due to even a single error, if ZFS send is used without ZFS receive. The error occurs in transit or on client side, not necessarily on remote. Also managing a chain of incrementals can get unwieldy. Ask r/zfs.

Octabrain 4y ago

Thank you for answering! Deja dup it's just a GUI over duplicity (https://en.wikipedia.org/wiki/Duplicity_(software) https://en.wikipedia.org/wiki/Duplicity_(software)). It allows me to have encryption, differential backups and restoring files in a granular way (even from Gnome context menu) with no effort. Very pleasant experience similar to Time machine in MacOS. Anyways, I'll definitely give it a go to ZFS.

bo0tzz 4y ago
ZFS can do all those same things!
- Octabrain 4y ago
  I'll give it a go for sure. It's gonna be an exciting evening!

aborsy 4y ago

Streaming ZFS snapshots to a file and storing that in a remote of your choice is not the usual way of backing up ZFS data at all (it’s actually recommended against). The usual way is ZFS send and receive to another ZFS server. But almost no cloud storage provider supports that, other than rsync.net (and zfs.rent), which starts with $60/month. So you have to set up and manage another ZFS server with secure remote access, PIA if you ask me.

gnfargbl 4y ago
Interesting. Who is making that recommendation, and why? For a while now, my backup process has been essentially to run zfs send -L -c -P "${snap}" | zstd - > "${out}" gsutil -o GSUtil:parallel_composite_upload_threshold=150M cp "${out}" "${gcs_path}/" It seems to work well. Am I carrying some unrecognised risk that I wouldn't be carrying if I were backing up into another ZFS filesystem? How does that risk balance against the risk of data loss on the backup server? i.e. I am pretty sure GCS won't lose my data, but not quite so sure about the other services you mention.
- 7e 4y ago
  If your monolithic backup suffers single a bit error, it's unrecoverable. If you zfs recv, though, you can recover all pristine files. And if you zfs recv to a redundant filesystem (mirror, RAID-Z), you can periodically zfs scrub and recover from those errors completely.
  - gnfargbl 4y ago
    Hmmn, I wasn't aware of that behaviour. I think this is the correct answer: thank you.
    - radiowave 4y ago
      I've been running ZFS continually since about 2009, currently using its replication as a major part of the disaster recovery strategy for my employer, and in the intervening time having been through a number of disk failures, controller failures, etc. Just chiming in here to say that I endorse every aspect of the answer that 7e has given you.
- aborsy 4y ago
  ZFS receive checks the stream to ensure correctness. The stream can be easily corrupted, due to even a single error, if ZFS send is used without ZFS receive. The error occurs in transit or on client side, not necessarily on remote. Also managing a chain of incrementals can get unwieldy. Ask r/zfs.