7 ms·

It remains a _delightfully_ straightforward way to do backups; I do incremental zfs send with hourly snapshots to my local NAS and also to an rsync.net ZFS acco

It remains a _delightfully_ straightforward way to do backups; I do incremental zfs send with hourly snapshots to my local NAS and also to an rsync.net ZFS account (https://www.rsync.net/products/zfsintro.html); https://www.rsync.net/products/zfsintro.html); it's incredibly resource un-intensive relative to your traditional scan-everything backup methods and I can trivially mount a past snapshot for browsing or spot restore or just do zfs send on the whole thing for catastrophic recovery.

throw0101a 5y ago
As an added bonus, with (Open)ZFS encryption, you can send the 'raw' encrypted zfs send stream (over an encrypted SSH connection) to a destination where the file system is encrypted and where the zfs receive command does not have to decrypt it to save things. That is, the remote destination does not need to have the key/passphrase to the file system to have an (encrypted) copy of the data. If your production server goes down, you can restore the encrypted file system without the encryption key, and only when you try to mount the restored ZFS file system will you be prompted for the key/passphrase. > This means that you can use ZFS replication to back up your data to an untrusted location, without concerns about your private data being read. With raw send, your data is replicated without ever being decrypted—and without the backup target ever being able to decrypt it at all. This means you can replicate your offsite backups to a friend's house or at a commercial service like rsync.net or zfs.rent without compromising your privacy, even if the service (or friend) is itself compromised. * https://arstechnica.com/gadgets/2021/06/a-quick-start-guide-to-openzfs-native-encryption/ https://arstechnica.com/gadgets/2021/06/a-quick-start-guide-...
- cyberpunk 5y ago
  Do you happen to know if rsync.net supports this?
  - throw0101a 5y ago
    They support zfs send-recv and snapshots: * https://www.rsync.net/products/zfsintro.html https://www.rsync.net/products/zfsintro.html * https://www.rsync.net/resources/howto/snapshots.html https://www.rsync.net/resources/howto/snapshots.html Whether their ZFS implementation supports OpenZFS encryption is something you'll have to e-mail them about. They have an HN account: * https://news.ycombinator.com/user?id=rsync https://news.ycombinator.com/user?id=rsync Edit: Yes, it seems that they do: * https://news.ycombinator.com/item?id=26506263 https://news.ycombinator.com/item?id=26506263
  - rsync 5y ago
    "Do you happen to know if rsync.net supports this?" Yes, we do. If you get a zpool from us (which is a bit different than a "normal" rsync.net account) it is running the latest stable ZoL codebase and, thus, supports encryption and "raw send", etc. However ... As elegant and performant as 'zfs send' is, the benefits are realized in large datasets where efficiency and performance really matters. If you're just sending 200 or 400 or 800 GB of data to an untrusted destination (like rsync.net) you should probably just use borg[1][2]. [1] https://borgbackup.readthedocs.io/en/stable/ https://borgbackup.readthedocs.io/en/stable/ [2] https://www.stavros.io/posts/holy-grail-backups/ https://www.stavros.io/posts/holy-grail-backups/
    - cyberpunk 5y ago
      Hmm, well currently all my machines, phones etc backup to a nas (zol) which I then reclone up to an encrypted b2 bucket, but tbh the restores suck and they charge for traffic so I was contemplating just throwing em on rsync.net, you recon Borg->rs.n is the way to go instead of zfs send? Under 2tb..
      - rsync 5y ago
        Well, it's really up to you but let me put it this way ... In order to give you a zpool of your own we need to give you a full blown VM (bhyve) with resource guarantees and your own IP address, etc. So there is a 1TB minimum and no discounts. Alternatively, if you just get a plain old rsync.net account the minimum account size is much smaller and if you're an expert and don't need any (borg specific) support there is a discounted plan[1]. [1] https://rsync.net/products/borg.html https://rsync.net/products/borg.html
        infogulch 5y ago
        Interesting I wasn't aware of the borg discount pricing, that may change my default position. I now know of three pricing schemes, basic, ZFS, and borg, the last two of which don't seem to be listed anywhere on the site. Got any other products hiding behind my ignorance? :) If OpenZFS supported some kind of "virtual zpool" feature (I guess it doesn't exist yet) that you could provision out of your main ZFS infrastructure and hand off to the customer, would that be useful to reduce the resource overhead related to offering ZFS directly?
rubatuga 5y ago
It's so handy! And I have personally tested a restore using it. It's a good idea to check out the read-only flags so you don't accidentally touch the file-system and invalidate an incremental backup.

throw0101a 5y ago

As an added bonus, with (Open)ZFS encryption, you can send the 'raw' encrypted zfs send stream (over an encrypted SSH connection) to a destination where the file system is encrypted and where the zfs receive command does not have to decrypt it to save things. That is, the remote destination does not need to have the key/passphrase to the file system to have an (encrypted) copy of the data. If your production server goes down, you can restore the encrypted file system without the encryption key, and only when you try to mount the restored ZFS file system will you be prompted for the key/passphrase. > This means that you can use ZFS replication to back up your data to an untrusted location, without concerns about your private data being read. With raw send, your data is replicated without ever being decrypted—and without the backup target ever being able to decrypt it at all. This means you can replicate your offsite backups to a friend's house or at a commercial service like rsync.net or zfs.rent without compromising your privacy, even if the service (or friend) is itself compromised. * https://arstechnica.com/gadgets/2021/06/a-quick-start-guide-to-openzfs-native-encryption/ https://arstechnica.com/gadgets/2021/06/a-quick-start-guide-...

cyberpunk 5y ago
Do you happen to know if rsync.net supports this?
- throw0101a 5y ago
  They support zfs send-recv and snapshots: * https://www.rsync.net/products/zfsintro.html https://www.rsync.net/products/zfsintro.html * https://www.rsync.net/resources/howto/snapshots.html https://www.rsync.net/resources/howto/snapshots.html Whether their ZFS implementation supports OpenZFS encryption is something you'll have to e-mail them about. They have an HN account: * https://news.ycombinator.com/user?id=rsync https://news.ycombinator.com/user?id=rsync Edit: Yes, it seems that they do: * https://news.ycombinator.com/item?id=26506263 https://news.ycombinator.com/item?id=26506263
- rsync 5y ago
  "Do you happen to know if rsync.net supports this?" Yes, we do. If you get a zpool from us (which is a bit different than a "normal" rsync.net account) it is running the latest stable ZoL codebase and, thus, supports encryption and "raw send", etc. However ... As elegant and performant as 'zfs send' is, the benefits are realized in large datasets where efficiency and performance really matters. If you're just sending 200 or 400 or 800 GB of data to an untrusted destination (like rsync.net) you should probably just use borg[1][2]. [1] https://borgbackup.readthedocs.io/en/stable/ https://borgbackup.readthedocs.io/en/stable/ [2] https://www.stavros.io/posts/holy-grail-backups/ https://www.stavros.io/posts/holy-grail-backups/
  - cyberpunk 5y ago
    Hmm, well currently all my machines, phones etc backup to a nas (zol) which I then reclone up to an encrypted b2 bucket, but tbh the restores suck and they charge for traffic so I was contemplating just throwing em on rsync.net, you recon Borg->rs.n is the way to go instead of zfs send? Under 2tb..
    - rsync 5y ago
      Well, it's really up to you but let me put it this way ... In order to give you a zpool of your own we need to give you a full blown VM (bhyve) with resource guarantees and your own IP address, etc. So there is a 1TB minimum and no discounts. Alternatively, if you just get a plain old rsync.net account the minimum account size is much smaller and if you're an expert and don't need any (borg specific) support there is a discounted plan[1]. [1] https://rsync.net/products/borg.html https://rsync.net/products/borg.html
      - infogulch 5y ago
        Interesting I wasn't aware of the borg discount pricing, that may change my default position. I now know of three pricing schemes, basic, ZFS, and borg, the last two of which don't seem to be listed anywhere on the site. Got any other products hiding behind my ignorance? :) If OpenZFS supported some kind of "virtual zpool" feature (I guess it doesn't exist yet) that you could provision out of your main ZFS infrastructure and hand off to the customer, would that be useful to reduce the resource overhead related to offering ZFS directly?

rubatuga 5y ago

It's so handy! And I have personally tested a restore using it. It's a good idea to check out the read-only flags so you don't accidentally touch the file-system and invalidate an incremental backup.