9 ms·

Sanoid – Replication and Backups with ZFS

125 points by mkj 7y ago

techslave 7y ago
yawn. hasn’t this been done a million times over?
- waz0wski 7y ago
  out of all the various zfs snapshot tools, this is one of the better ones, and the creator is also an active community member on various tech forums
- dang 7y ago
  Please don't be a jerk on HN. Also, note this guideline: "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - techslave 7y ago
    in my defense, i wasn’t dismissing his work. i was wondering why it merits HN attention. it’s completely ordinary stuff. the world needs ordinary stuff but this isn’t buzznews “the 3 surprising tricks about zfs snapshots”.
mckmk 7y ago
Sanoid seems like a great tool. It was at the top of my list when I was looking for this kind of solution. I ended up going with zfs_autobackup - https://github.com/psy0rz/zfs_autobackup https://github.com/psy0rz/zfs_autobackup because it doesn't need anything set up on the computer to backup. Just a script on the backup server that does pull backups. I'm not trying to hijack the thread but I've often appreciated links to similar software in threads like this.
- waz0wski 7y ago
  sanoid works great, used it for quite some time and the creator is actively involved in the zfs community in the spirit of link sharing for other tools, I've been moving many of my personal infrastructure & tools to golang-based apps, and I started using this for zfs replication https://github.com/zrepl/zrepl https://github.com/zrepl/zrepl No issues thus far in about 6mo of usage for local-and-remote snapshots
- Youden 7y ago
  The issue I have with that is that if everything is done by the backup server, it has the ability to both create and destroy (prune) datasets (snapshots), which means it has the ability to erase all of the data and all of the backups. One thing I like about Sanoid/Syncoid is that if you set up Sanoid on both sides and use Syncoid to pull, the backup server only requires the ability to send snapshots (which you can configure with "zfs allow"). The backup server also doesn't require root in this case. This means neither server can delete the other's data. And you still get all the automatic snapshot creation (done on the primary server) and pruning (done on both servers).
linsomniac 7y ago
This looks pretty solid. On one of my machines I'm using "zfs-auto-snapshot", which works great for something I can just install and not have to manage, for getting some history. For replication of snapshots, I haven't done that recently, but in the past I just used some scripts I had written to do one-time migrations. That was a decade ago. This looks much more capable than those scripts.
- kchr 7y ago
  Yeah, there are base templates for configuration and they can be overridden on a single (or recursively on all children of a) dataset where needed.
kchr 7y ago
Great piece of software; the templating model for configuration is neat and works out well for my case.
rsync 7y ago
This is the piece I was looking for: "Which would push-replicate the specified ZFS filesystem from the local host to remotehost over an SSH tunnel ..." syncoid root@remotehost:data/images/vm backup/images/vm Now, if only there were a cloud storage provider that you could directly zfs-send to ... If only ...
- dangerboysteve 7y ago
  like rsync.net ? this is just storage.
  - icebraining 7y ago
    The previous user runs rsync.net, he's pushing his service.
    - dangerboysteve 7y ago
      Totally missed that.
- dmd 7y ago
  If only there were one that were a reasonable price increment on top of raw storage, like 1.5x or even 2x, instead of 5x what, say, S3 or B2 costs.
  - cpach 7y ago
    Feel free to start a competitor…
    - papyruskaleido 7y ago
      It's probably easier and cheaper to just use Sanoid though.
  - blattimwind 7y ago
    Remote access to a ZFS gives you much different guarantees from your average S3-lookalike, so an integer price difference does not seem unreasonable to me.
beagle3 7y ago
Only tangentially related, but seems like a good thread to ask: Does ZFS (or any other FOSS file system) have a setup like Isilon or NetApp, where you could have any 2 disks in a box of 8 or any box out of say 6 die, and still not lose anything other than bandwidth?
- acd 7y ago
  raidz2 for RAID6 like support where 2 disks can die.
- oarsinsync 7y ago
  RAIDZ2 / RAID6 (through linux md-raid, or anything else) both handle the 'lose 2 out of N+2 disks' case.
- wtallis 7y ago
  If you want something that deals with multiple tiers of fault tolerance, you should be looking at distributed filesystems (eg. Ceph) rather than single-system filesystems like ZFS and btrfs. Also keep an eye out for erasure coding as a keyword, since that usually implies more control over the degree of redundancy than a handful of RAID levels.
- antongribok 7y ago
  I would say ZFS would be a lot closer to NetApp. For Isilon, a better comparison would be Ceph. With Ceph you can have a cluster of 10s or 100s of servers, and you can lose 1 disk, or 1 server, or 1 rack or servers and it will recover very quickly. As long as your minimum requirements for your type of replication or erasure coding profile are satisfied, your data stays safe and available.
grafelic 7y ago
I've been using it for my daily zfs snapshotting needs for a year. Rock solid software. Thanks.
timeattack 7y ago
Another one, which creates backups on locally connected backup pool — https://github.com/reconquest/zeus https://github.com/reconquest/zeus
polskibus 7y ago
Is this a good fit for an on-prem small data center backup solution?
crashbunny 7y ago
sanoid is great. I asked how to stop sanoid from snapshotting docker created datasets in rpool/var/lib/docker when rpool has the recursive flag, and they added a feature to do it the next day.
kuon 7y ago
What advantages does it have over zrep?
hartzell 7y ago
For FreeBSD users, it's available from ports: https://www.freshports.org/sysutils/sanoid/ https://www.freshports.org/sysutils/sanoid/ (I'm the port maintainer, open to improvements)
e12e 7y ago
Oh my. It's perl. Might be worth noting. For those that actually use perl - I know perl has had (fairly sane?) package management for a while... Is it reasonable to use system perl version in anger? I would typically reach for a version manager/virtual env with python or ruby - do you, with perl as well? I'm fairly happy with asdf day to day. https://asdf-vm.com/ https://asdf-vm.com/
paxswill 7y ago
I love using Sanoid+Syncoid for keeping my laptop (with Xubuntu on ZFS) backed up to my NAS (running FreeNAS). In addition to the periodic snapshots Sanoid manages, I also have a small script (shameless plug [0]) that hooks into APT to take snapshots right before upgrades occur. It's already saved me decent chunk of time that would've been spent rebuilding from a botched upgrade. 0: https://github.com/paxswill/zfs-apt-snapshot https://github.com/paxswill/zfs-apt-snapshot
parshimers 7y ago
I wanted to use this tool for backing up my laptop to my NAS, however it only allows SSH to be used for data transfer, which is a real bottleneck for speed. It basically means you can only transfer as fast as one core can decrypt or encrypt data on either end of the pipe. I ended up using https://github.com/oetiker/znapzend https://github.com/oetiker/znapzend instead, because it can use mbuffer for the actual data transfer, which is much faster.
- kogir 7y ago
  Is SSH speed a real limitation you’ve measured? I was able to easily saturate a 1 Gbps link for days using ZFS send over SSH with an Intel Atom when initializing my remote backups.
  - parshimers 7y ago
    Yes. I have an Atom on one end of the pipe as well, but it's over 10Gbe and to a 6 drive mirrored and striped vdev. Using SSH as the transport it would only approach 100MB/s, using mbuffer it can max out the storage at about 250-275MB/s.
r1ch 7y ago
I've used a few different ZFS snapshot management tools, the one feature I'm missing is something to assist with data retrieval. I'd like to be able to specify a dataset, path and date/time and the tool would mount the closest snapshot to the specified time and pull the appropriate data for me.
- rainbowzootsuit 7y ago
  Not exactly what you're asking I think, but in Solaris-derived OSs the snapshots are exposed by default as previous versions in windows. You can also use SAMBA to expose the snapshots as previous versions in windows. SAMBA general documentation: https://www.samba.org/samba/docs/current/man-html/vfs_shadow_copy2.8.html https://www.samba.org/samba/docs/current/man-html/vfs_shadow... Randomly found ZFS specific: https://blog.chaospixel.com/linux/2017/09/zfs-auto-snapshots-and-windows-shadow-copies-with-samba.html https://blog.chaospixel.com/linux/2017/09/zfs-auto-snapshots...

125 points by mkj 7y ago

techslave 7y ago

yawn. hasn’t this been done a million times over?

waz0wski 7y ago
out of all the various zfs snapshot tools, this is one of the better ones, and the creator is also an active community member on various tech forums
dang 7y ago
Please don't be a jerk on HN. Also, note this guideline: "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- techslave 7y ago
  in my defense, i wasn’t dismissing his work. i was wondering why it merits HN attention. it’s completely ordinary stuff. the world needs ordinary stuff but this isn’t buzznews “the 3 surprising tricks about zfs snapshots”.

mckmk 7y ago

Sanoid seems like a great tool. It was at the top of my list when I was looking for this kind of solution. I ended up going with zfs_autobackup - https://github.com/psy0rz/zfs_autobackup https://github.com/psy0rz/zfs_autobackup because it doesn't need anything set up on the computer to backup. Just a script on the backup server that does pull backups. I'm not trying to hijack the thread but I've often appreciated links to similar software in threads like this.

waz0wski 7y ago
sanoid works great, used it for quite some time and the creator is actively involved in the zfs community in the spirit of link sharing for other tools, I've been moving many of my personal infrastructure & tools to golang-based apps, and I started using this for zfs replication https://github.com/zrepl/zrepl https://github.com/zrepl/zrepl No issues thus far in about 6mo of usage for local-and-remote snapshots
Youden 7y ago
The issue I have with that is that if everything is done by the backup server, it has the ability to both create and destroy (prune) datasets (snapshots), which means it has the ability to erase all of the data and all of the backups. One thing I like about Sanoid/Syncoid is that if you set up Sanoid on both sides and use Syncoid to pull, the backup server only requires the ability to send snapshots (which you can configure with "zfs allow"). The backup server also doesn't require root in this case. This means neither server can delete the other's data. And you still get all the automatic snapshot creation (done on the primary server) and pruning (done on both servers).

linsomniac 7y ago

This looks pretty solid. On one of my machines I'm using "zfs-auto-snapshot", which works great for something I can just install and not have to manage, for getting some history. For replication of snapshots, I haven't done that recently, but in the past I just used some scripts I had written to do one-time migrations. That was a decade ago. This looks much more capable than those scripts.

kchr 7y ago
Yeah, there are base templates for configuration and they can be overridden on a single (or recursively on all children of a) dataset where needed.

kchr 7y ago

Great piece of software; the templating model for configuration is neat and works out well for my case.

rsync 7y ago

This is the piece I was looking for: "Which would push-replicate the specified ZFS filesystem from the local host to remotehost over an SSH tunnel ..." syncoid root@remotehost:data/images/vm backup/images/vm Now, if only there were a cloud storage provider that you could directly zfs-send to ... If only ...

dangerboysteve 7y ago
like rsync.net ? this is just storage.
- icebraining 7y ago
  The previous user runs rsync.net, he's pushing his service.
  - dangerboysteve 7y ago
    Totally missed that.
dmd 7y ago
If only there were one that were a reasonable price increment on top of raw storage, like 1.5x or even 2x, instead of 5x what, say, S3 or B2 costs.
- cpach 7y ago
  Feel free to start a competitor…
  - papyruskaleido 7y ago
    It's probably easier and cheaper to just use Sanoid though.
- blattimwind 7y ago
  Remote access to a ZFS gives you much different guarantees from your average S3-lookalike, so an integer price difference does not seem unreasonable to me.

beagle3 7y ago

Only tangentially related, but seems like a good thread to ask: Does ZFS (or any other FOSS file system) have a setup like Isilon or NetApp, where you could have any 2 disks in a box of 8 or any box out of say 6 die, and still not lose anything other than bandwidth?

acd 7y ago
raidz2 for RAID6 like support where 2 disks can die.
oarsinsync 7y ago
RAIDZ2 / RAID6 (through linux md-raid, or anything else) both handle the 'lose 2 out of N+2 disks' case.
wtallis 7y ago
If you want something that deals with multiple tiers of fault tolerance, you should be looking at distributed filesystems (eg. Ceph) rather than single-system filesystems like ZFS and btrfs. Also keep an eye out for erasure coding as a keyword, since that usually implies more control over the degree of redundancy than a handful of RAID levels.
antongribok 7y ago
I would say ZFS would be a lot closer to NetApp. For Isilon, a better comparison would be Ceph. With Ceph you can have a cluster of 10s or 100s of servers, and you can lose 1 disk, or 1 server, or 1 rack or servers and it will recover very quickly. As long as your minimum requirements for your type of replication or erasure coding profile are satisfied, your data stays safe and available.

grafelic 7y ago

I've been using it for my daily zfs snapshotting needs for a year. Rock solid software. Thanks.

timeattack 7y ago

Another one, which creates backups on locally connected backup pool — https://github.com/reconquest/zeus https://github.com/reconquest/zeus

polskibus 7y ago

Is this a good fit for an on-prem small data center backup solution?

crashbunny 7y ago

sanoid is great. I asked how to stop sanoid from snapshotting docker created datasets in rpool/var/lib/docker when rpool has the recursive flag, and they added a feature to do it the next day.

kuon 7y ago

What advantages does it have over zrep?

hartzell 7y ago

For FreeBSD users, it's available from ports: https://www.freshports.org/sysutils/sanoid/ https://www.freshports.org/sysutils/sanoid/ (I'm the port maintainer, open to improvements)

e12e 7y ago

Oh my. It's perl. Might be worth noting. For those that actually use perl - I know perl has had (fairly sane?) package management for a while... Is it reasonable to use system perl version in anger? I would typically reach for a version manager/virtual env with python or ruby - do you, with perl as well? I'm fairly happy with asdf day to day. https://asdf-vm.com/ https://asdf-vm.com/

paxswill 7y ago

I love using Sanoid+Syncoid for keeping my laptop (with Xubuntu on ZFS) backed up to my NAS (running FreeNAS). In addition to the periodic snapshots Sanoid manages, I also have a small script (shameless plug [0]) that hooks into APT to take snapshots right before upgrades occur. It's already saved me decent chunk of time that would've been spent rebuilding from a botched upgrade. 0: https://github.com/paxswill/zfs-apt-snapshot https://github.com/paxswill/zfs-apt-snapshot

parshimers 7y ago

I wanted to use this tool for backing up my laptop to my NAS, however it only allows SSH to be used for data transfer, which is a real bottleneck for speed. It basically means you can only transfer as fast as one core can decrypt or encrypt data on either end of the pipe. I ended up using https://github.com/oetiker/znapzend https://github.com/oetiker/znapzend instead, because it can use mbuffer for the actual data transfer, which is much faster.

kogir 7y ago
Is SSH speed a real limitation you’ve measured? I was able to easily saturate a 1 Gbps link for days using ZFS send over SSH with an Intel Atom when initializing my remote backups.
- parshimers 7y ago
  Yes. I have an Atom on one end of the pipe as well, but it's over 10Gbe and to a 6 drive mirrored and striped vdev. Using SSH as the transport it would only approach 100MB/s, using mbuffer it can max out the storage at about 250-275MB/s.

r1ch 7y ago

I've used a few different ZFS snapshot management tools, the one feature I'm missing is something to assist with data retrieval. I'd like to be able to specify a dataset, path and date/time and the tool would mount the closest snapshot to the specified time and pull the appropriate data for me.

rainbowzootsuit 7y ago
Not exactly what you're asking I think, but in Solaris-derived OSs the snapshots are exposed by default as previous versions in windows. You can also use SAMBA to expose the snapshots as previous versions in windows. SAMBA general documentation: https://www.samba.org/samba/docs/current/man-html/vfs_shadow_copy2.8.html https://www.samba.org/samba/docs/current/man-html/vfs_shadow... Randomly found ZFS specific: https://blog.chaospixel.com/linux/2017/09/zfs-auto-snapshots-and-windows-shadow-copies-with-samba.html https://blog.chaospixel.com/linux/2017/09/zfs-auto-snapshots...