3 ms·
"For a simple tool, use rsnapshot ..." Even simpler is to do a dumb, 1:1 mirror, with rsync and then rotate zfs snapshots on the server side. Two benefits her
by rsync 6y ago
"For a simple tool, use rsnapshot ..."
Even simpler is to do a dumb, 1:1 mirror, with rsync and then rotate zfs snapshots on the server side.
Two benefits here over rsnapshot (or rsync snapshots, with hardlinks) are:
1) rsync snapshots diff on a file granularity - if you change one bit of the file, your penalty is the storage space for that entire file. ZFS, on the other hand, diffs on a bits level so your penalty is just the changed portions of that paticular file.
2) Depending on your implementation, ZFS snapshots are immutable/read-only ... so neither a misconfiguration nor ransomware/mallory can destroy those backups if you lose your credentials.
Historically, people would create and maintain "rsync snapshots" inside their rsync.net accounts using 'cp -al' and other unix commands run over SSH.[1]
After transitioning to ZFS, however, we encourage all customers to just do a "dumb" 1:1 rsync to us and let us maintain and rotate their preferred snapshot schedule (days/weeks/months/quarters/years).
[1] https://www.rsync.net/resources/howto/remote_commands.html https://www.rsync.net/resources/howto/remote_commands.html
- derekp7 6y agoJust curious, how do you sync file ownership and permissions when going to a ZFS target server with a non privliged account? Or would you have all the files on the target owned by the user ID accessing the target server, then keep the metadata (owner/mode/SE Linux attributes / ACLs, etc) in a separate log file? And do you typically turn on ZFS compression? I tried that years ago with ZFS under Linux, but had major slowdowns after a while. But I'm sure it has improved quite a bit in more recent years. Also how does ZFS stack up against Btrfs in your experience? But for a direct-to-cloud backup, the way you do it is really great -- to me it is much easier than messing with things like S3 buckets and similar technoligies.
- rsync 6y agoWell, if you are using a sophisticated tool like restic or borg or duplicity, the file attributes are bundled up into the encrypted chunks and all is well. However, if you are using plain old rsync, you can use the --fake-super command line switch to maintain those attributes and I believe rsync.net customers are using that successfully. Yes, you are correct - in the old days (2001-2010 or so) we maintained a pair of "permission scripts" which allowed customers to dump all their attributes to a file and then reapply it later. Again, --fake-super seems to "fix" all of this. Generally speaking, you should always enable ZFS encryption. It improves many aspects of ZFS behavior and performance - not just space usage. Unless you have very bizarre workloads or hardware, I think that in 2020 there is no reason to not enable compression. Deduplication is another story and is very expensive to maintain - rsync.net has never run de-dupe on any systems. We (rsync.net) and I (personally) have zero experience with btrfs ...
- onetimertwo 6y agoThat shold be "you should always enable ZFS compression"?
- rsync 6y agoOh, sorry - yes, you are correct: s/encryption/compression