7 ms·
Beware that this script only uses rsync with the "--archive" flag. This may be enough for some users, but "--archive" does not copy all file metadata, so it ma
by adrian_b 5y ago
Beware that this script only uses rsync with the "--archive" flag.
This may be enough for some users, but "--archive" does not copy all file metadata, so it may cause surprises.
rsync must be invoked with "--archive --xattrs --acls" to guarantee complete file copies.
Unfortunately all command-line utilities for file copying that are available for UNIX-like operating systems use default options that do not copy most file metadata and they usually need many non-default flags to make complete file copies.
Nevertheless, rsync is the best of all alternatives, because when invoked correctly it accurately copies all file metadata even between different file systems and different operating systems, in cases when other copying utilities may lose some file metadata without giving any errors or warnings for the users.
- chronomex 5y agoRsync also doesn't track hardlinks by default! I use "-avPzHAXS" for my backups.
- Jenda_ 5y agoAlso have a look at --numeric-ids, otherwise it translates UIDs back and forth according to real user names - which will end up in a terrible mess, especially when you are restoring the backup from a live "CD" (which has different UID ←→ username mapping than the target system).
- amelius 5y agoI use getfacl to save permissions correctly, in case the backup server uses different user/group mappings. And setfacl after restoring. (I wish this was handled by rsync.)
- deleted 5y ago[deleted]
- throwawayboise 5y agoThis also presumes that the target's rsync implementation and filesystem supports the same metadata.
- adrian_b 5y agoMost modern file systems have been updated to support equivalent metadata. However many old versions of file systems lacked support for things like high-resolution timestamps, extended attributes or access-control lists. Because such support has been introduced much later, there are a lot of programs for archiving, backup and copying which lose such metadata, at least with their default options. While equivalent metadata exists on Linux XFS and EXT4, Windows NTFS, FreeBSD UFS and ZFS etc. each platform has different APIs for accessing file metadata (and on Linux the API may vary even between the many available file systems), so only few programs, like rsync, are able to copy everything while taking care to use the correct API to replicate the metadata without losing information.
- chalst 5y agoBetter than rsync is using support for snapshots and replication in the filesystem, such as described for ZFS in https://serverfault.com/questions/842531/how-to-perform-incremental-continuous-backups-of-zfs-pool https://serverfault.com/questions/842531/how-to-perform-incr... Btrfs and apfs also support auto-snapshotting and replication, but I don't have experience with them.
- colejohnson66 5y agoZFS snapshots also have the added benefit of not taking up duplicate space. And, should a Linux ransomware ever come into existence, snapshots help against them.
- Quasimarion 5y agoBtrfs and apfs also support copy-on-write snapshots. Also, snapshots are NOT backups. And one definitely should not rely on snapshots along to keep their files safe.
- colejohnson66 5y agoThey aren’t; a backup should preferably be offsite, but nothing is perfect
- tombrossman 5y agoFor years I had a custom script sync my ~/.ssh directory on my primary workstation to my laptop, to pick up new keys and config changes. It failed after I switched from Ubuntu to Fedora, and I was surprised to discover --xattrs fixed it. tl;dr try this if rsync fails in unexpected ways: echo "Syncing ~/.ssh directory" rsync --archive --delete --xattrs ~/.ssh/ laptop:.ssh/
- uhradone 5y agoHey Adrian, thanks for the improvements and hints!
- ranger_danger 5y agoProbably because 99% of users do not have or need xattrs/acls.