4 ms·

rsync is a strange beast to me First thing that makes it so weird is how it assigns different meaning between paths including vs not including trailing slash.

rsync is a strange beast to me First thing that makes it so weird is how it assigns different meaning between paths including vs not including trailing slash. Completely different from how most command line tools I am used to behave in Linux and FreeBSD. That alone is enough to remind me every time I try to use rsync why I don’t like and generally don’t use rsync.

pastage 3y ago
Lost too much data that way, trailing slash with delete. I still feel bitter, the UX is terrible considering the effect are so different from copy and delete.
pessimizer 3y ago
If it didn't do that, it would have to add a switch that you'd still have to look up. It's a tool that's most often doing a merge and update, but looks like a copy command. I think that made it friendlier. How would you separate "merge this directory into that one and update files with the same name" from "copy this directory into that directory"?
- Karunamon 3y ago
  Halt with an error message explaining the ambiguity and the proper switch to use. Same principle as rm requiring --no-preserve-root if you actually want to nuke / - right now it's way too easy to accidentally and destructively do the wrong thing.
- codetrotter 3y ago
  > How would you separate "merge this directory into that one and update files with the same name" from "copy this directory into that directory"? I would make "copy this directory into that directory" out of scope for the tool. Let’s imagine a tool similar to rsync, but less confusing, and more in tune with what I want to do, personally. There are for sure a bunch of things that rsync can do, that this imagined tool can’t. That’s fine by me. Let’s call the tool nsync. It would work like this: nsync /some/src/dir /some/dest/dir And running that would behave exactly the same with or without trailing slashes. I.e the above and the following would all be equivalent: nsync /some/src/dir/ /some/dest/dir/ nsync /some/src/dir/ /some/dest/dir nsync /some/src/dir /some/dest/dir/ And what would this do? It would inspect source and dest dirs. Then it would copy files from source to dest for which last modified was greater in source dir than in dest, or where files in source dir did not exist in dest dir. In other words, it would overwrite older files that had older last modified time stamp, and it would copy files that did not exist. Like rsync it would also work with ssh (scp/sftp). Maybe some other protocols too, but only if those other protocols supported the comparisons we need to make. Prefer fewer protocols, and this way of working over trying to be the subset of what works across a gazillion protocols. If a file exists in dest but not in source dir, it is kept untouched in dest. Not deleted. Not copied back to source dir. Then there would be one other mode; destructive mode. The flag for it would be -d. nsync -d /whatever/a/b/c/ /wherever/x/y/z/ This would work similar to the normal mode. But it would remove any files in dest dir that are not present in source dir. Before actually deleting anything it would list all of the files that will be deleted, and ask for keyboard confirmation. [y/N] so that you have to explicitly hit y and then enter. Enter alone will be interpreted as no. You would be able to override the confirmation with the -y argument. nsync -dy /whatever/a/b/c/ /wherever/x/y/z/ And that’s it. That’s what I would want rsync to be for me. There probably are some programs that behave exactly like this. I’ll eventually write one too. It’ll have a user base of 1. Me.
  - xk3 3y ago
    rclone move will do this https://github.com/chapmanjacobd/journal/blob/main/programming/linux/misconceptions.md#mv-src-vs-mv-src https://github.com/chapmanjacobd/journal/blob/main/programmi...
ape4 3y ago
Yes, a complete redesign of the rsync command line would be great.
labawi 3y ago
While rsync is different than cp and mv, I dislike the cp and mv destination-state-dependent behaviour. With rsync, destination paths are defined by the command itself and the command is idempotent. I don't usually need to know what the destination is like to form a proper command (though oopsies happen, so verify before using --delete). With cp/mv - the result depends on the presence and type of the destination. E.g. try running cp or mv, canceling then restarting. Do you need to change the arguments? Why? mkdir s1 s2 d1 touch s1/s1.txt s2/s2.txt # this seems inconsistent cp -r s1 d1 # generates d1/s1/s1.txt cp -r s2 d2 # generates d2/s2.txt mkdir s1 s2 d1 touch s1/s1.txt s2/s2.txt # I don't use this form, but it is consitent rsync -r s1 d1 # generates d1/s1/s1.txt rsync -r s2 d2 # generates d2/s2/s2.txt # Same as above but more explicit rsync -r s1 d1/ # generates d1/s1/s1.txt rsync -r s2 d2/ # generates d2/s1/s2.txt # I prefer this form most of the time: rsync -r s1/ d1/ # generates d1/s1.txt rsync -r s2/ d2/ # generates d2/s2.txt I simply try to use trailing slashes wherever permitted and the result is amply clear.
everybodyknows 3y ago
Gotta write your own application-specific wrapper script to put up the missing guardrails e.g. require/forbid trailing '/', set "--maxdelete=0", ...
xk3 3y ago
> I am used to behave in Linux and FreeBSD the only strange thing about rsync is that it follows _BSD_ syntax https://wiki.archlinux.org/title/rsync#Trailing_slash_caveat https://wiki.archlinux.org/title/rsync#Trailing_slash_caveat
- deleted 3y ago
  [deleted]
- codetrotter 3y ago
  Weird! I never noticed that. Indeed, it does! cd "$( mktemp -d )" mkdir a b c touch a/f1 a/f2 a/f3 cp -r a/ b/ ls b Resulting content of directory b on one my FreeBSD machines f1 f2 f3 Guess I usually 1. Don’t use cp -r so often, and 2. When I do use cp -r apparently I don’t put a trailing slash Cause I only ever had problems when trying to use rsync, not when using cp or mv :S

pastage 3y ago

Lost too much data that way, trailing slash with delete. I still feel bitter, the UX is terrible considering the effect are so different from copy and delete.

pessimizer 3y ago

If it didn't do that, it would have to add a switch that you'd still have to look up. It's a tool that's most often doing a merge and update, but looks like a copy command. I think that made it friendlier. How would you separate "merge this directory into that one and update files with the same name" from "copy this directory into that directory"?

Karunamon 3y ago
Halt with an error message explaining the ambiguity and the proper switch to use. Same principle as rm requiring --no-preserve-root if you actually want to nuke / - right now it's way too easy to accidentally and destructively do the wrong thing.
codetrotter 3y ago
> How would you separate "merge this directory into that one and update files with the same name" from "copy this directory into that directory"? I would make "copy this directory into that directory" out of scope for the tool. Let’s imagine a tool similar to rsync, but less confusing, and more in tune with what I want to do, personally. There are for sure a bunch of things that rsync can do, that this imagined tool can’t. That’s fine by me. Let’s call the tool nsync. It would work like this: nsync /some/src/dir /some/dest/dir And running that would behave exactly the same with or without trailing slashes. I.e the above and the following would all be equivalent: nsync /some/src/dir/ /some/dest/dir/ nsync /some/src/dir/ /some/dest/dir nsync /some/src/dir /some/dest/dir/ And what would this do? It would inspect source and dest dirs. Then it would copy files from source to dest for which last modified was greater in source dir than in dest, or where files in source dir did not exist in dest dir. In other words, it would overwrite older files that had older last modified time stamp, and it would copy files that did not exist. Like rsync it would also work with ssh (scp/sftp). Maybe some other protocols too, but only if those other protocols supported the comparisons we need to make. Prefer fewer protocols, and this way of working over trying to be the subset of what works across a gazillion protocols. If a file exists in dest but not in source dir, it is kept untouched in dest. Not deleted. Not copied back to source dir. Then there would be one other mode; destructive mode. The flag for it would be -d. nsync -d /whatever/a/b/c/ /wherever/x/y/z/ This would work similar to the normal mode. But it would remove any files in dest dir that are not present in source dir. Before actually deleting anything it would list all of the files that will be deleted, and ask for keyboard confirmation. [y/N] so that you have to explicitly hit y and then enter. Enter alone will be interpreted as no. You would be able to override the confirmation with the -y argument. nsync -dy /whatever/a/b/c/ /wherever/x/y/z/ And that’s it. That’s what I would want rsync to be for me. There probably are some programs that behave exactly like this. I’ll eventually write one too. It’ll have a user base of 1. Me.
- xk3 3y ago
  rclone move will do this https://github.com/chapmanjacobd/journal/blob/main/programming/linux/misconceptions.md#mv-src-vs-mv-src https://github.com/chapmanjacobd/journal/blob/main/programmi...

ape4 3y ago

Yes, a complete redesign of the rsync command line would be great.

labawi 3y ago

While rsync is different than cp and mv, I dislike the cp and mv destination-state-dependent behaviour. With rsync, destination paths are defined by the command itself and the command is idempotent. I don't usually need to know what the destination is like to form a proper command (though oopsies happen, so verify before using --delete). With cp/mv - the result depends on the presence and type of the destination. E.g. try running cp or mv, canceling then restarting. Do you need to change the arguments? Why? mkdir s1 s2 d1 touch s1/s1.txt s2/s2.txt # this seems inconsistent cp -r s1 d1 # generates d1/s1/s1.txt cp -r s2 d2 # generates d2/s2.txt mkdir s1 s2 d1 touch s1/s1.txt s2/s2.txt # I don't use this form, but it is consitent rsync -r s1 d1 # generates d1/s1/s1.txt rsync -r s2 d2 # generates d2/s2/s2.txt # Same as above but more explicit rsync -r s1 d1/ # generates d1/s1/s1.txt rsync -r s2 d2/ # generates d2/s1/s2.txt # I prefer this form most of the time: rsync -r s1/ d1/ # generates d1/s1.txt rsync -r s2/ d2/ # generates d2/s2.txt I simply try to use trailing slashes wherever permitted and the result is amply clear.

everybodyknows 3y ago

Gotta write your own application-specific wrapper script to put up the missing guardrails e.g. require/forbid trailing '/', set "--maxdelete=0", ...

xk3 3y ago

> I am used to behave in Linux and FreeBSD the only strange thing about rsync is that it follows _BSD_ syntax https://wiki.archlinux.org/title/rsync#Trailing_slash_caveat https://wiki.archlinux.org/title/rsync#Trailing_slash_caveat

deleted 3y ago
[deleted]
codetrotter 3y ago
Weird! I never noticed that. Indeed, it does! cd "$( mktemp -d )" mkdir a b c touch a/f1 a/f2 a/f3 cp -r a/ b/ ls b Resulting content of directory b on one my FreeBSD machines f1 f2 f3 Guess I usually 1. Don’t use cp -r so often, and 2. When I do use cp -r apparently I don’t put a trailing slash Cause I only ever had problems when trying to use rsync, not when using cp or mv :S