7 ms·
The Unexpected Importance of the Trailing Slash
- colordrops 5y agorsync is what taught me to respect the trailing slash.
- 6LLvveMx2koXfwn 5y agoCame here to say exactly that, smiled at yours being the first comment. And further I used to hate rsync's parsing of the trailing slash and now have come to love it, I don't know when the transition happened.
- l33tman 5y agoI'm still scared every time I use rsync because of that and have to thoroughly check the man-page examples :)
- thwarted 5y agoNo need to fear! Use --dry-run to confirm your command line before making the changes. It's documented in that very man page!
- 0des 5y agoI have a list of aliases I pass to my minions, one is to alias rsync with this, and the real rsync is aliased as rsyncplz
- teddyh 5y agoThat kind of stuff sounds good, but it’s dangerous. You risk getting used to it, and then getting hosed when you’re on a normal system. Some people use a similar thing where they set "rm" as an alias to "rm -i", i.e. ask interactively before removal of each file. I once heard a story about some SunOS consultant who was used to "rm" being an alias for "rm -i", and the first thing this consultant did, as root, was to "cd /etc" and then "rm *".
- layer8 5y agoIn principle this could be solved by having two aliases `rsync-dry-run` and `rsync-for-real`, and aliasing `rsync` to `echo "Use rsync-dry-run or rsync-for-real“`.
- 0des 5y agoThat's great, I'll do that now. Thanks!
- justinsaccount 5y agoOne of my first introductions to the fact that GNU getopt is much different from BSD/UNIX getopt was running `rm * -i` on Solaris. Instead of prompting me for which files to delete, it removed all of my files and then printed an error about failing to remove the non existent file "-i".
- vq 5y agoEvery time I use one of the --delete/--delete-* arguments I first try with --dry-run since I don't trust myself that I got the trailing slashes in the right spot.
- matejn 5y agoI've recently had to copy a bunch of files over two Windows Servers and was pleasantly surprised by robocopy's (basically Windows' rsync) solution to this dilemma: the first two parameters always only refer to directories, and then you can list specific files starting with the third parameter. This way there can be no confusion whether you're copying the contents of a directory or the whole directory, and you can't create a duplicated directory within your destination by mistake.
- paol 5y agoI love rsync for what it can do, but I hate that with a passion. It's terrible design that should have never seen the light of day.
- deleted 5y ago[deleted]
- mhalle 5y agoI end up reposting this comment whenever rsync's slash treatment gets mentioned. rsync has "weird" syntax for a reason. Unlike other unix-like commands, it treats trailing slashes as significant AND consistently. If a directory has a trailing slash, it means "contents" of the directory". No slash means "the directory itself". These are two different concepts, and a program that copies directories should take the difference into account. scp (and cp, for that matter) don't take this difference into account. That leads to gotchas with recursive (-r) copying. Most importantly, scp isn't idemopotent: scp -r fromdir todir If todir doesn't exist, scp will copy the contents of directory fromdir to a new directory named todir. Execute the same command again (now that todir exists), and scp will copy fromdir to todir/fromdir . On the other hand: rsync -a fromdir/ todir will always copy the contents of fromdir into a directory named todir (effectively, a directory rename operation), whether todir exists or not. rsync -a fromdir todir will always copy the directory fromdir into the directory todir, whether todir exists or not. These rsync operations are idempotent, which it important because rsync is designed to incrementally re-sync directories. It is expected that it will commonly be run more than once, which is why it needed to address this IMHO fundamental bug/limitation in cp and scp.
- leephillips 5y agoGreat explanation! I prefer rsync to cp, etc., because of this consistency, but I never appreciated the idempotency implications.
- nijave 5y agoIsn't that problem also solved with the `dir/.` syntax? Adding the `.` signifies content versus the directly itself I think it's `cp` that uses that syntax
- adrian_b 5y agoActually a trailing slash is significant not only in rsync, but in most other command-line utilities. Unfortunately, rsync lacks a helpful option like the "--strip-trailing-slashes" of GNU cp and mv. I never use bare cp or mv, but only aliases which replace their undesirable default options with other more useful options, including "--strip-trailing-slashes", which makes the presence or absence of a trailing slash irrelevant.
- gjvc 5y agosame. my rule of thumb is: always always always use the trailing slash on source/ and target/ paths and it will always do the right thing.
- nickcw 5y agoWhen I was implementing rclone which implements a lot of rsync's functionality but for cloud storage, I deliberately didn't copy the trailing slash feature. This annoys people who know rsync well, but newbies are very happy with it! Even though I know rsync extremely well I am super careful with this. Test first with `--dry-run`!
- the_svd_doctor 5y agoThanks so much for rclone. Awesome piece of software.
- dataflow 5y agoDon't forget the variation where you also end it with a dot: foo/.
- linsomniac 5y ago"I really mean it".
- pabs3 5y agoWhat does that do?
- dataflow 5y agoGreat question, I was wondering how if anyone would be interested to ask! tl;dr: Basically, it's a way to say "do this operation inside this folder". For example, if you do cp -r A B an A is a directory, the behavior is different depending on whether B already exists or not. If B is a file, you get an error. If B is a directory, then it tries to make a copy of A inside B (i.e. B/A). If B doesn't exist, B itself becomes a copy of A. This is pretty terrible behavior. The question is how do you avoid this. On Linux, if you always want A to be overlaid on top of B, then you can do cp -T A B which disambiguates in favor of overlay. On the other hand, if you always want A to be copied inside B, then you can do cp A B/. which disambiguates in favor of creating a subdirectory (and gives you an error if B doesn't exist). Note: I have not checked whether this behavior is uniform across other POSIX systems, but I hope it is.
- larusso 5y agoAhh well the pitfalls of writing portable shell scripts. There is still the myth flying around that bash scripts are the best solution for write once run anywhere scripts. I tip my head to the projects that are written in plain bash or sh.
- pixelbeat__ 5y agoYes there are inconsistences and surprises with trailing slash handling. For e.g rmdir doesn't follow symlink-to-dir/ on Linux. I tried to at least clarify what was happening in this case recently: https://github.com/coreutils/coreutils/commit/9de1d153 https://github.com/coreutils/coreutils/commit/9de1d153
- thanatos519 5y agoHot tip: when making a symlink to a directory, include the trailing slash ('ln -s dirname/ linkname') so that if the directory disappears, a copy to the link results in an error instead of creating a file called 'linkname'.
- CGamesPlay 5y agoDo you mean a file called dirname? Interesting behavior in either case.
- deleted 5y ago[deleted]
- leephillips 5y agoNo, `linkname` is correct. It’s the name of the link in the example. So the command `cp <file> linkname/` will give you an error if the taget is gone.
- vicaya 5y agoNo, `dirname` is correct. The goal of `ln -s dirname/ linkname` is that even if you accidentally do `cp file linkname` (note without slash) when dirname is gone, it will error out. Without the trailing slash in the ln command, a file (instead of directory) named dirname will be created.
- fooblat 5y agoBack in the day, the company I worked for required all newly hired system and software engineers to take their Unix Fundamentals class. 8 hours a day for 4 days they covered how the shell, coreutils, and filesystems work. Sometimes people would complain that they didn't need a "beginner" class but I bet there wasn't a single person who didn't learn some important subtlety like how the trailing / works in different situations. I've never had another company require or even offer a training like that. Do other companies still teach this stuff? edit: typos
- jciochon 5y agoI’m not familiar with the “Unix Fundamentals” class (got a link?) but when I have interns I make the “missing semester”[1] course part of their onboarding. I find it’s a nice balance of pragmatism and the classroom structure they are already familiar with. I will also note that I think skills like this end up being way more valuable to me as a mentor than hard CS knowledge. [1] https://missing.csail.mit.edu/ https://missing.csail.mit.edu/
- fooblat 5y agoUnfortunately I don't have a link. It was an internally developed class at that company. This was in 2002.
- dec0dedab0de 5y agoWhen I was much younger I worked at a small ISP/CLEC started with a week of training, including editing regex mail filters, configuring bind, and sending email via telnet. It was a tech support job for mostly dial up customers. The manager had developed a binder of training materials that included many things to help us troubleshoot.
- nijave 5y agoJP Morgan Chase does (or did) offer a 18 month mainframe training program called zUniversity that combined in-house materials and IBM training programs. Not sure about new hires (it would have varied by team and line of business), but they also had lots of formal and instructor led training courses for new technology, frameworks, and patterns
- daneel_w 5y agoI personally always found it intuitive. But the rules are easy to remember: dir/ means "inside dir", dir means "dir and what's inside".
- dicknuckle 5y ago/dir to me is "the dir itself".
- PoignardAzur 5y agoI'm starting to be convinced that symlinks are way more trouble than they're worth, especially in modern filesystems with reflink support. Reflink copies are basically invisible, except when you need to meter disk usage. Symlinks and hardlinks add a billion corner cases to file handling.
- yencabulator 5y agoZFS not having reflinks is still so sad. https://github.com/openzfs/zfs/issues/405 https://github.com/openzfs/zfs/issues/405