4 ms·
Tar combined with ssh is a decent replacement for scp. A little more dangerous if you're in the wrong directory, etc, but... $ cd /source/dir && tar -cf - .
by tyingq 6y ago
Tar combined with ssh is a decent replacement for scp. A little more dangerous if you're in the wrong directory, etc, but...
$ cd /source/dir && tar -cf - . | ssh rhost 'cd /dest/dir && tar -xvf -'
Similar for the other direction.
- adrian_b 6y agoI prefer to use rsync over SSH as a replacement for scp: alias scp='/usr/bin/rsync --archive --xattrs --acls --progress --rsh="ssh"' I have completely deprecated for myself the use of both scp and sftp many years ago and I disable the sftp server (in sshd.conf) on all my servers. While scp and sftp are also slower, usually being unable to reach link speed on fast links, the main reason is that I have discovered that when copying files with them they sometimes were losing file metadata (e.g. parts of the timestamps or extended attributes) without any warnings or errors. rsync does not have such problems, it can make exact copies even when copying between different operating systems or different file systems. Because I have stopped using both scp and sftp many years ago, I do not know whether meanwhile there was any effort to remove the bugs from scp and sftp, but I doubt it.
- tyingq 6y agoI agree rsync is better than using tar, but often I'm using that ssh/tar command line before rsync is available on the other end.
- dvdbloc 6y ago> before rsync is available on the other end. I’m curious, is this in an embedded environment or interacting with some sort of boot loader?
- regecks 6y agoNothing so drastic; I often find that Debian/Ubuntu VPS servers do not come with rsync preinstalled.
- eru 6y agoAnd rsync can be curiously slow under some circumstances.
- dredmorbius 6y agoTar can also be used with, e.g., netcat (included in busybox), useful for bootstrapping / building / rescuing systems.
- chaganated 6y agowhy the explicit rsh?
- adrian_b 6y agoThe rsync option --rsh="command_name" or -e "command_name" is required to be certain that "ssh" is used for the transfers. The rsync program might have been compiled to use "ssh" even if you do not use the --rsh= or -e options, but you cannot know this for sure (unless you have compiled it yourself and you have read the sources to verify that).
- pabs3 6y agoI suggest also adding -H (hard links) and -S (sparse files) to that command-line.
- tremon 6y agoNot sure if it's common across all tar implementations, but you can use -C to let GNU tar change directories before performing any operation. So this command is equivalent: $ tar -cC /source/dir . | ssh rhost tar -xvC /dest/dir (I also left out -f - because it's the default)
- tyingq 6y agoHelpful, thanks. Old habits die hard. Tar wasn't always GNU tar back when this became useful to me.
- wahern 6y agoFWIW, while tar isn't even a POSIX command, -C is supported by bsdtar[1] (including macOS), AIX tar (confirmed AIX 7.1), and Solaris tar (confirmed Solaris 11.4). Argument ordering and bundling is really funky on AIX and Solaris, though, especially for extraction. [1] Seems like everybody except OpenBSD has migrated to libarchive's bsdtar, but OpenBSD also supports -C.
- deleted 6y ago[deleted]
- keeperofdakeys 6y agoJust a sidenote, this is probably the fastest way to copy lots of tiny files across a high latency link (but without -v [1]). Most protocols transfer files individually, waiting for the server to finish writing it - adding a tiny delay for each file transferred. The tar above simply pushes the files across the pipe as fast as it can without waiting, while the receiving tar can write them as soon as the data is received. Recommended to pair with lzop for fast compression. Second sidenote, using this with pv is a great way to view total progress. http://www.ivarch.com/programs/pv.shtml http://www.ivarch.com/programs/pv.shtml 1. With -v every filename is written to the terminal, which causes context switches and IO waits. This can significantly slow file copies with lots of tiny files.