2 ms·

The coolest thing I ever saw in the wild, a guy I once worked with wanted to transfer a directory of files from my machine to his (and maybe show off a little.)

The coolest thing I ever saw in the wild, a guy I once worked with wanted to transfer a directory of files from my machine to his (and maybe show off a little.) This was long enough ago that I didn't know quite how to proceed (time before rsync, scp...? Nah...) so he asked if he could do it and I let him have the keyboard. He tar'd (with z) the dir, piped the output of tar to ssh, with a remote command to cat it out there through tar again, all in one CLI line. Blew my mind at the time. UNIX philo FTW.

e12e 7y ago
Note that upstream suggests rsync rather than scp in the general case. Of course pipes work too. I sometimes forget, and the remember half way through a big recursive scp operation. Just use rsync.
rsync 7y ago
From a very old notes file I have ... tar cf - /mnt/data2 | ssh user@10.0.0.10 "cat > /mnt/data1/file1/file_data2.tar" ... and if the directory is large, you can 'split' it into multiple files: tar cfp - /mnt/data1 | ssh user@10.0.0.10 "split - -b 1024m /mnt/data1/file1/file_data2.tar" This is really no longer relevant because, of course, we all just use rsync ... but in the modern world, my favorite example of the unix philosophy is: mysqldump -u mysql db | ssh user@rsync.net "dd of=db_dump"
- jandrese 7y ago
  None of those are quite right IMHO: tar -czvf - <sourcedir> | ssh <user>@<remotehost> tar -xzf - -C <remotedir> This is _much_ faster if you're sending over a directory with a lot of small files, especially if the link has even a modest amount of latency. The 'z' parameter can be omitted if the source files are not compressible (media files or already compressed). If the files are highly compressible but very large you might consider this instead: tar -cvf - <sourcedir> | pbzip2 -c | ssh <user>@<remotehost> tar -xf - -C <remotedir>
m463 7y ago
I think rsync has obsoleted a lot of that: rsync -av --progress indir/ x:outdir/

e12e 7y ago

Note that upstream suggests rsync rather than scp in the general case. Of course pipes work too. I sometimes forget, and the remember half way through a big recursive scp operation. Just use rsync.

rsync 7y ago

From a very old notes file I have ... tar cf - /mnt/data2 | ssh user@10.0.0.10 "cat > /mnt/data1/file1/file_data2.tar" ... and if the directory is large, you can 'split' it into multiple files: tar cfp - /mnt/data1 | ssh user@10.0.0.10 "split - -b 1024m /mnt/data1/file1/file_data2.tar" This is really no longer relevant because, of course, we all just use rsync ... but in the modern world, my favorite example of the unix philosophy is: mysqldump -u mysql db | ssh user@rsync.net "dd of=db_dump"

jandrese 7y ago
None of those are quite right IMHO: tar -czvf - <sourcedir> | ssh <user>@<remotehost> tar -xzf - -C <remotedir> This is _much_ faster if you're sending over a directory with a lot of small files, especially if the link has even a modest amount of latency. The 'z' parameter can be omitted if the source files are not compressible (media files or already compressed). If the files are highly compressible but very large you might consider this instead: tar -cvf - <sourcedir> | pbzip2 -c | ssh <user>@<remotehost> tar -xf - -C <remotedir>

m463 7y ago

I think rsync has obsoleted a lot of that: rsync -av --progress indir/ x:outdir/