8 ms·

It includes an interesting comment on the scp vulnerability. > This release contains mitigation for a weakness in the scp(1) tool and protocol (CVE-2019-6111)

It includes an interesting comment on the scp vulnerability. > This release contains mitigation for a weakness in the scp(1) tool and protocol (CVE-2019-6111) [...] The scp protocol is outdated, inflexible and not readily fixed. We recommend the use of more modern protocols like sftp and rsync for file transfer instead. I think it's the time to "alias scp=sftp". If the developers officially believe that scp should be retired, let's do the switch. Both are parts of OpenSSH and the commandline argument is almost identical. Also, it has > ssh(1), sshd(8): Add experimental quantum-computing resistant key exchange method, based on a combination of Streamlined NTRU Prime 4591^761 and X25519. This is big. Together with XMSS signature, it means we already have a complete suite of post-quantum cryptography (experimentally) deployed in OpenSSH! It may be the first mass deployment of post-quantum cryptography in a major protocol. One month ago, I commented that the introduction of XMSS post-quantum signature as "useless" (https://news.ycombinator.com/item?id=19160739 https://news.ycombinator.com/item?id=19160739), as the decryption of key exchange is much more vulnerable than spoofing the signature. But now NTRU+X25519 is deployed, great progress here!

djmdjm 8y ago
Unfortunately, you can't alias scp=sftp as the commandline syntax is different. I'd love someone to write a scp replacement that attempted the sftp protocol first and optionally fell back to scp if that didn't work.
- low_key 8y ago
  This. I've been using rsync for a long time just for the resume feature even without knowing about some of the other benefits. I tend to use the flags "-avzpP"
- pnutjam 8y ago
  Not all ssh servers enable the sftp subsystem by default.
bArray 8y ago
Checking out the mitigation for CVE-2019-6111 [1], how does `sftp` help? * scp(1): Relating to the above changes to scp(1); the scp protocol relies on the remote shell for wildcard expansion, so there is no infallible way for the client's wildcard matching to perfectly reflect the server's. If there is a difference between client and server wildcard expansion, the client may refuse files from the server. For this reason, we have provided a new "-T" flag to scp that disables these client-side checks at the risk of reintroducing the attack described above. You could just do a remote `ls` and then `sftp` all the files listed to your client - nothing stopping a malicious return from `ls`? Surely it's the risk of running any kind of wild card copy? If the server is compromised then there's no telling what will be returned from such a command? The scp protocol is outdated, inflexible and not readily fixed. We recommend the use of more modern protocols like sftp and rsync for file transfer instead. The protocol itself might be outdated, but surely one of the best aspects of `scp` comes from its simplicity [2]? [1] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6111 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6111 [2] https://github.com/openssh/openssh-portable/blob/master/scp.c https://github.com/openssh/openssh-portable/blob/master/scp....
- djmdjm 8y ago
  scp only appears to have simplicity because it outsources large parts of its functionality (e.g. glob expansion) to the destination host's shell. scp/rcp was a great protocol for 1981 (yes, that's when it was introduced) but not for 2019
  - tgragnato 8y ago
    And this leads to the question.. OpenBSD is famous for removing unwanted or outdated code. What makes scp a special case? Is it too used to be deprecated and removed?
    - djmdjm 8y ago
      yeah, pretty much. If someone implemented scp's command-line with sftp underneath then we could start the (slow) deprecation process.
0XAFFE 8y ago
This works pretty well: alias scp="rsync --partial --progress --rsh=ssh" You should keep in mind that it is rsync in the background, because parameter are different.
- Arkanosis 8y ago
  I'd recommend to use `--partial-dir=.rsync-partial` instead of `--partial`, as the latter does not prevent an incomplete transfer from looking like a complete one.
MayeulC 8y ago
Frankly, I had not realized scp and sftp were two distinct protocols, despite my heavily using them. I thought sftp was an interactive client, while scp was meant for scripting, and never really bothered to dig further. I now will, thank you. However, I expect plenty of people to be in the same case.
- simias 8y ago
  I was definitely in the same boat. I guess I'll just progressively phase out my use of scp in favor of rsync.
- djmdjm 8y ago
  actually, sftp is a bit more scriptable than scp. You can pass sftp a "batchfile" of commands and control per-command whether their success should terminate the batch. scp wins on brevity of commandline syntax
  - throwaway2048 8y ago
    The script-ability of sftp is pretty awkward, it doesen't conform to how virtually every other tool works in terms of "command + args" in a shell scripting context, instead you need to pass it commands through a pipeline, don't love it. Would be nice to be able to just type sftp user@domain:/file and have it do the right thing.
    - MayeulC 8y ago
      Oh, that means you have to open a pipe to the process if you want to control it from another programming language? That would be a bit harder to leverage for automation (though I guess it's just a matter of writing a few helper functions, or you could always use bash and redirections if the language is ill-suited to pipes).
    - LinuxBender 8y ago
      Add lftp on top of sftp and you get extensive scriptability. Working public anonymous demo: [1] [1] - https://tinyvpn.org/sftp/#lftp https://tinyvpn.org/sftp/#lftp
xorcist 8y ago
> I think it's the time to "alias scp=sftp" May I suggest using rsync instead? The syntax is much closer to scp than sftp is, to the point of being compatible for the most trivial use cases. But rsync is actively developed and has much saner defaults, can continue broken transfers etc.
adrian_b 8y ago
The only good file transfer solution has always been to use rsync over ssh instead of scp or sftp. I have stopped using scp and sftp many years ago. Besides the fact that sftp can be too slow, both scp and sftp fail to make identical copies of the transferred files because sometimes they can lose some metadata, e.g. extended attributes or high resolution time stamps.
- adrian_b 8y ago
  What I am normally using is: alias scp='rsync --archive --xattrs --acls --progress --rsh="ssh"'
  - chrisweekly 8y ago
    This gets my vote for nicest alias in the thread.
  - deleted 8y ago
    [deleted]
  - danudey 8y ago
    I've become a huge fan lately of using "--info=progress --no-inc-recursive" for rsync. It can result in more memory usage, since it builds the file list ahead of time rather than dynamically, but it provides you with an overall progress report of the entire transfer, rather than per-file progress reports which may be much less useful.
  - seqizz 8y ago
    Aaaah I was so happy to switch until I see "The source and destination cannot both be remote."
- tyingq 8y ago
  Using ssh with tar on both ends is my go to for a new box that doesn't have rsync. Long command lines, but it retains metadata. Like: ssh remote 'cd /src;tar -cf - .' | (cd /dst;tar -xf -) Doesn't call you out so well on typos though.
- 8y ago
jolmg 8y ago
> I think it's the time to "alias scp=sftp". If the developers officially believe that scp should be retired, let's do the switch. Both are parts of OpenSSH and the commandline argument is almost identical. I've never used sftp before. Does it offer equivalents to: scp server:'*.pdf' . or scp server:'*(oc[1])' . for getting the last created file with zsh extended globs, or scp server:'$(custom_remote_program)' . where custom_remote_program is shell code that outputs a list of files using server-side state? EDIT: Reading: > * scp(1): Relating to the above changes to scp(1); the scp protocol relies on the remote shell for wildcard expansion, so there is no infallible way for the client's wildcard matching to perfectly reflect the server's. If there is a difference between client and server wildcard expansion, the client may refuse files from the server. For this reason, we have provided a new "-T" flag to scp that disables these client-side checks at the risk of reintroducing the attack described above. I guess, I'll actually want: alias scp='scp -T' or just use `-T` on a need by basis. I admit I don't need that feature every week, but it is a relief that they didn't remove the functionality outright. I think the closest equivalent is something like: ssh server 'tar c $(custom_remote_program)' | tar xC destination which I'd rather not be writing.
fanf2 8y ago
PuTTY's pscp has a command-line UI like scp, but (by default) it is implemented using the sftp protocol. It's a pity OpenSSH scp doesn't work the same way.

djmdjm 8y ago

Unfortunately, you can't alias scp=sftp as the commandline syntax is different. I'd love someone to write a scp replacement that attempted the sftp protocol first and optionally fell back to scp if that didn't work.

low_key 8y ago
This. I've been using rsync for a long time just for the resume feature even without knowing about some of the other benefits. I tend to use the flags "-avzpP"
pnutjam 8y ago
Not all ssh servers enable the sftp subsystem by default.

bArray 8y ago

Checking out the mitigation for CVE-2019-6111 [1], how does `sftp` help? * scp(1): Relating to the above changes to scp(1); the scp protocol relies on the remote shell for wildcard expansion, so there is no infallible way for the client's wildcard matching to perfectly reflect the server's. If there is a difference between client and server wildcard expansion, the client may refuse files from the server. For this reason, we have provided a new "-T" flag to scp that disables these client-side checks at the risk of reintroducing the attack described above. You could just do a remote `ls` and then `sftp` all the files listed to your client - nothing stopping a malicious return from `ls`? Surely it's the risk of running any kind of wild card copy? If the server is compromised then there's no telling what will be returned from such a command? The scp protocol is outdated, inflexible and not readily fixed. We recommend the use of more modern protocols like sftp and rsync for file transfer instead. The protocol itself might be outdated, but surely one of the best aspects of `scp` comes from its simplicity [2]? [1] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6111 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6111 [2] https://github.com/openssh/openssh-portable/blob/master/scp.c https://github.com/openssh/openssh-portable/blob/master/scp....

djmdjm 8y ago
scp only appears to have simplicity because it outsources large parts of its functionality (e.g. glob expansion) to the destination host's shell. scp/rcp was a great protocol for 1981 (yes, that's when it was introduced) but not for 2019
- tgragnato 8y ago
  And this leads to the question.. OpenBSD is famous for removing unwanted or outdated code. What makes scp a special case? Is it too used to be deprecated and removed?
  - djmdjm 8y ago
    yeah, pretty much. If someone implemented scp's command-line with sftp underneath then we could start the (slow) deprecation process.

0XAFFE 8y ago

This works pretty well: alias scp="rsync --partial --progress --rsh=ssh" You should keep in mind that it is rsync in the background, because parameter are different.

Arkanosis 8y ago
I'd recommend to use `--partial-dir=.rsync-partial` instead of `--partial`, as the latter does not prevent an incomplete transfer from looking like a complete one.

MayeulC 8y ago

Frankly, I had not realized scp and sftp were two distinct protocols, despite my heavily using them. I thought sftp was an interactive client, while scp was meant for scripting, and never really bothered to dig further. I now will, thank you. However, I expect plenty of people to be in the same case.

simias 8y ago
I was definitely in the same boat. I guess I'll just progressively phase out my use of scp in favor of rsync.
djmdjm 8y ago
actually, sftp is a bit more scriptable than scp. You can pass sftp a "batchfile" of commands and control per-command whether their success should terminate the batch. scp wins on brevity of commandline syntax
- throwaway2048 8y ago
  The script-ability of sftp is pretty awkward, it doesen't conform to how virtually every other tool works in terms of "command + args" in a shell scripting context, instead you need to pass it commands through a pipeline, don't love it. Would be nice to be able to just type sftp user@domain:/file and have it do the right thing.
  - MayeulC 8y ago
    Oh, that means you have to open a pipe to the process if you want to control it from another programming language? That would be a bit harder to leverage for automation (though I guess it's just a matter of writing a few helper functions, or you could always use bash and redirections if the language is ill-suited to pipes).
  - LinuxBender 8y ago
    Add lftp on top of sftp and you get extensive scriptability. Working public anonymous demo: [1] [1] - https://tinyvpn.org/sftp/#lftp https://tinyvpn.org/sftp/#lftp

xorcist 8y ago

> I think it's the time to "alias scp=sftp" May I suggest using rsync instead? The syntax is much closer to scp than sftp is, to the point of being compatible for the most trivial use cases. But rsync is actively developed and has much saner defaults, can continue broken transfers etc.

adrian_b 8y ago

The only good file transfer solution has always been to use rsync over ssh instead of scp or sftp. I have stopped using scp and sftp many years ago. Besides the fact that sftp can be too slow, both scp and sftp fail to make identical copies of the transferred files because sometimes they can lose some metadata, e.g. extended attributes or high resolution time stamps.

adrian_b 8y ago
What I am normally using is: alias scp='rsync --archive --xattrs --acls --progress --rsh="ssh"'
- chrisweekly 8y ago
  This gets my vote for nicest alias in the thread.
- deleted 8y ago
  [deleted]
- danudey 8y ago
  I've become a huge fan lately of using "--info=progress --no-inc-recursive" for rsync. It can result in more memory usage, since it builds the file list ahead of time rather than dynamically, but it provides you with an overall progress report of the entire transfer, rather than per-file progress reports which may be much less useful.
- seqizz 8y ago
  Aaaah I was so happy to switch until I see "The source and destination cannot both be remote."
tyingq 8y ago
Using ssh with tar on both ends is my go to for a new box that doesn't have rsync. Long command lines, but it retains metadata. Like: ssh remote 'cd /src;tar -cf - .' | (cd /dst;tar -xf -) Doesn't call you out so well on typos though.
8y ago

jolmg 8y ago

> I think it's the time to "alias scp=sftp". If the developers officially believe that scp should be retired, let's do the switch. Both are parts of OpenSSH and the commandline argument is almost identical. I've never used sftp before. Does it offer equivalents to: scp server:'*.pdf' . or scp server:'*(oc[1])' . for getting the last created file with zsh extended globs, or scp server:'$(custom_remote_program)' . where custom_remote_program is shell code that outputs a list of files using server-side state? EDIT: Reading: > * scp(1): Relating to the above changes to scp(1); the scp protocol relies on the remote shell for wildcard expansion, so there is no infallible way for the client's wildcard matching to perfectly reflect the server's. If there is a difference between client and server wildcard expansion, the client may refuse files from the server. For this reason, we have provided a new "-T" flag to scp that disables these client-side checks at the risk of reintroducing the attack described above. I guess, I'll actually want: alias scp='scp -T' or just use `-T` on a need by basis. I admit I don't need that feature every week, but it is a relief that they didn't remove the functionality outright. I think the closest equivalent is something like: ssh server 'tar c $(custom_remote_program)' | tar xC destination which I'd rather not be writing.

fanf2 8y ago

PuTTY's pscp has a command-line UI like scp, but (by default) it is implemented using the sftp protocol. It's a pity OpenSSH scp doesn't work the same way.