4 ms·

> Sometimes, you can't avoid the shell. What circumstances have you got in mind?

tene 10y ago
My favorite awkward environment to cite is running a command remotely over ssh. As far as I've been able to tell from casual testing, without having read the source code yet, ssh does something very similar to what Windows does here and just glues everything together with spaces and passes it to the remote shell for interpretation, so you have to deal with the shell and provide your own quoting.
- dozzie 10y ago
  >>> Sometimes, you can't avoid the shell. >> What circumstances have you got in mind? > My favorite awkward environment to cite is running a command remotely over ssh. Dude, if you run remote commands by calling them through SSH, you didn't just got things backward: you fucked things up heavily. SSH was never ever designed as a batch, unsupervised tool, despite many people using it as such. Remote code that is parametrized should be run exactly as that: as a remote procedure call, a technique known for over thirty years now. One of the reasons is quoting (because for non-interactive SSH call the command needs to be quoted exactly twice if in shell, and exactly once when run from exec()), but there are problems with distributing keys, maintaining usable home directories, and disabling unnecessary things that are enabled by default (port forwarding, among the others), and that doesn't exhaust the list of issues. Proper RPC protocol, like XML-RPC (which was released twenty years ago and is still usable while being quite simple), covers quoting -- or actually, serializing data -- without programmers worrying if they got their list of metacharacters right and did enough passes for things to work correctly. On the other hand, I'm not surprised that people do this through SSH (and a variant of this stupidity: adding apache user to sudoers, so a web panel can add firewall rules). After all, I've never seen an easy to use RPC server that has all the procedures passed in its configuration. I needed to write such thing myself (once in Perl, as xmlrpcd, and recently in Python, with custom protocol that can do a little more, as harpd of HarpCaller project).
  - grymoire1 10y ago
    I'm not aware of a XML-RPC protocol which is secure.
    - dozzie 10y ago
      I don't understand your statement. XML-RPC is not inherently "secure" or "insecure", no more than just HTTP is.
      - grymoire1 10y ago
        You were the one who said RPC should be used instead of SSH. People use SSH because it's secure, despite the issues in passing parameters. Suggesting that RPC be used instead of SSH while ignoring security is terrible advice.
        dozzie 10y ago
        You know what's the difference between XML-RPC and SSH with regard to payload encryption and client authentication? Only the fact that SSH has the two covered by mandatory parts of the protocol, and XML-RPC has this part optional (HTTPs and either HTTP authentication or client certificates). Nothing prevents you from exposing procedures only through HTTPs and only to authenticated clients. In fact, my xmlrpcd works exactly this way. Security-wise, your advice to use something that makes building correct system virtually impossible (because quoting issues, unnecessary features enabled by default, and others) is simply stupid and dangerous.
  - tene 10y ago
    You're entirely right; I completely agree. And yet, sometimes you're working in an environment that already has a good ssh configuration for other reasons, and you're very low on engineering time that you can invest into something, and ssh is good enough for a first pass implementation. Alternately, you may be working on some kind of ad-hoc data collection or maintenance task that's not going to become part of any long-term infrastructure (or will be replaced by something better), and you don't yet have any better systems in place to run ad-hoc programs across the cluster. I completely agree with you that good RPC is a much better foundation to build reliable systems on. [edited to add]: HarpCaller looks like a pretty interesting project, and similar to several things I've considered building in the past. Nice work.
- EvilTerran 10y ago
  That's correct - AIUI, the ssh utility has to smush the command & arguments into a single string, because that's all the protocol's "exec" request can handle: https://tools.ietf.org/html/rfc4254#section-6.5 https://tools.ietf.org/html/rfc4254#section-6.5 And I guess it can't really do any quoting/escaping, because there's no guarantees in the protocol as to how such things will be interpreted server-side - the command line could be interpreted by cmd.exe on the server for all `ssh` knows ;) With that in mind, I've made it a habit to always quote the command-and-parameters part of `ssh` lines into a single argument - I figure, that's what it's doing anyway, so it's best to be explicit about it. And when I know I'm working with bash on both ends, my pattern of choice is ssh host "$(printf '%q ' cmd arg 'arg with spaces' "$var" ...)"
joeyh 10y ago
As well as running things over ssh (which IIRC requires double-shell-escaping), su -c and similar take a single parameter containing the command to run.

tene 10y ago

My favorite awkward environment to cite is running a command remotely over ssh. As far as I've been able to tell from casual testing, without having read the source code yet, ssh does something very similar to what Windows does here and just glues everything together with spaces and passes it to the remote shell for interpretation, so you have to deal with the shell and provide your own quoting.

dozzie 10y ago
>>> Sometimes, you can't avoid the shell. >> What circumstances have you got in mind? > My favorite awkward environment to cite is running a command remotely over ssh. Dude, if you run remote commands by calling them through SSH, you didn't just got things backward: you fucked things up heavily. SSH was never ever designed as a batch, unsupervised tool, despite many people using it as such. Remote code that is parametrized should be run exactly as that: as a remote procedure call, a technique known for over thirty years now. One of the reasons is quoting (because for non-interactive SSH call the command needs to be quoted exactly twice if in shell, and exactly once when run from exec()), but there are problems with distributing keys, maintaining usable home directories, and disabling unnecessary things that are enabled by default (port forwarding, among the others), and that doesn't exhaust the list of issues. Proper RPC protocol, like XML-RPC (which was released twenty years ago and is still usable while being quite simple), covers quoting -- or actually, serializing data -- without programmers worrying if they got their list of metacharacters right and did enough passes for things to work correctly. On the other hand, I'm not surprised that people do this through SSH (and a variant of this stupidity: adding apache user to sudoers, so a web panel can add firewall rules). After all, I've never seen an easy to use RPC server that has all the procedures passed in its configuration. I needed to write such thing myself (once in Perl, as xmlrpcd, and recently in Python, with custom protocol that can do a little more, as harpd of HarpCaller project).
- grymoire1 10y ago
  I'm not aware of a XML-RPC protocol which is secure.
  - dozzie 10y ago
    I don't understand your statement. XML-RPC is not inherently "secure" or "insecure", no more than just HTTP is.
    - grymoire1 10y ago
      You were the one who said RPC should be used instead of SSH. People use SSH because it's secure, despite the issues in passing parameters. Suggesting that RPC be used instead of SSH while ignoring security is terrible advice.
      - dozzie 10y ago
        You know what's the difference between XML-RPC and SSH with regard to payload encryption and client authentication? Only the fact that SSH has the two covered by mandatory parts of the protocol, and XML-RPC has this part optional (HTTPs and either HTTP authentication or client certificates). Nothing prevents you from exposing procedures only through HTTPs and only to authenticated clients. In fact, my xmlrpcd works exactly this way. Security-wise, your advice to use something that makes building correct system virtually impossible (because quoting issues, unnecessary features enabled by default, and others) is simply stupid and dangerous.
- tene 10y ago
  You're entirely right; I completely agree. And yet, sometimes you're working in an environment that already has a good ssh configuration for other reasons, and you're very low on engineering time that you can invest into something, and ssh is good enough for a first pass implementation. Alternately, you may be working on some kind of ad-hoc data collection or maintenance task that's not going to become part of any long-term infrastructure (or will be replaced by something better), and you don't yet have any better systems in place to run ad-hoc programs across the cluster. I completely agree with you that good RPC is a much better foundation to build reliable systems on. [edited to add]: HarpCaller looks like a pretty interesting project, and similar to several things I've considered building in the past. Nice work.
EvilTerran 10y ago
That's correct - AIUI, the ssh utility has to smush the command & arguments into a single string, because that's all the protocol's "exec" request can handle: https://tools.ietf.org/html/rfc4254#section-6.5 https://tools.ietf.org/html/rfc4254#section-6.5 And I guess it can't really do any quoting/escaping, because there's no guarantees in the protocol as to how such things will be interpreted server-side - the command line could be interpreted by cmd.exe on the server for all `ssh` knows ;) With that in mind, I've made it a habit to always quote the command-and-parameters part of `ssh` lines into a single argument - I figure, that's what it's doing anyway, so it's best to be explicit about it. And when I know I'm working with bash on both ends, my pattern of choice is ssh host "$(printf '%q ' cmd arg 'arg with spaces' "$var" ...)"

joeyh 10y ago

As well as running things over ssh (which IIRC requires double-shell-escaping), su -c and similar take a single parameter containing the command to run.