7 ms·

Subprocesses are generally bad news

32 points by tbodt 9y ago

cafxx 9y ago
Hello inetd, my old friend I've come to talk with you again Because bugs softly creeping Pwned my server while I was sleeping...
userbinator 9y ago
"I'll just escape it" is a typical and yet unacceptable response to this. There seems to be some sort of "cult" (for lack of better term) around this mentality of "it's too hard so don't do it", which unfortunately only makes things worse overall. Escaping rules are well-defined (they must be, by necessity), and sooner or later you're going to have to do something involving it; not to mention the important superset of things to keep in mind when writing code, of which escaping is only a fraction: the general principle of "consider all input". Always assume that all 256 values of a byte can and will show up in any external input, and plan accordingly.
- asveikau 9y ago
  > Escaping rules are well-defined Except when they aren't? I'm thinking of things like the bash exported function bug, an obscure feature nobody thought was there turns evaluating environment variables into RCE. Not to mention different shells may do things slightly differently and you may get caught doing something unexpected. So why not leave the shell out of the picture entirely? I think execve and friends are much easier to get right than escaping. There are still possible security bugs to consider (eg., if using the -p family such as execvp, who controls PATH? Is anything on PATH writeable?), but the shell won't be among them.
  - blindgeek 9y ago
    As often as I've seen people get shell-escaping wrong, escaping rules probably aren't well-understood, regardless of how well-defined they are. Bourne shell's quoting rules are byzantine at best. There are ways to reliably and securely quote for the Bourne shell, but the best solution is not to pass untrusted input to the shell. Unrelated to the shell, I've seen people -- including myself -- fail at dot-stuffing. Experience has taught me to avoid hand-rolled escaping when I can; escaping is definitely a code smell.
- 3pt14159 9y ago
  It just gets kinda exhausting. If you whitelist characters you have constant maintenance, if you don't some stupid edge case slips through. You're right of course, but I sometimes get discouraged at how much work every little thing is.
- geofft 9y ago
  > There seems to be some sort of "cult" (for lack of better term) around this mentality of "it's too hard so don't do it" This cult is called engineering reliable systems. If you want to build a system that actually works (instead of mathematically prove that a workable system could exist), you have to take the following into account: - Describing a complicated system is harder than describing a simple system; there may be spec bugs. - Implementing a complicated system is harder than implementing a simple system; there are almost certainly implementation bugs, and they are not evenly distributed throughout the entire implementation. - Code that isn't regularly exercised with full and meaningful coverage is likely to have unexplored bugs, so if you're using third-party code (like a shell), there's little sense in building your system on top of the most obscure parts of it. - Full and meaningful coverage could mean tests, but is not very likely to, because humans are just not very good at writing good tests and keeping up with them. Humans who write your third-party dependencies (like your /bin/sh) are not subject to your organization's policies about test coverage. - Code changes, and realistically you have to keep up with changes from external code. Code that matches your expectations today might fail to match them tomorrow, for the same reason that the obscure parts might change because no one is using them. - You, personally, are fallible. You will have off days. You will forget the edge case you thought of last week. You will come back to your code a year later and not understand the complexity, and then you will be responsible for figuring out the complexity. - You're not the only person maintaining your code, and even if you had a hiring veto, you're not going to be able to enforce that everyone on your team has the same sense for the same complexities that come naturally to you. Your complex code will get modified by a teammate when you're on vacation, and then you'll get paged for it when you return. It's in your own interest not to set up this situation. - In general, the people implementing and running your systems are humans, and they have all sorts of weird human incentives: shipping the feature instead of spending tons of time on the edge cases, fixing the broken system in the fastest way possible when it fails at 4:30 on a Friday, not correcting their more-senior teammate's code in reviews if they're not confident they're right, etc. You are much more likely to succeed at engineering around these failure modes than getting humans to stop having human faults. - If doing the complex thing is inevitable, there's an art to figuring out whether you should do so rarely to minimize the number of mistakes, or frequently so that you get good at it. Both are reasonable options and the better choice depends a lot on the risks. And even if you want to take the mathematical proof route, you're still subject to incompleteness. I'd much prefer the approach that empirically gives me a high chance of building reliable systems in practice. In particular, I disagree with the claim "Escaping rules are well-defined (they must be, by necessity)" - there is no necessity that the specification be complete, and there's certainly no necessity that whatever implementation you're using faithfully implements all the rules.
codemac 9y ago
This is sanitizing input, yes? Didn't seem to be too specific to subprocesses, just some places where folks use input directly.
StefanKarpinski 9y ago
I had similar thoughts some time ago but came to a very different conclusion: anything the shell can do a programming language could do just as well, given the right interface. The fact that languages don't make it easy to spawn subprocesses safely doesn't mean that they couldn't do so. The right solution is making it both convenient and safe to do shell-like things from within a real programming language. So I made sure that Julia does this right: https://julialang.org/blog/2012/03/shelling-out-sucks https://julialang.org/blog/2012/03/shelling-out-sucks https://julialang.org/blog/2013/04/put-this-in-your-pipe https://julialang.org/blog/2013/04/put-this-in-your-pipe
- elcritch 9y ago
  Never realized you could do that. Hey hank you, that’s handy! One question though, is it possible to intersperse pure Julia functions but still have them piped, e.g. write custom “shell” commends?
  - StefanKarpinski 9y ago
    It's not currently, but I'm planning on adding that feature in a future version along with making shell metacharacters like `>` and `|` act much like they do shells when they appear inside of Julia command literals. When we do that, we'll be able to support splicing Julia functions and/or tasks into pipelines.
    - elcritch 9y ago
      That'd be fun! I'll keep an eye out for it. Yesterday I was playing around with `eshell` in Emacs and the ability to redirect command output into buffers and modify them with code turns out surprisingly handy.
- bjoli 9y ago
  Scsh also does this right. It makes me sad that no one actually successfully ported it to other schemes.
ggm 9y ago
Because the shell is a generalized engine for executing things, and because magic defines textfiles as candidate executables nominating either the shell or a shell-exec binary to interpret them, and because system() jacks all of this to say you can invoke a shell with all its awesome to invoke scripts which invoke a binary to parse them. If the shell was only rsh, and if the set of binaries you can invoke was constrained, and if the network and system calls were accessed through strace() barriers which limited what you could do.. We might have less problems. Except that in the end, people don't code or script for secure execution so the context of 'what harm can I do from here' turns out to be a lot wider than many people think.
AstralStorm 9y ago
The premise is faulty. Subprocesses are fine, it is the POSIX shell that should be avoided. Or any interpreted language. Smallest tool for the job. Use exec* calls, right options for your process handling class and there is no problem. As a bonus feel free to spawn/fork your current application. Linking pipes and file descriptors is also not that hard. (Though they are a cruddy that may be nicer to wrap in something like 0mq.)

32 points by tbodt 9y ago

Hello inetd, my old friend I've come to talk with you again Because bugs softly creeping Pwned my server while I was sleeping...

userbinator 9y ago

"I'll just escape it" is a typical and yet unacceptable response to this. There seems to be some sort of "cult" (for lack of better term) around this mentality of "it's too hard so don't do it", which unfortunately only makes things worse overall. Escaping rules are well-defined (they must be, by necessity), and sooner or later you're going to have to do something involving it; not to mention the important superset of things to keep in mind when writing code, of which escaping is only a fraction: the general principle of "consider all input". Always assume that all 256 values of a byte can and will show up in any external input, and plan accordingly.

asveikau 9y ago
> Escaping rules are well-defined Except when they aren't? I'm thinking of things like the bash exported function bug, an obscure feature nobody thought was there turns evaluating environment variables into RCE. Not to mention different shells may do things slightly differently and you may get caught doing something unexpected. So why not leave the shell out of the picture entirely? I think execve and friends are much easier to get right than escaping. There are still possible security bugs to consider (eg., if using the -p family such as execvp, who controls PATH? Is anything on PATH writeable?), but the shell won't be among them.
- blindgeek 9y ago
  As often as I've seen people get shell-escaping wrong, escaping rules probably aren't well-understood, regardless of how well-defined they are. Bourne shell's quoting rules are byzantine at best. There are ways to reliably and securely quote for the Bourne shell, but the best solution is not to pass untrusted input to the shell. Unrelated to the shell, I've seen people -- including myself -- fail at dot-stuffing. Experience has taught me to avoid hand-rolled escaping when I can; escaping is definitely a code smell.
3pt14159 9y ago
It just gets kinda exhausting. If you whitelist characters you have constant maintenance, if you don't some stupid edge case slips through. You're right of course, but I sometimes get discouraged at how much work every little thing is.
geofft 9y ago
> There seems to be some sort of "cult" (for lack of better term) around this mentality of "it's too hard so don't do it" This cult is called engineering reliable systems. If you want to build a system that actually works (instead of mathematically prove that a workable system could exist), you have to take the following into account: - Describing a complicated system is harder than describing a simple system; there may be spec bugs. - Implementing a complicated system is harder than implementing a simple system; there are almost certainly implementation bugs, and they are not evenly distributed throughout the entire implementation. - Code that isn't regularly exercised with full and meaningful coverage is likely to have unexplored bugs, so if you're using third-party code (like a shell), there's little sense in building your system on top of the most obscure parts of it. - Full and meaningful coverage could mean tests, but is not very likely to, because humans are just not very good at writing good tests and keeping up with them. Humans who write your third-party dependencies (like your /bin/sh) are not subject to your organization's policies about test coverage. - Code changes, and realistically you have to keep up with changes from external code. Code that matches your expectations today might fail to match them tomorrow, for the same reason that the obscure parts might change because no one is using them. - You, personally, are fallible. You will have off days. You will forget the edge case you thought of last week. You will come back to your code a year later and not understand the complexity, and then you will be responsible for figuring out the complexity. - You're not the only person maintaining your code, and even if you had a hiring veto, you're not going to be able to enforce that everyone on your team has the same sense for the same complexities that come naturally to you. Your complex code will get modified by a teammate when you're on vacation, and then you'll get paged for it when you return. It's in your own interest not to set up this situation. - In general, the people implementing and running your systems are humans, and they have all sorts of weird human incentives: shipping the feature instead of spending tons of time on the edge cases, fixing the broken system in the fastest way possible when it fails at 4:30 on a Friday, not correcting their more-senior teammate's code in reviews if they're not confident they're right, etc. You are much more likely to succeed at engineering around these failure modes than getting humans to stop having human faults. - If doing the complex thing is inevitable, there's an art to figuring out whether you should do so rarely to minimize the number of mistakes, or frequently so that you get good at it. Both are reasonable options and the better choice depends a lot on the risks. And even if you want to take the mathematical proof route, you're still subject to incompleteness. I'd much prefer the approach that empirically gives me a high chance of building reliable systems in practice. In particular, I disagree with the claim "Escaping rules are well-defined (they must be, by necessity)" - there is no necessity that the specification be complete, and there's certainly no necessity that whatever implementation you're using faithfully implements all the rules.

codemac 9y ago

This is sanitizing input, yes? Didn't seem to be too specific to subprocesses, just some places where folks use input directly.

StefanKarpinski 9y ago

I had similar thoughts some time ago but came to a very different conclusion: anything the shell can do a programming language could do just as well, given the right interface. The fact that languages don't make it easy to spawn subprocesses safely doesn't mean that they couldn't do so. The right solution is making it both convenient and safe to do shell-like things from within a real programming language. So I made sure that Julia does this right: https://julialang.org/blog/2012/03/shelling-out-sucks https://julialang.org/blog/2012/03/shelling-out-sucks https://julialang.org/blog/2013/04/put-this-in-your-pipe https://julialang.org/blog/2013/04/put-this-in-your-pipe

elcritch 9y ago
Never realized you could do that. Hey hank you, that’s handy! One question though, is it possible to intersperse pure Julia functions but still have them piped, e.g. write custom “shell” commends?
- StefanKarpinski 9y ago
  It's not currently, but I'm planning on adding that feature in a future version along with making shell metacharacters like `>` and `|` act much like they do shells when they appear inside of Julia command literals. When we do that, we'll be able to support splicing Julia functions and/or tasks into pipelines.
  - elcritch 9y ago
    That'd be fun! I'll keep an eye out for it. Yesterday I was playing around with `eshell` in Emacs and the ability to redirect command output into buffers and modify them with code turns out surprisingly handy.
bjoli 9y ago
Scsh also does this right. It makes me sad that no one actually successfully ported it to other schemes.

ggm 9y ago

Because the shell is a generalized engine for executing things, and because magic defines textfiles as candidate executables nominating either the shell or a shell-exec binary to interpret them, and because system() jacks all of this to say you can invoke a shell with all its awesome to invoke scripts which invoke a binary to parse them. If the shell was only rsh, and if the set of binaries you can invoke was constrained, and if the network and system calls were accessed through strace() barriers which limited what you could do.. We might have less problems. Except that in the end, people don't code or script for secure execution so the context of 'what harm can I do from here' turns out to be a lot wider than many people think.

AstralStorm 9y ago

The premise is faulty. Subprocesses are fine, it is the POSIX shell that should be avoided. Or any interpreted language. Smallest tool for the job. Use exec* calls, right options for your process handling class and there is no problem. As a bonus feel free to spawn/fork your current application. Linking pipes and file descriptors is also not that hard. (Though they are a cruddy that may be nicer to wrap in something like 0mq.)