7 ms·

Shelling Out Sucks

77 points by vgnet 15y ago

nessus42 15y ago
I remember ages ago getting in an argument with the kernel hackers on unix-wizards. My position was that filenames shouldn't be allowed to have meta-characters in them because it made shell programming difficult and dangerous. Their argument was that the spirit of Unix was that the kernel should be as simple as it possibly could be, and that you should "Live free or die". Therefore it was not the kernel's business to dictate what characters should be able to be in a filename. They were wrong of course. The true spirit of Unix was to enable programmer productivity by being able to easily piece together new programs out of reusable components. E.g., via shell scripting, and the kernel hackers' notions about filenames and living free was directly at odds with this true spirit. The OP claims that they've solved this dilemma in Julia. I await with bated breath.
- rcthompson 15y ago
  How can the kernel possibly dictate what filenames are acceptable if it wants to interoperate with filesystems created by other operating systems?
  - nessus42 15y ago
    if it wants to interoperate with filesystems created by other operating systems? That, of course, was not an issue back in the day, as Unix did not in fact support anything such thing. As for today, please do tell me how Unix can interoperate with filesystems that allow nulls and/or forward slashes in filenames. I'm sure that you might come up with an existing or potential solution, and well, there's your answer!
- perlgeek 15y ago
  Note that different shells have different meta characters. Which one would you forbid? All of them? Then I'm sure somebody comes up with a shell where nearly all characters are meta characters, leaving only one or two to make up file names.
  - nessus42 15y ago
    Back in the day, there were only two shells: The Bourne Shell and the C Shell. And before that there was only whatever the Bourne Shell was based on. In any case, the answer to this is that the kernel developers and the shell developers get together and decide on something, and then future shells have to live within those constraints. At the point that they should have done this, they should have made sure that this was flexible enough for future needs. Personally, I would have been fine if filenames were restricted to more or less to the same degree that variables names are restricted in, let's say, C. Why can't I name my variables whatever I want to? I thought it was "Live free or die!"?!?!?!?!
    - jarman 15y ago
      YES! And while we are at it, let's get rid of SQL injections by forbidding spaces and quotes in [VAR]CHAR fields! Limiting valid file names won't protect from injections, won't make such protection easier (asserting lack of special characters isn't significantly easier than escaping same characters), but will cripple system functionality. Personally_comma_I_would_hate_seeing_such_file_names.
      - nessus42 15y ago
        This discussion has ABSOLUTELY NOTHING to do with SQL injection. That is a completely different issue, as you may indeed want and need to put all of War and Peace into a char field, but there is no valid use case for putting all of War and Peace into a filename. Furthermore, it was quite traditional for Unix filenames to be written exactly has you have done so above. (Although dashes would have been more commonly used than underscores.) And, in fact, it was considered fairly anti-social to put spaces, carriage returns, etc., in your filenames, as Unix utilities could not cope with them. The fact that for decades Unix utilities could only cope with filenames written as you have above, proves my point, not yours. Even now, when Unix utilities are finally getting to the point where they might be able to cope with any combination of characters in a filename (other than null or "/"), this has come at great extra complexity and requiring much extra care on the part of the programmer. Which again proves my point, not yours. Now, of course, whenever anyone asserts that something requires too much care, someone else will quite wrongly assert that programmers should not be encouraged to be lazy. That real programmers should work for a living. This is, of course, absurd. The best programmers will always be lazy... in the right way. They will want to solve a problem that accomplishes the most, in the least amount of time, with the least amount of code. I.e., they will want to be productive. People who make quips about lazy programmers are doing a huge disservice to the world by promulgating a world with less productive programmers who consequently accomplish less. Which means that, for all you know, cancer won't be cured when otherwise it might have been. As to your proposed solution of having the shell escape special characters, rather than having the kernel disallow them, that could probably work too, at the cost of a consistency. I.e., when you look at a filename in the shell, it's going to look different from how it will look in your GUI browser. E.g., the shell is going to display your filename in the manner that you have done, only it probably won't do as good a job as you would at encoding your intent into a limited character set. And, if you are going to take this approach, it doesn't have to be by allowing special characters in filenames. It can be done either by encoding special characters in non-special characters, or by having a display name that is different from the filename. Additionally, I find it amusing when people assert that having limitations in filenames is crippling system functionality, and yet they don't make the same assertion about identifier names in programming languages. Oh no! Having a limited set of characters in variable names has crippled my ability to program... NOT! In summary, history has proved me right.
- tesseract 15y ago
  > The OP claims that they've solved this dilemma in Julia. I await with bated breath. The obvious solution would seem to be to provide an easy way to directly create a pipeline from within Julia. I.e. you use syntax similar to shell syntax, but rather than spawning a shell Julia spawns the desired processes directly. That way it can directly pass arguments to execv (avoiding escaping issues altogether), wait on all of the child processes individually (allowing clean and detailed error detection), and avoid the overhead of actually starting a shell.
  - deleted 15y ago
    [deleted]
mappu 15y ago
> Although shell escaping does work, realistically, no one actually bothers — it’s too much trouble. I bother : ( Next you're going to tell me that nobody bothers to sanitise input in their SQL queries. Well, i suppose that's true, in a sense. Maybe the solution is to follow SQL and implement a prepared-query system for shell commands?
- hesselink 15y ago
  > Maybe the solution is to follow SQL and implement a prepared-query system for shell commands? You mean shell scripts?
  - angelbob 15y ago
    No. Shell scripts have the same problem where you're not separating options from filenames and you're often not preventing a single user-supplied argument from expanding into multiple final arguments. The list-form shell commands referenced earlier are closer, but still don't separate filenames (input) from arguments unless you remember to add "--". But they do prevent things like pipes, which is a big deal.
sigil 15y ago
1. Shell-escaping all values used to construct commands Avoid shell-escaping and the shell altogether! Use list form program execution wherever possible. In Perl: open(my $fh, "-|", "find", $dir, "-type", "f", "-print0") or die; In Python: f = os.popen2(["find", dir, "-type", "f", "-print0"]) If you're executing a pipeline, you do need to get the shell involved. But I'd probably write the author's examples like so (once again avoiding the need to shell escape arguments): f = os.popen2(["sh", "-c", \ 'find "$0" -type f -print0 | xargs -0 grep foo | wc -l', dir]) 2. Prefixing each multi-command pipeline with “set -o pipefail; Alas, pipefail is not in POSIX /bin/sh, which many of us prefer for shell scripting. It's in bash and ksh though. 3. Explicitly checking for failure after each shelled out command Valid. You should always do this. In shell scripts, "set -e" is a good thing, although pipelines without "set -o pipefail" are still a problem.
- zwp 15y ago
  The article does not address another significant point: filenames should also be classified as untrusted input. Using -- to separate flags from arguments can help: /tmp$ mkdir x /tmp$ cd x /tmp/x$ echo hello > a /tmp/x$ grep hello * hello /tmp/x$ touch -- -q /tmp/x$ grep hello * /tmp/x$ grep hello -- * a:hello /tmp/x$ It's rare to see protection against this problem either when shelling out or in native shell scripts.
- joeyh 15y ago
  Your quoting of "$0" does not protect it from the shell. An attacker need only provide an input such as: /tmp"; rm -rf /; " There are pipeline libraries that avoid the shell for several languages. Some I can recommend: * libpipeline http://libpipeline.nongnu.org/ http://libpipeline.nongnu.org/ easy, if somewhat verbose, pipeline construction in C. * hsh https://github.com/jgoerzen/hsh/wiki https://github.com/jgoerzen/hsh/wiki makes pipelines in haskell, using operators so they really look like pipelines, but without involving the shell. Example: "ls -l" -|- "wc -l"
  - sigil 15y ago
    Your quoting of "$0" does not protect it from the shell. An attacker need only provide an input such as: /tmp"; rm -rf /; " Not true. $ sh -c 'ls "$0"' '/tmp"; rm -rf /; "' ls: cannot access /tmp"; rm -rf /; ": No such file or directory If you were right, there would be no reasonable way to write shell scripts.
emmelaich 15y ago
I think the silent failures are the worse of the three by far. Also, related, very few unix commands have a consistent easy to parse format. I have a dream -- a unix-like OS with a strong convention (or even enforcement!) of file typing and format. Optional perhaps. The efficiency aspect is often exaggerated. As long as you do a fair bit of work in the spawned command the fork/exec cost disappears to nothing. Also, it should be rare in any decent language.
- raggi 15y ago
  The efficiency aspect can creep up on people easily though, regardless of language. It's the standard concern: you don't know how your code will be used in future. One example: http://tenderlovemaking.com/2011/12/05/profiling-rails-startup-with-dtrace/ http://tenderlovemaking.com/2011/12/05/profiling-rails-start...
- omaranto 15y ago
  Have you used Microsoft Powershell? It (indirectly) does a useful portion of what you're dreaming about. It's a UNIX like shell where commands pipe .NET objects instead of plain text. Just like UNIX commands take and produce arbitrary text but most think of it as a sequence of lines, Powershell commands can take and produce arbitrary .NET objects but most use an object which is a collection of records. Powershell comes with builtin commands for most of the simple UNIX utilities, and you build new commands in any .NET language. Additionally, you can interop with UNIX style programs that produce plain text by using a parsing filter (there are a few builtin, you can make more). It's actually pretty good and a lot more regular and modern-feeling than any UNIX shell I've tried. I still prefer to use GNU/Linux, but wish there was a good port of Powershell...
rwmj 15y ago
This article is nonsense. In libguestfs[1] we invoke shell commands for many filesystem utilities ... from C which (if you believe the article) would be the worst place to do this. All of the problems are avoided by having a smart way to run external commands and capture their errors. See the functions [2] and examples of use [3] [4]. [1] http://libguestfs.org http://libguestfs.org [2] https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab18bfffc7e4397531192610a7371/daemon/daemon.h#L59 https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab1... [3] https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab18bfffc7e4397531192610a7371/daemon/mkfs.c#L36 https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab1... [4] https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab18bfffc7e4397531192610a7371/daemon/ext2.c#L359 https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab1...
- hesselink 15y ago
  Why is it nonsense? It's arguing exactly for your solution: a smart way to run external commands and capture their errors.
luriel 15y ago
The rc shell by Tom Duff (original for Unix 10th Edition, later more known as the Plan 9 shell) got escaping right: http://rc.cat-v.org http://rc.cat-v.org After using rc for scripting for years, I dread writing more than a couple of lines of sh (and csh is best forgotten).
- omaranto 15y ago
  If you like rc, check out es [^1]. It's inspired by rc but has higher order functions and a few other things to support functional programming. [^1]: http://192.220.96.201/es/es-usenix-winter93.html http://192.220.96.201/es/es-usenix-winter93.html
ggchappell 15y ago
The article is quite correct, of course. But this kind of bug gets tiresome. We saw the same kinds of troubles with SQL for years. And we'll see it again the next time we get a popular technique that involves code that generates code using outside input, and passes it as plain text. So we need a new rule: If you're going to provide an interface that allows programs to generate code and pass it to something, then you need to deal with these problems proactively. You can provide sanitizing/escaping functionality. Or you can avoid passing code as plain text. Or you can do something else just as good. And proper error handling needs to be the default. But if you don't address these issues, then your interface is lousy, and it deserves to be treated as such.
mark_l_watson 15y ago
I can't really agree with the article. I use Ruby a lot for small one-off utility programs that might be used for a day to get something done. Using the back tick quotes to run external processes and capture stdout, I can solve real problems in a few lines of code. I don't think that I have every been bit by the "silent failure" problem mentioned in the article. A little off topic, but I don't throw away small one-off utility programs when I am done with them. Instead I rename them to long and meaningful file names, toss them in a special directory, and save even more time in the future by hacking on them for similar purposes. I may be typical: although I usually design and implement robust systems with a long life, I also work a lot with data and this requires a lot of sloppy little bits of code to change data formats, get statistics, etc. I do agree that shelling out in production code would suck.
- StefanKarpinski 15y ago
  The trouble is that so much production code starts as a one-off. Worse still, in a startup environment there's often not much distinction between production code and one-off code.
cheatercheater 15y ago
Check out ShQQ, a Haskell module that does something like that: http://mainisusuallyafunction.blogspot.com/2011/10/shqq-embedding-shell-commands-in.html http://mainisusuallyafunction.blogspot.com/2011/10/shqq-embe... > λ> let x = "/proc/uptime" in [sh| sha1sum $x |] > "337ec3fb998fb3a4650a18e0785f0992762b3cda /proc/uptime\n"
- rcthompson 15y ago
  Yeah, I kind of figured Haskell would have solved this problem in a way that makes the typical syntax do the right thing (shell-quote the interpolated variables) and then provides an override for those few cases when you're doing something weird.

77 points by vgnet 15y ago

I remember ages ago getting in an argument with the kernel hackers on unix-wizards. My position was that filenames shouldn't be allowed to have meta-characters in them because it made shell programming difficult and dangerous. Their argument was that the spirit of Unix was that the kernel should be as simple as it possibly could be, and that you should "Live free or die". Therefore it was not the kernel's business to dictate what characters should be able to be in a filename. They were wrong of course. The true spirit of Unix was to enable programmer productivity by being able to easily piece together new programs out of reusable components. E.g., via shell scripting, and the kernel hackers' notions about filenames and living free was directly at odds with this true spirit. The OP claims that they've solved this dilemma in Julia. I await with bated breath.

rcthompson 15y ago
How can the kernel possibly dictate what filenames are acceptable if it wants to interoperate with filesystems created by other operating systems?
- nessus42 15y ago
  if it wants to interoperate with filesystems created by other operating systems? That, of course, was not an issue back in the day, as Unix did not in fact support anything such thing. As for today, please do tell me how Unix can interoperate with filesystems that allow nulls and/or forward slashes in filenames. I'm sure that you might come up with an existing or potential solution, and well, there's your answer!
perlgeek 15y ago
Note that different shells have different meta characters. Which one would you forbid? All of them? Then I'm sure somebody comes up with a shell where nearly all characters are meta characters, leaving only one or two to make up file names.
- nessus42 15y ago
  Back in the day, there were only two shells: The Bourne Shell and the C Shell. And before that there was only whatever the Bourne Shell was based on. In any case, the answer to this is that the kernel developers and the shell developers get together and decide on something, and then future shells have to live within those constraints. At the point that they should have done this, they should have made sure that this was flexible enough for future needs. Personally, I would have been fine if filenames were restricted to more or less to the same degree that variables names are restricted in, let's say, C. Why can't I name my variables whatever I want to? I thought it was "Live free or die!"?!?!?!?!
  - jarman 15y ago
    YES! And while we are at it, let's get rid of SQL injections by forbidding spaces and quotes in [VAR]CHAR fields! Limiting valid file names won't protect from injections, won't make such protection easier (asserting lack of special characters isn't significantly easier than escaping same characters), but will cripple system functionality. Personally_comma_I_would_hate_seeing_such_file_names.
    - nessus42 15y ago
      This discussion has ABSOLUTELY NOTHING to do with SQL injection. That is a completely different issue, as you may indeed want and need to put all of War and Peace into a char field, but there is no valid use case for putting all of War and Peace into a filename. Furthermore, it was quite traditional for Unix filenames to be written exactly has you have done so above. (Although dashes would have been more commonly used than underscores.) And, in fact, it was considered fairly anti-social to put spaces, carriage returns, etc., in your filenames, as Unix utilities could not cope with them. The fact that for decades Unix utilities could only cope with filenames written as you have above, proves my point, not yours. Even now, when Unix utilities are finally getting to the point where they might be able to cope with any combination of characters in a filename (other than null or "/"), this has come at great extra complexity and requiring much extra care on the part of the programmer. Which again proves my point, not yours. Now, of course, whenever anyone asserts that something requires too much care, someone else will quite wrongly assert that programmers should not be encouraged to be lazy. That real programmers should work for a living. This is, of course, absurd. The best programmers will always be lazy... in the right way. They will want to solve a problem that accomplishes the most, in the least amount of time, with the least amount of code. I.e., they will want to be productive. People who make quips about lazy programmers are doing a huge disservice to the world by promulgating a world with less productive programmers who consequently accomplish less. Which means that, for all you know, cancer won't be cured when otherwise it might have been. As to your proposed solution of having the shell escape special characters, rather than having the kernel disallow them, that could probably work too, at the cost of a consistency. I.e., when you look at a filename in the shell, it's going to look different from how it will look in your GUI browser. E.g., the shell is going to display your filename in the manner that you have done, only it probably won't do as good a job as you would at encoding your intent into a limited character set. And, if you are going to take this approach, it doesn't have to be by allowing special characters in filenames. It can be done either by encoding special characters in non-special characters, or by having a display name that is different from the filename. Additionally, I find it amusing when people assert that having limitations in filenames is crippling system functionality, and yet they don't make the same assertion about identifier names in programming languages. Oh no! Having a limited set of characters in variable names has crippled my ability to program... NOT! In summary, history has proved me right.
tesseract 15y ago
> The OP claims that they've solved this dilemma in Julia. I await with bated breath. The obvious solution would seem to be to provide an easy way to directly create a pipeline from within Julia. I.e. you use syntax similar to shell syntax, but rather than spawning a shell Julia spawns the desired processes directly. That way it can directly pass arguments to execv (avoiding escaping issues altogether), wait on all of the child processes individually (allowing clean and detailed error detection), and avoid the overhead of actually starting a shell.
- deleted 15y ago
  [deleted]

mappu 15y ago

> Although shell escaping does work, realistically, no one actually bothers — it’s too much trouble. I bother : ( Next you're going to tell me that nobody bothers to sanitise input in their SQL queries. Well, i suppose that's true, in a sense. Maybe the solution is to follow SQL and implement a prepared-query system for shell commands?

hesselink 15y ago
> Maybe the solution is to follow SQL and implement a prepared-query system for shell commands? You mean shell scripts?
- angelbob 15y ago
  No. Shell scripts have the same problem where you're not separating options from filenames and you're often not preventing a single user-supplied argument from expanding into multiple final arguments. The list-form shell commands referenced earlier are closer, but still don't separate filenames (input) from arguments unless you remember to add "--". But they do prevent things like pipes, which is a big deal.

sigil 15y ago

1. Shell-escaping all values used to construct commands Avoid shell-escaping and the shell altogether! Use list form program execution wherever possible. In Perl: open(my $fh, "-|", "find", $dir, "-type", "f", "-print0") or die; In Python: f = os.popen2(["find", dir, "-type", "f", "-print0"]) If you're executing a pipeline, you do need to get the shell involved. But I'd probably write the author's examples like so (once again avoiding the need to shell escape arguments): f = os.popen2(["sh", "-c", \ 'find "$0" -type f -print0 | xargs -0 grep foo | wc -l', dir]) 2. Prefixing each multi-command pipeline with “set -o pipefail; Alas, pipefail is not in POSIX /bin/sh, which many of us prefer for shell scripting. It's in bash and ksh though. 3. Explicitly checking for failure after each shelled out command Valid. You should always do this. In shell scripts, "set -e" is a good thing, although pipelines without "set -o pipefail" are still a problem.

zwp 15y ago
The article does not address another significant point: filenames should also be classified as untrusted input. Using -- to separate flags from arguments can help: /tmp$ mkdir x /tmp$ cd x /tmp/x$ echo hello > a /tmp/x$ grep hello * hello /tmp/x$ touch -- -q /tmp/x$ grep hello * /tmp/x$ grep hello -- * a:hello /tmp/x$ It's rare to see protection against this problem either when shelling out or in native shell scripts.
joeyh 15y ago
Your quoting of "$0" does not protect it from the shell. An attacker need only provide an input such as: /tmp"; rm -rf /; " There are pipeline libraries that avoid the shell for several languages. Some I can recommend: * libpipeline http://libpipeline.nongnu.org/ http://libpipeline.nongnu.org/ easy, if somewhat verbose, pipeline construction in C. * hsh https://github.com/jgoerzen/hsh/wiki https://github.com/jgoerzen/hsh/wiki makes pipelines in haskell, using operators so they really look like pipelines, but without involving the shell. Example: "ls -l" -|- "wc -l"
- sigil 15y ago
  Your quoting of "$0" does not protect it from the shell. An attacker need only provide an input such as: /tmp"; rm -rf /; " Not true. $ sh -c 'ls "$0"' '/tmp"; rm -rf /; "' ls: cannot access /tmp"; rm -rf /; ": No such file or directory If you were right, there would be no reasonable way to write shell scripts.

emmelaich 15y ago

I think the silent failures are the worse of the three by far. Also, related, very few unix commands have a consistent easy to parse format. I have a dream -- a unix-like OS with a strong convention (or even enforcement!) of file typing and format. Optional perhaps. The efficiency aspect is often exaggerated. As long as you do a fair bit of work in the spawned command the fork/exec cost disappears to nothing. Also, it should be rare in any decent language.

raggi 15y ago
The efficiency aspect can creep up on people easily though, regardless of language. It's the standard concern: you don't know how your code will be used in future. One example: http://tenderlovemaking.com/2011/12/05/profiling-rails-startup-with-dtrace/ http://tenderlovemaking.com/2011/12/05/profiling-rails-start...
omaranto 15y ago
Have you used Microsoft Powershell? It (indirectly) does a useful portion of what you're dreaming about. It's a UNIX like shell where commands pipe .NET objects instead of plain text. Just like UNIX commands take and produce arbitrary text but most think of it as a sequence of lines, Powershell commands can take and produce arbitrary .NET objects but most use an object which is a collection of records. Powershell comes with builtin commands for most of the simple UNIX utilities, and you build new commands in any .NET language. Additionally, you can interop with UNIX style programs that produce plain text by using a parsing filter (there are a few builtin, you can make more). It's actually pretty good and a lot more regular and modern-feeling than any UNIX shell I've tried. I still prefer to use GNU/Linux, but wish there was a good port of Powershell...

rwmj 15y ago

This article is nonsense. In libguestfs[1] we invoke shell commands for many filesystem utilities ... from C which (if you believe the article) would be the worst place to do this. All of the problems are avoided by having a smart way to run external commands and capture their errors. See the functions [2] and examples of use [3] [4]. [1] http://libguestfs.org http://libguestfs.org [2] https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab18bfffc7e4397531192610a7371/daemon/daemon.h#L59 https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab1... [3] https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab18bfffc7e4397531192610a7371/daemon/mkfs.c#L36 https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab1... [4] https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab18bfffc7e4397531192610a7371/daemon/ext2.c#L359 https://github.com/libguestfs/libguestfs/blob/d0cf52b9118ab1...

hesselink 15y ago
Why is it nonsense? It's arguing exactly for your solution: a smart way to run external commands and capture their errors.

luriel 15y ago

The rc shell by Tom Duff (original for Unix 10th Edition, later more known as the Plan 9 shell) got escaping right: http://rc.cat-v.org http://rc.cat-v.org After using rc for scripting for years, I dread writing more than a couple of lines of sh (and csh is best forgotten).

omaranto 15y ago
If you like rc, check out es [^1]. It's inspired by rc but has higher order functions and a few other things to support functional programming. [^1]: http://192.220.96.201/es/es-usenix-winter93.html http://192.220.96.201/es/es-usenix-winter93.html

ggchappell 15y ago

The article is quite correct, of course. But this kind of bug gets tiresome. We saw the same kinds of troubles with SQL for years. And we'll see it again the next time we get a popular technique that involves code that generates code using outside input, and passes it as plain text. So we need a new rule: If you're going to provide an interface that allows programs to generate code and pass it to something, then you need to deal with these problems proactively. You can provide sanitizing/escaping functionality. Or you can avoid passing code as plain text. Or you can do something else just as good. And proper error handling needs to be the default. But if you don't address these issues, then your interface is lousy, and it deserves to be treated as such.

mark_l_watson 15y ago

I can't really agree with the article. I use Ruby a lot for small one-off utility programs that might be used for a day to get something done. Using the back tick quotes to run external processes and capture stdout, I can solve real problems in a few lines of code. I don't think that I have every been bit by the "silent failure" problem mentioned in the article. A little off topic, but I don't throw away small one-off utility programs when I am done with them. Instead I rename them to long and meaningful file names, toss them in a special directory, and save even more time in the future by hacking on them for similar purposes. I may be typical: although I usually design and implement robust systems with a long life, I also work a lot with data and this requires a lot of sloppy little bits of code to change data formats, get statistics, etc. I do agree that shelling out in production code would suck.

StefanKarpinski 15y ago
The trouble is that so much production code starts as a one-off. Worse still, in a startup environment there's often not much distinction between production code and one-off code.

cheatercheater 15y ago

Check out ShQQ, a Haskell module that does something like that: http://mainisusuallyafunction.blogspot.com/2011/10/shqq-embedding-shell-commands-in.html http://mainisusuallyafunction.blogspot.com/2011/10/shqq-embe... > λ> let x = "/proc/uptime" in [sh| sha1sum $x |] > "337ec3fb998fb3a4650a18e0785f0992762b3cda /proc/uptime\n"

rcthompson 15y ago
Yeah, I kind of figured Haskell would have solved this problem in a way that makes the typical syntax do the right thing (shell-quote the interpolated variables) and then provides an override for those few cases when you're doing something weird.