2 ms·

I was intrigued by the author's library "envoy", which is intended to provide a more intuitive interface to running processes from python. (https://github.com/k

I was intrigued by the author's library "envoy", which is intended to provide a more intuitive interface to running processes from python. (https://github.com/kennethreitz/envoy https://github.com/kennethreitz/envoy) The back story is that the older APIs that Python comes with -- os.popen and os.system -- are deprecated. Programmers are urged to use the "subprocess" module instead. Although this doesn't have the problems of the original functions, it has a rather arcane interface, in particular if you want to read the output (stdout or stderr) of a subprocess. "envoy" seems to aim at fixing this, by providing sane defaults and being optimized for the common case. However, these defaults have drawbacks of their own. 1. envoy defaults to keeping the process output in memory, as a giant string. This can be a bad choice with regard to memory usage and performance. 2. You can run several processes in a pipe using ("cat foo | grep bla"). But otherwise as far as I can see, run() ignores regular Shell semantics, such as quotes. I imagine this can lead to unexpected results. The amount of data passed from one process to the next is capped at 10 MB -- recipe for bugs that are hard to find. 3. subprocess.call() accepts an array in the style of ["ls", "-l", "/mnt/My SD card"]. This has obvious advantages over having to deal with escaping shell characters. A good API should preserve this advantage over os.system(). 4. The defaults cannot be overridden, and no preperations have been made to allow changing them. Of course this can be changed in the future. However, one of the reasons the subprocess.* API is convoluted is that it allows all kinds of flexibility, much of which is needed in many serious programs. It may be difficult to add this flexibility to envoy at a later stage. The point is that a flexible API is hard. None of this is to discourage this initiative, which seems to me a much-needed improvement over Python's built-in API. Also, with a version number as low as 0.0.2, there is probably little need to worry about API compatibility.

bobbyi 15y ago
> it has a rather arcane interface, in particular if you want to read the output (stdout or stderr) of a subprocess. oputput = subprocess.check_output(my_command)
Silhouette 15y ago
> subprocess.call() accepts an array in the style of ["ls", "-l", "/mnt/My SD card"]. This has obvious advantages over having to deal with escaping shell characters. Unless you're running on Windows, in which case IME it will corrupt your carefully constructed parameters in completely inappropriate ways that can be debugged only at the cost of (a) changing the call() to execute a script that dumps the actual parameters supplied verbatim, and (b) at least an hour of your life that you're never getting back. This "feature" is about one step above MS Word's default autoreplace behaviour in irritation level. What happened to "Explicit is better than implicit" and "Special cases aren't special enough to break the rules"?
- loevborg 15y ago
  On Unix, a new process is supplied argv[], an array that contains the executable name and invidual arguments. Clearly, supplying call() a list of arguments is the right thing to do. I seem to remember that on Win32, all you get is an argument string, and the process is required to do the parsing itself. This is simply a different model, and the Unix way is cleaner and easier to work with. It seems to me reasonable to optimize for the saner (Unix) way of doing things instead. I confess I don't understand your analogy to Word.
  - Silhouette 15y ago
    The big difference is that the UNIX shell does all kinds of expansion/interpolation for you, while Windows basically leaves things alone. You get an argv[] array in both cases if you're writing in C. Neither of these approaches is inherently superior, they're just placing responsibility for certain operations in different places. But the fact is that because Windows programs don't have to assume a certain set of conventions for their command line, many do not, and if you have the misfortune to want to automate those using subprocess, you're in for a world of pain (until you just give up and use the single-string version instead of the list of arguments, having realised that this is enough to stop it messing around with your carefully crafted strings and just pass them through verbatim).

bobbyi 15y ago

> it has a rather arcane interface, in particular if you want to read the output (stdout or stderr) of a subprocess. oputput = subprocess.check_output(my_command)

Silhouette 15y ago

> subprocess.call() accepts an array in the style of ["ls", "-l", "/mnt/My SD card"]. This has obvious advantages over having to deal with escaping shell characters. Unless you're running on Windows, in which case IME it will corrupt your carefully constructed parameters in completely inappropriate ways that can be debugged only at the cost of (a) changing the call() to execute a script that dumps the actual parameters supplied verbatim, and (b) at least an hour of your life that you're never getting back. This "feature" is about one step above MS Word's default autoreplace behaviour in irritation level. What happened to "Explicit is better than implicit" and "Special cases aren't special enough to break the rules"?

loevborg 15y ago
On Unix, a new process is supplied argv[], an array that contains the executable name and invidual arguments. Clearly, supplying call() a list of arguments is the right thing to do. I seem to remember that on Win32, all you get is an argument string, and the process is required to do the parsing itself. This is simply a different model, and the Unix way is cleaner and easier to work with. It seems to me reasonable to optimize for the saner (Unix) way of doing things instead. I confess I don't understand your analogy to Word.
- Silhouette 15y ago
  The big difference is that the UNIX shell does all kinds of expansion/interpolation for you, while Windows basically leaves things alone. You get an argv[] array in both cases if you're writing in C. Neither of these approaches is inherently superior, they're just placing responsibility for certain operations in different places. But the fact is that because Windows programs don't have to assume a certain set of conventions for their command line, many do not, and if you have the misfortune to want to automate those using subprocess, you're in for a world of pain (until you just give up and use the single-string version instead of the list of arguments, having realised that this is enough to stop it messing around with your carefully crafted strings and just pass them through verbatim).