4 ms·
I disagree. Here's a warning from subprocess[1]: > Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of th
by ben509 7y ago
I disagree. Here's a warning from subprocess[1]:
> Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
The trouble with `communicate()` is it only handles very simple cases, basically, your output has to fit in memory. Same problem exists in asyncio.[2]
Yes, you can usually work around this. That doesn't mean it's a good replacement; whereas complex pipelines are so trivial in bash that any user-defined function can be used in a pipeline, subprocess generally forces you to do the dumb thing and create a mess of temporary files.
And that's not even considering you're writing 10 times as much code than you would to accomplish the same task.
[1]: https://docs.python.org/3/library/subprocess.html#subprocess.Popen.stderr https://docs.python.org/3/library/subprocess.html#subprocess...
[2]: https://docs.python.org/3/library/asyncio-subprocess.html#asyncio.asyncio.subprocess.Process.stderr https://docs.python.org/3/library/asyncio-subprocess.html#as...
- kortex 7y agoWhy is this still such a problem? People have been talking about replacing bash with python for years, and yet trying to actually do just that, is a pain. Plumbum doesn't quite cut it. I think in part it's because the POSIX-style shell is so tightly wound with the OS, it's extremely proficient at spawning and forking processes, working with files, and communicating, and that experience is seamless. Python feels like a different world, and doing any subproc, pipes, or file i/o always feels like crossing some boundary and back.
- detaro 7y agoI feel like this could be a situation parallel to JS on the web: You can change the language that's available everywhere only slowly and in limited ways, so people build compilers targeting it as the output language from "nicer" languages.
- lalaland1125 7y agoYes, stdin.write, stdout.read and stderr.read can cause deadlocks if not used carefully. However, those operations are rarely needed and are complicated in Bash as well (usually requiring sub-shells, etc). 99% of Bash scripts can be written without using those operations (you simply pipe input/output from one program to the next rather than trying to manually forward it in your program). This is actually one of the main advantages of using Python. You can solve all those deadlock issues through careful use of threads, which are much better than the Bash sub-shell equivalent. If anyone has an example script that requires the use of those functions and is cumbersome in Python, please share it. We can always improve the Python standard library if necessary.