28 ms·

I've primarily used Python 2 for 10+ years and I often find cases where shell scripts are preferable. The major differentiator is usually "shelling out" in Pyt

by pfranz 7y ago

I've primarily used Python 2 for 10+ years and I often find cases where shell scripts are preferable. The major differentiator is usually "shelling out" in Python kind of sucks. It's verbose, output collection and error handling suck, and escaping can be miserable. I often will reimplement things in pure Python if I have the time. A recent example was I needed to tar+split large files. `tar cf - -C / $filename | split --bytes ${size}MB --verbose - $tmpfile.` My pure-Python implementation used the Tarfile library and I wrote a custom file-like object to split it--at least 50 lines. Both methods have pros/cons, but I had both implementations handy depending on the context I needed it in. Another recent example was something I wrote to merge multiple video file "parts" into a single video using ffmpeg. It's 4 lines of shell script and would at least be 3x as long to write in Python. The rule of thumb I have is if it's longer than 20 lines reevaluate if a shell scripting language is appropriate and I haven't really seen a need to change that in ~15 years I've used it.

Pxtl 7y ago
Yeah, most of that comes from the verbose process needed to invoke a process, right? That's something I noticed when going back and forth between PowerShell and C# - that if C# had clean support for invoking a process and collecting the results as an IEnumerable like PowerShell does, PS wouldn't really need to exist, since 90% of the time you're dropping into C#/.net objects to get anything done anyways.
- desc 7y ago
  I actually invested some time a while back in building a nicer API for C# to invoke shell commands and process the results. The only downside IMO is the Rx library dependency for STDOUT/STDERR; I personally try to avoid depending on libraries which themselves have extra dependencies. Since I did this at work it belongs to my employer, so I can't currently publish it freely, but it's not part of our core product so they may be open to publishing it under MIT licence or similar at some stage. So it can be done, and has been done, but I guess most people are sufficiently happy with Powershell, Python, etc to not bother bridging the gap for C# too.
- pfranz 7y ago
  > Yeah, most of that comes from the verbose process needed to invoke a process, right? I think it's more incongruence between the languages. os.system() will technically call a command. subprocess is a big step up from popen2, but Perl was much more streamlined and terse. That also applies to one-liners when you're in a shell. With Python you just have differences in a bunch of things; error handling? I often have to look up error codes, catch the exception then check for error codes. Pipe data in? Test for sys.stdin.isatty() then read from sys.stdin or fallback to sys.argv--it's not obscure, just not particularly Pythonic. The list goes on and on. It will probably take ~20 lines to properly deal with shelling out and if you do things wrong you could deadlock[1]. On the plus side, discouraging shelling out means your code is more portable =P [1] https://docs.python.org/3.8/library/subprocess.html#subprocess.Popen.wait https://docs.python.org/3.8/library/subprocess.html#subproce...
bproven 7y ago
I agree, but this lib does make it easier: https://github.com/amoffat/sh https://github.com/amoffat/sh

by pfranz 7y ago

Pxtl 7y ago

Yeah, most of that comes from the verbose process needed to invoke a process, right? That's something I noticed when going back and forth between PowerShell and C# - that if C# had clean support for invoking a process and collecting the results as an IEnumerable like PowerShell does, PS wouldn't really need to exist, since 90% of the time you're dropping into C#/.net objects to get anything done anyways.

desc 7y ago
I actually invested some time a while back in building a nicer API for C# to invoke shell commands and process the results. The only downside IMO is the Rx library dependency for STDOUT/STDERR; I personally try to avoid depending on libraries which themselves have extra dependencies. Since I did this at work it belongs to my employer, so I can't currently publish it freely, but it's not part of our core product so they may be open to publishing it under MIT licence or similar at some stage. So it can be done, and has been done, but I guess most people are sufficiently happy with Powershell, Python, etc to not bother bridging the gap for C# too.
pfranz 7y ago
> Yeah, most of that comes from the verbose process needed to invoke a process, right? I think it's more incongruence between the languages. os.system() will technically call a command. subprocess is a big step up from popen2, but Perl was much more streamlined and terse. That also applies to one-liners when you're in a shell. With Python you just have differences in a bunch of things; error handling? I often have to look up error codes, catch the exception then check for error codes. Pipe data in? Test for sys.stdin.isatty() then read from sys.stdin or fallback to sys.argv--it's not obscure, just not particularly Pythonic. The list goes on and on. It will probably take ~20 lines to properly deal with shelling out and if you do things wrong you could deadlock[1]. On the plus side, discouraging shelling out means your code is more portable =P [1] https://docs.python.org/3.8/library/subprocess.html#subprocess.Popen.wait https://docs.python.org/3.8/library/subprocess.html#subproce...

bproven 7y ago

I agree, but this lib does make it easier: https://github.com/amoffat/sh https://github.com/amoffat/sh