5 ms·

What's funny is that some modern scripting languages like Python, with things like the "os" module, are almost passable as a general purpose shell scripting env

What's funny is that some modern scripting languages like Python, with things like the "os" module, are almost passable as a general purpose shell scripting environment. I actually tend to write Python scripts instead of bash shell scripts for one-off file manipulation scripting, mostly because even after a decade-plus long career, I've never become much of a wizard at bash shell staples like sed or awk. But it's just that a general purpose programming language like Python doesn't treat executing subprocesses in enough of a first-class manner, in the way the bash shell does, to make it feel like a seamless shell-scripting experience. You still have to go through the motions of "import os", etc. just to get started, and you can't directly execute other programs, instead having to use "system" or the more cumbersome subprocess module. But still, it's.... tantalizingly close to something like a "better" shell that includes support for lists and other data structures instead of just "everything is a string".

lgas 10y ago
You might find ipython more compelling as an interactive shell: https://ipython.org/ipython-doc/3/interactive/shell.html https://ipython.org/ipython-doc/3/interactive/shell.html
- jhasse 10y ago
  ptpython is also nice: https://github.com/jonathanslenders/ptpython https://github.com/jonathanslenders/ptpython
thomastjeffery 10y ago
For python, Xonsh makes a good attempt at bridging this gap. Ruby also does this well, since everything in backticks runs in a shell (just like in bash).
theonemind 10y ago
The near-miss of Python is why I write perl scripts. It supports the 'backtick' notation, and pretty much kills the problem domain of awk, sed, and bash.
- tannhaeuser 10y ago
  I can see where you come from, but try to run some obscure nontrivial Perl package of the 90s and see how far it goes. I had the displeasure to exhume legacy Perl programs three times in the last two decades. It always ended with a total rewrite in another language. In contrast, awk is such a nice and minimalist language with multiple implementations available.
- hpcjoe 10y ago
  I generally write simple things in bash shells, and save more complex stuff for Perl these days. Trying to move more actively to Julia and others for this though. Perl6 looks like it extends on good things in Perl5, but, of course, in the process, breaks many things I am used to in Perl5. The nice aspect is that there are many good tools, and quite a few mediocre ones to solve problems. Pick your tool sets and be productive with them, taking care to learn others over time.
msravi 10y ago
In Julia, a shell command is a semicolon away in the REPL: ;ls -al Plus, it borrows the backtick notation from Perl: run(`ls -al`) And you can read the string returned for processing further within Julia: x = readstring(`ls -al`) And it supports pipelines... http://docs.julialang.org/en/latest/manual/running-external-programs.html http://docs.julialang.org/en/latest/manual/running-external-...
- milesrout 10y ago
  That's basically giving up, though. It doesn't make running subprocesses better, it makes running shell subprocesses better.
empath75 10y ago
I almost never wrote bash scripts more than a handful of lines long. Anything more complex and I use Python.
Too 10y ago
Well, since you are in python already, you shouldn't have to subprocess very often. Then you would rather do os.listdir() instead of subprocess.check_output(["ls"]). For the other cases where you do need subprocess, the more sane escaping rules (compared to shell) almost makes the verbosity worth it.
btschaegg 10y ago
> But it's just that a general purpose programming language like Python doesn't treat executing subprocesses in enough of a first-class manner, That's my general impression on scripting, too. Shells are great at doing day-to-day things I want to do interactively on a system, but start to develop problems (awkward syntax, lacking readability etc) when it comes to more complex programming. On the other hand, languages like Python do that really well but have problems with the first part. I sometimes wonder if there's really no way to combine the two in a practical way. One datapoint (on the negative side) would be Powershell, which (as a more recent shell) gives its user very useful tools but still fails (IMO) on the syntax front. An idea I pondered on for a while was that maybe a shell-specific notation could help. Take xonsh, for example. The concept of it is something I could get used to, but I can't really get around the fact that there's some magic involved in differentiating between shell and python commands[1]. Perhaps a more modal approach (think vim) could help? Say I can switch explicitly between python- and shell-mode (avoiding all the magic) and can write shell-constructs in a script in a dedicated notation (like JS has its own notation for regexes, for example). Then, the main problem would most likely be the interaction between both modes (i.e. how one can pipe the stdout of a subprocess to a variable)... [1]: http://xon.sh/tutorial.html#python-mode-vs-subprocess-mode http://xon.sh/tutorial.html#python-mode-vs-subprocess-mode

lgas 10y ago

You might find ipython more compelling as an interactive shell: https://ipython.org/ipython-doc/3/interactive/shell.html https://ipython.org/ipython-doc/3/interactive/shell.html

jhasse 10y ago
ptpython is also nice: https://github.com/jonathanslenders/ptpython https://github.com/jonathanslenders/ptpython

thomastjeffery 10y ago

For python, Xonsh makes a good attempt at bridging this gap. Ruby also does this well, since everything in backticks runs in a shell (just like in bash).

theonemind 10y ago

The near-miss of Python is why I write perl scripts. It supports the 'backtick' notation, and pretty much kills the problem domain of awk, sed, and bash.

tannhaeuser 10y ago
I can see where you come from, but try to run some obscure nontrivial Perl package of the 90s and see how far it goes. I had the displeasure to exhume legacy Perl programs three times in the last two decades. It always ended with a total rewrite in another language. In contrast, awk is such a nice and minimalist language with multiple implementations available.
hpcjoe 10y ago
I generally write simple things in bash shells, and save more complex stuff for Perl these days. Trying to move more actively to Julia and others for this though. Perl6 looks like it extends on good things in Perl5, but, of course, in the process, breaks many things I am used to in Perl5. The nice aspect is that there are many good tools, and quite a few mediocre ones to solve problems. Pick your tool sets and be productive with them, taking care to learn others over time.

msravi 10y ago

In Julia, a shell command is a semicolon away in the REPL: ;ls -al Plus, it borrows the backtick notation from Perl: run(`ls -al`) And you can read the string returned for processing further within Julia: x = readstring(`ls -al`) And it supports pipelines... http://docs.julialang.org/en/latest/manual/running-external-programs.html http://docs.julialang.org/en/latest/manual/running-external-...

milesrout 10y ago
That's basically giving up, though. It doesn't make running subprocesses better, it makes running shell subprocesses better.

empath75 10y ago

I almost never wrote bash scripts more than a handful of lines long. Anything more complex and I use Python.

Too 10y ago

Well, since you are in python already, you shouldn't have to subprocess very often. Then you would rather do os.listdir() instead of subprocess.check_output(["ls"]). For the other cases where you do need subprocess, the more sane escaping rules (compared to shell) almost makes the verbosity worth it.

btschaegg 10y ago

> But it's just that a general purpose programming language like Python doesn't treat executing subprocesses in enough of a first-class manner, That's my general impression on scripting, too. Shells are great at doing day-to-day things I want to do interactively on a system, but start to develop problems (awkward syntax, lacking readability etc) when it comes to more complex programming. On the other hand, languages like Python do that really well but have problems with the first part. I sometimes wonder if there's really no way to combine the two in a practical way. One datapoint (on the negative side) would be Powershell, which (as a more recent shell) gives its user very useful tools but still fails (IMO) on the syntax front. An idea I pondered on for a while was that maybe a shell-specific notation could help. Take xonsh, for example. The concept of it is something I could get used to, but I can't really get around the fact that there's some magic involved in differentiating between shell and python commands[1]. Perhaps a more modal approach (think vim) could help? Say I can switch explicitly between python- and shell-mode (avoiding all the magic) and can write shell-constructs in a script in a dedicated notation (like JS has its own notation for regexes, for example). Then, the main problem would most likely be the interaction between both modes (i.e. how one can pipe the stdout of a subprocess to a variable)... [1]: http://xon.sh/tutorial.html#python-mode-vs-subprocess-mode http://xon.sh/tutorial.html#python-mode-vs-subprocess-mode