5 ms·

I've occasionally wondered why Python isn't the default scripting language on *nix by now. It's almost always installed by default, is more capable than bash,

I've occasionally wondered why Python isn't the default scripting language on *nix by now. It's almost always installed by default, is more capable than bash, is dynamic/interpreted, has a repl, and seemingly everything else you would want in a Bash replacement. Is it just Bash's momentum, or something else that keeps bash hanging around?

mdaniel 5y ago
Because shell is really good at gluing together commands and has reasonably straightforward error management mechanisms Contrast that with: subprocess.run("some_thing with-output") subprocess.run("the_next_thing") which requires that the caller take steps to grab, check, and then re-emit any errors (or even just the "normal" output) produced by either of those, not to mention the horrors of output buffering for long-running processes and the ever-present encoding woes since subprocess interfaces as bytes not str I guess it is the same situation with all programming -- diligent programmers can make it work, less disciplined users inflict pain upon all downstream users of poorly coded scripts
enriquto 5y ago
Common shell tasks (piping, process substitution, create a string with the output of a program, creating and redirecting output and input to files) are extremely cumbersome in python.
nixpulvis 5y ago
Please show me how to do `ls | grep -i old` in Python? For a short while I was following https://github.com/matz/streem https://github.com/matz/streem (from the original author of Ruby) with interest in possibly being adapted to superseed Bash. I have no idea how realistic that actually is. I also use fish a lot, but it has a lot of it's own problems. The thing about all these "shells" that really seperates them from Python (et all) is that they lookup commands from PATH and execute them directly. Anything else is simply not a shell language no matter how great it is at scripting.
- tkot 5y ago
  How about Groovy? ("ls".execute() | "grep -i old".execute()).waitForProcessOutput(System.out, System.out) It can also be written as "ls".execute().or("grep -i old".execute()).waitForProcessOutput(System.out, System.out) It's not exactly the same because it uses a thread and a buffer instead of an OS pipe but I guess it's close enough.
  - josephcsible 5y ago
    I wonder why Groovy decided to map "or" to the shell's "|". Wouldn't it have made more sense to map "or" to the shell's "||" and "pipe" to the shell's "|"?
    - tkot 5y ago
      I think this is just how operator overloading works in Groovy: https://groovy-lang.org/operators.html#Operator-Overloading https://groovy-lang.org/operators.html#Operator-Overloading "Or" is the name of the "|" operator. From the looks of it it's impossible to overload "||" in groovy. By the way the "|" operator and the "or" method both call "pipeTo" directly so a third way to write it would be: "ls".execute().pipeTo("grep -i old".execute()).waitForProcessOutput(System.out, System.out)
  - nixpulvis 5y ago
    Why should I need to explicitly state that I want to wait for my process output? I'm in a shell, I put a `&` afterwards if I want things in the background. The model is synchronous by default.
- jcranmer 5y ago
  > Please show me how to do `ls | grep -i old` in Python? for filename in os.listdir(os.curdir): if 'old' in filename.lower(): print(filename) One of the advantages of using a language like Python is you don't need to set up a lengthy pipeline of program executions just to reformat program output. In my personal experience using Python for basic automation scripting, I just haven't really come across any need for me to pipe output from one command into another. The problem I have with a lot of shells is that their variable syntax is almost completely broken if you want to do anything slightly harder than trivial. A recent example I had was trying to write a script that reads a shell and executes the last line with some extra arguments. So that should be `PROG=$(tail -n1 commands.sh); creduce-script "$PROG"`, nothing hard about that... except the last line was `clang "-DFOO= " bar.c`.
Yen 5y ago
I've also wanted a better shell scripting experience, and have bashed [no pun intended] my head against this several times. I think some of the major pain points that resist adoption of a language like Python or Ruby for in-line shell scripting or simple automation is: * These languages prefer to operate on structured data. If it parses from json, you have a much easier time. But, most commands you'd invoke from the shell emit unstructured text by default. You can deal with this, but it's a pain point, and it means any serious scripting starts off with a "here's how you parse the output of <foo>", and several rounds of debugging. * The shell's first and foremost usage is a user-facing interactive interface to the computer. Most of what you do is invoking other programs and seeing their output, and doing this is very easy. While python & ruby have REPLs, these are mostly focused on trying out language features or testing code, not invoking other programs, nor navigating a file tree. A lot of shell scripts start as 1-liners that grow and grow. * Invoking other programs: In sh, invoking other programs is the primary activity, and it's relatively straightforward - you type the name of that program, press enter, and hope that it's on your path. In Python or Ruby, it requires importing the proper library, choosing the correct command, wrapping it and arguments, and ensuring everything escaped correctly. [Ruby _does_ have the backticks operator, which does actually make a lot of 1-off scripts easy, but this is not a panacea] * In sh, a lot of the 'utility' programs like cut, sed, awk, grep, head, tail, etc., are standing in for the capabilities of the language itself. In pure Python or Ruby, you'd do these kinds of things with language built-ins. But, that's a learning curve, and perhaps a bit more error-prone than "| head". * On top of all that, yes, momentum. If tomorrow you showed me a shell replacement for nix that was unambiguously* improved in every way, had excellent documentation, community support, and was actually pre-installed on every machine, it would still take a decade or more before it was really a default. ----- I want it to happen, so I'd never discourage anyone from taking a swing. IMO, some of the top-level considerations that are necessary for making a successful sh alternative are: * minimize the additional # of characters required to invoke a program with arguments, compared to bash. * Decide which suite of typical utilities should actually be built-ins (i.e., things like cd, ls, cp, grep, curl), and make those standard library, built-in, without additional import or namespacing. * Focus on an append-style workflow. Functional programing styles can kind of help here. Wrapping things in loops or blocks is a point of friction. * An additional highly-desired feature which just isn't in sh by default, to overcome momentum. I have no idea what this would be. More reliability and better workflow are _nice_, but sh is sticky.
deckard1 5y ago
historically, this is exactly the role Perl has played. It's also why people keep repeating the nonsense that Perl looks like line noise. Because the only Perl code people are familiar with are throwaway scripts written by (usually) some system admin person who is absolutely not a developer.
- sundarurfriend 5y ago
  And JAPH (Just Another Perl Hacker) type constructs that are meant to bend the language in weird ways and show off the person's knowledge of the language intricacies. They were a lot of fun, but likely gave a distorted idea of an average Perl program, to those unfamiliar with it.

mdaniel 5y ago

Because shell is really good at gluing together commands and has reasonably straightforward error management mechanisms Contrast that with: subprocess.run("some_thing with-output") subprocess.run("the_next_thing") which requires that the caller take steps to grab, check, and then re-emit any errors (or even just the "normal" output) produced by either of those, not to mention the horrors of output buffering for long-running processes and the ever-present encoding woes since subprocess interfaces as bytes not str I guess it is the same situation with all programming -- diligent programmers can make it work, less disciplined users inflict pain upon all downstream users of poorly coded scripts

enriquto 5y ago

Common shell tasks (piping, process substitution, create a string with the output of a program, creating and redirecting output and input to files) are extremely cumbersome in python.

nixpulvis 5y ago

Please show me how to do `ls | grep -i old` in Python? For a short while I was following https://github.com/matz/streem https://github.com/matz/streem (from the original author of Ruby) with interest in possibly being adapted to superseed Bash. I have no idea how realistic that actually is. I also use fish a lot, but it has a lot of it's own problems. The thing about all these "shells" that really seperates them from Python (et all) is that they lookup commands from PATH and execute them directly. Anything else is simply not a shell language no matter how great it is at scripting.

tkot 5y ago
How about Groovy? ("ls".execute() | "grep -i old".execute()).waitForProcessOutput(System.out, System.out) It can also be written as "ls".execute().or("grep -i old".execute()).waitForProcessOutput(System.out, System.out) It's not exactly the same because it uses a thread and a buffer instead of an OS pipe but I guess it's close enough.
- josephcsible 5y ago
  I wonder why Groovy decided to map "or" to the shell's "|". Wouldn't it have made more sense to map "or" to the shell's "||" and "pipe" to the shell's "|"?
  - tkot 5y ago
    I think this is just how operator overloading works in Groovy: https://groovy-lang.org/operators.html#Operator-Overloading https://groovy-lang.org/operators.html#Operator-Overloading "Or" is the name of the "|" operator. From the looks of it it's impossible to overload "||" in groovy. By the way the "|" operator and the "or" method both call "pipeTo" directly so a third way to write it would be: "ls".execute().pipeTo("grep -i old".execute()).waitForProcessOutput(System.out, System.out)
- nixpulvis 5y ago
  Why should I need to explicitly state that I want to wait for my process output? I'm in a shell, I put a `&` afterwards if I want things in the background. The model is synchronous by default.
jcranmer 5y ago
> Please show me how to do `ls | grep -i old` in Python? for filename in os.listdir(os.curdir): if 'old' in filename.lower(): print(filename) One of the advantages of using a language like Python is you don't need to set up a lengthy pipeline of program executions just to reformat program output. In my personal experience using Python for basic automation scripting, I just haven't really come across any need for me to pipe output from one command into another. The problem I have with a lot of shells is that their variable syntax is almost completely broken if you want to do anything slightly harder than trivial. A recent example I had was trying to write a script that reads a shell and executes the last line with some extra arguments. So that should be `PROG=$(tail -n1 commands.sh); creduce-script "$PROG"`, nothing hard about that... except the last line was `clang "-DFOO= " bar.c`.

Yen 5y ago

I've also wanted a better shell scripting experience, and have bashed [no pun intended] my head against this several times. I think some of the major pain points that resist adoption of a language like Python or Ruby for in-line shell scripting or simple automation is: * These languages prefer to operate on structured data. If it parses from json, you have a much easier time. But, most commands you'd invoke from the shell emit unstructured text by default. You can deal with this, but it's a pain point, and it means any serious scripting starts off with a "here's how you parse the output of <foo>", and several rounds of debugging. * The shell's first and foremost usage is a user-facing interactive interface to the computer. Most of what you do is invoking other programs and seeing their output, and doing this is very easy. While python & ruby have REPLs, these are mostly focused on trying out language features or testing code, not invoking other programs, nor navigating a file tree. A lot of shell scripts start as 1-liners that grow and grow. * Invoking other programs: In sh, invoking other programs is the primary activity, and it's relatively straightforward - you type the name of that program, press enter, and hope that it's on your path. In Python or Ruby, it requires importing the proper library, choosing the correct command, wrapping it and arguments, and ensuring everything escaped correctly. [Ruby _does_ have the backticks operator, which does actually make a lot of 1-off scripts easy, but this is not a panacea] * In sh, a lot of the 'utility' programs like cut, sed, awk, grep, head, tail, etc., are standing in for the capabilities of the language itself. In pure Python or Ruby, you'd do these kinds of things with language built-ins. But, that's a learning curve, and perhaps a bit more error-prone than "| head". * On top of all that, yes, momentum. If tomorrow you showed me a shell replacement for nix that was unambiguously* improved in every way, had excellent documentation, community support, and was actually pre-installed on every machine, it would still take a decade or more before it was really a default. ----- I want it to happen, so I'd never discourage anyone from taking a swing. IMO, some of the top-level considerations that are necessary for making a successful sh alternative are: * minimize the additional # of characters required to invoke a program with arguments, compared to bash. * Decide which suite of typical utilities should actually be built-ins (i.e., things like cd, ls, cp, grep, curl), and make those standard library, built-in, without additional import or namespacing. * Focus on an append-style workflow. Functional programing styles can kind of help here. Wrapping things in loops or blocks is a point of friction. * An additional highly-desired feature which just isn't in sh by default, to overcome momentum. I have no idea what this would be. More reliability and better workflow are _nice_, but sh is sticky.

deckard1 5y ago

historically, this is exactly the role Perl has played. It's also why people keep repeating the nonsense that Perl looks like line noise. Because the only Perl code people are familiar with are throwaway scripts written by (usually) some system admin person who is absolutely not a developer.

sundarurfriend 5y ago
And JAPH (Just Another Perl Hacker) type constructs that are meant to bend the language in weird ways and show off the person's knowledge of the language intricacies. They were a lot of fun, but likely gave a distorted idea of an average Perl program, to those unfamiliar with it.