11 ms·

> Traditional shell scripting languages are notoriously limited<snip> I feel like people looking to replace shells and shell languages need to really think dee

> Traditional shell scripting languages are notoriously limited<snip> I feel like people looking to replace shells and shell languages need to really think deep and hard about this if it's something they believe. Shell scripts are really anything but limited, and in fact most replacements are more limited (either by design or by accident), often imposing awkward control flow on you or making things that should be simple much much more complicated. Error prone? Sure. Difficult to maintain? Yep, often. But when the thing you're doing is working with external processes and hooking them together into some kind of routine, the reason shell scripts have endured is precisely because they are extremely good at that, and provide immense amounts of flexibility and power to how you go about it in a concise way.

hnlmorg 5y ago
I couldn't agree with this statement more. As someone who has developed their own shell over the last decade I've been keeping a close eye on what other people have been building too. So often I see people writing shells that are inspired by programming languages so their syntax looks amazing in documents. But they always strike me as being hugely tedious for repetitive and often quite dull tasks. The kind of 5 minute jobs that you drop into an interactive REPL to solve an immediate problem. And then maybe save as a shell script later if you find you're doing that task frequently. People look at the problems with error handling and maintainability (which, as you highlighted yourself, are very real) but they end up throwing the baby out with the bath water when trying to solve those problems. Ultimately it doesn't matter how well a shell is for scripting if it still creates more trouble as an interactive shell compared to (for example) Bash. I say that because we already have Python, Ruby, Perl, node.js, Jua, and a plethora of other languages that can be used for scripting. We don't need more scripting languages. We need better shells.
- klibertp 5y ago
  > We need better shells. Obviously. I don't think this is the most important missing part, though. I would say it differently: we need way, way better REPLs. IPython is an example of a REPL that's passable as a shell. It can run in a terminal and has a GUI version based on Qt, which allows displaying images inline. You can drop into a "real" shell with a single `!` character (you get pipes, output capture, and (Python) variable interpolation), and it even has some syntactic shortcuts for the parts where Python's own syntax is irritatingly verbose. If you like Python, then IPython can be your day-to-day shell right now. You just need to remember not to start ncurses programs from within qtconsole (works ok in terminal). I used it for a few years when I was forced to work on Windows. Before my time, I heard it was popular to use tclsh as a shell on Windows. I think that it proves that almost any language can be used as a shell, as long as its REPL is as rich and featureful enough. Since you can use Python as a shell, which as a language is not exactly the epitome of terseness and expressiveness, you could definitely make do with almost any other interpreted language, too. The problem is that very, very few languages have REPLs that are anywhere near IPython. It's so bad sometimes that you're advised to use `rlwrap` just to get basic line editing and history! I've been working on a new shell based on GNU Smalltalk[1]. I really like the syntax - or lack of thereof - and being able to dump an image (ie. full state of the VM) at any time (and load it later) seemed like a good idea. The only change I needed was to add the `|>` pseudo-operator, which puts what's on the left into parens. Being able to introspect the running session was my primary motivation: I wanted to make the shell and the whole environment as discoverable as possible. I wrote some code for that and then realized that the default REPL uses readline from C, so it freezes the entire VM when waiting for input (including all background threads). My workaround was to set up a socket server and connect to it via rlwrapped telnet... Anyway, I think "do we need a new shell" is the wrong question; instead, we should focus on improving REPLs to the point where a separate shell becomes unnecessary. [1] https://github.com/piotrklibert/stshell https://github.com/piotrklibert/stshell
  - hnlmorg 5y ago
    The REPL is the shell. I even used the term "REPL" (and a couple of synonyms too) in my comment. So I do agree it's critical but that doesn't make the language irrelevant. Your point about how Python shells have had to create syntactic sugar for REPl usage is a good illustration of my point about how it matters a lot. Also you can render images in quite a few terminal emulators already. Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. eg https://github.com/lmorg/murex/blob/master/config/defaults/profile_all.mx#L809-L871 https://github.com/lmorg/murex/blob/master/config/defaults/p... There's definitely room for improvement in the whole TTY / shell / terminal emulator integration space though. But that's not going to happen without breaking support for existing CLI tools. Which basically means it's never going to happen given it wouldn't ever gain traction.
    - nine_k 5y ago
      REPLs are highly interactive. Shells are also needed to run scripts. A Bourne shell implementation without Readline is a pain to use as a REPL, but is a completely adequate shell scripting tool.
      - hnlmorg 5y ago
        Scripting without a shell is just scripting. The "shell" is the UI component. Shell's don't even need to be CLI based either (eg explorer.exe, web shells, etc). https://en.wikipedia.org/wiki/Shell_(computing) https://en.wikipedia.org/wiki/Shell_(computing)
    - klibertp 5y ago
      > Your point about how Python shells have had to create syntactic sugar for REPL usage is a good illustration of my point about how it matters a lot. Of course it matters. The fact that Python is hardly suited for one-liners means that, if you want to write one-liners in it, you'll have to first extend or change its syntax and stdlib, pouring time and effort into creating a DSL for the interactive use-case. Some languages basically are DSLs for one-liners, like AWK, jq, TCL, Perl and Raku; some require minimal effort, like Lisp, Smalltalk, Ruby or Groovy; and some require a lot of work to make the syntax you want work, like Python, Lua, or JavaScript (or Java). So yes, the choice of language definitely matters, but in principle every language with a REPL can be made into a shell with more or less effort. The problem is that even when you do make that effort, you're stuck with a shell nice for scripting, but with line-editing capabilities of raw readline at best. I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell. > Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. That's an interesting bit of functionality, I will take a look. Thanks!
iamwpj 5y ago
Yes, Python comes with 100% overhead from Shell. I have packages inside either the system or virtual environment tools. Honestly I err toward sh/bash because I write it, ship it, and forget it!
deltaonefour 5y ago
>the reason shell scripts have endured is precisely because they are extremely good at that It's very easy to make something extremely good at this and much less error prone. That's the main problem. >often imposing awkward control flow on you or making things that should be simple much much more complicated. There's literally one primitive paradigm for control flow, both bash and other languages incorporate it. if... else. <----That''s it.
chasil 5y ago
The weakness of the shell is that it did not emerge as an LR-parsed language that could easily be expressed with a concise yacc grammar. The "one true awk," for example, bundled a yacc grammar as part of the build until relatively recently. The complex grammar of the POSIX shell is ambiguous in the extreme in many situations. The shell language was further constrained by POSIX that removed much Korn shell functionality, even though a public domain implementation of Korn existed. This was done to allow the shell parser to be very small, and Debian's Almquist shell compiles to under 100k on i386 (unlike bash, which trails it also in speed). This new Hush shell looks a bit too wordy and reminiscent of javascript. Anyone designing a new shell grammar should also design in such a way that Busybox could bundle this new interpreter, should popularity justify it.
- nine_k 5y ago
  While being extremely small is a worthy goal, I suppose the aim of Hush is to make writing larger shell scripts easier and less error-prone. It's more for the niche of Perl of old than of minimal shells like ash. For a very limited device, a very limited shell like that in Busybox is sufficient, because it likely does not need large shell scripts, or a lot of interactive work. Looking at [1], current Hush is under 700k, which is still way smaller than Python or Perl, with much of its expressiveness. [1]: https://github.com/hush-shell/hush/releases https://github.com/hush-shell/hush/releases
  - chasil 5y ago
    The whole of Busybox is a megabyte. If Hush is 700k, then there are many, many places that it cannot go. As such, it cannot be a standard default shell. Use on smaller embedded platforms will not be possible. That is the trade.
    - nine_k 5y ago
      zsh is also rather large, but it does not prevent zsh from being useful and popular. I'm not going to run it on my router, though. OTOH I'm not going to run Busybox as the main shell on my laptop. Right tool for the job.
chubot 5y ago
Yes, exactly. I have a few slogans for this, but one is that the shell language should compose with the rest of the operating system, which is "processes and files". It's a situated language. So I have been making the point that the "narrow waist" of shell is still processes and files. Whereas the narrow waist of Python and Lua is functions and compound data structures like dicts/lists (tables). One thing that is not quite obvious until you design AND USE a shell language is that if you have these conflicting narrow waists, you have problems of composition, and you will offload this complexity onto your users. From a very quick look at the hush docs, it is Lua with some shell-like stuff grafted on. I believe that will cause cause such problems in programs. They will be longer and harder to write. You will have more edge cases. ---- I hit this issue many times in Oil, and I wrote many posts tagged #software-architecture about it: http://www.oilshell.org/blog/tags.html?tag=software-architecture#software-architecture http://www.oilshell.org/blog/tags.html?tag=software-architec... Unix Shell: Philosophy, Design, and FAQs http://www.oilshell.org/blog/2021/01/philosophy-design.html http://www.oilshell.org/blog/2021/01/philosophy-design.html makes several of these points. This recent post http://www.oilshell.org/blog/2022/03/backlog-arch.html http://www.oilshell.org/blog/2022/03/backlog-arch.html asks: - Should shells have two tiers? - Both external processes and internal "functions"? - Both pipelines of bytes and pipelines of structured data? - Another one I'm working on now: Both exit codes and Python-like exceptions? In Oil the answer is yes for the first 2, but a subtle point is that the former are PRIMARY and still the narrow waist; the latter are helpers. Dicts and Lists exist to help you deal with files (and produce JSON and TSV); but they're not the primary structures of the program. I've also discussed this issue pretty extensively with the authors of the NGS and Elvish shells. A related recent comment: https://old.reddit.com/r/ProgrammingLanguages/comments/ub5vij/what_was_considered_problematic_with_unsigned/i63at3x/ https://old.reddit.com/r/ProgrammingLanguages/comments/ub5vi... Having unsigned / signed and sync / async (what color is your function?) are similar problems of composition in language design. I call them "Perlis-Thompson" problems. There is no easy answer but IMO it's vital to explicitly think about and address these problems. C has a flawed solution to the combinatorial explosion (implicit conversions), and it's caused problems and been lamented for ~50 years. There's also a good quote in that thread about why unsigned sizes in STL were a language design mistake.
jerf 5y ago
I want a shell language that wonderfully encompasses both interactive usage and the creation of easy, safe, effective shell scripts. I've come to the conclusion I can want that all I want, but it's not possible. Shell scripts and interactive usage have too many fundamentally opposing forces in them for one language to bridge the gap. You can be great at one or the other, and you can be bad at both, but you can't be great at both. There simply isn't such a thing, even in theory. Closest you could get is a language that has two very closely related dialects, but the "dialects" aren't going to be "the same thing, but with a couple of little changes"; there's going to be a lot of differences, enough that people are going to be arguing seriously and with some merit that it's just two languages tied together at the hip.
- klibertp 5y ago
  > but you can't be great at both My latest discovery - Raku - does try, and with not bad results. Also, Scala and Ammonite work quite well. Clojure and Babashka, too. > There simply isn't such a thing, even in theory. That's a very strong claim. What makes you think it's impossible to design such a language? The fact that none exists currently doesn't mean that it can't exist. > arguing seriously and with some merit that it's just two languages tied together at the hip. Just use Racket :-) It's a bundle of tens of languages anyway, so nobody will complain if you add one more. https://rash-lang.org https://rash-lang.org does this. Due to how Racket works, you can write scripts for that shell in anything from Scheme to Typed Racket to Datalog to Brainf*ck (and more, obviously).
  - jerf 5y ago
    "What makes you think it's impossible to design such a language?" 1. An interactive shell bases its error handling on the human being right there to handle the error. A programming languages bases its error handling on a human not being right there to handle it. This is a fundamental difference, and a language must privilege one or the other. Shell languages currently privilege the first, which is a non-trivial element of why they're dangerous to program in. 2. An interactive shell prioritizes keystrokes, because a human is right there in the moment tapping the keyboard trying to do something. A programming language won't necessarily ignore this, but it's definitely down the list of priorities. 3. An interactive shell bases its expectations of what state the user has in their head as being based on the live session state and the user's experience. A programming language bases its expectations of what state the user has based on the parameters to the functions and other language environment stuff. "cd" is really quite dangerous in shell scripts, especially once you start feeding it variables, but generally safe in interactive use because you see where you are (unless your PS1 is too bare, but you should fix that). Yes, obviously, we've all still messed that up at some point but I don't think it's necessarily above the baseline for any other human messup. 4. An interactive shell expects you to base your grounding of what's going to happen when you hit enter on the contents of the line when you hit enter. Again, yes, there are human messups, but that's the model. You don't wrap a whole bunch of defensive programming around every command line you type, you just use your eyes and your own understanding. A programming language does expect defensive programming. Defaults that make sense in one case are incredibly annoying or dangerous in the other. In my shell scripts, I want an empty (or perhaps "undefined") variable to be an error. In my interactive use, I may like a button to expand them all in place so I can see what I'm actually running but don't want it to necessarily be an error. And that's just one example of the sort of differences I'd expect. Another is string handling; there is a reason that almost all programming languages require all strings to be delimited via quotes (and the ones that don't generally consider it a mistake), and almost all shells do not. I wouldn't be surprised this criterion alone could be used as a razor to separate the categories with very high accuracy. I suspect that were I to take the time to examine the 6 things you cite as things that are putatively both good scripting languages and good shell languages that I could tell you which category I put them in quite easily and why I wouldn't consider using them for the other thing, but that's rather more than I can take on for an HN post.
pmontra 5y ago
Agreed. The problem with bash & Co. (if that's a problem) is that it's the result of a process of adding clunk after clunk of stuff in the only few ways that wouldn't break the language. So we have things like i=$(( i + 1 )) and [ $? -eq 0 ] and [ "$?" = "0" ]. Add to it the differences between "$@" "$" and $ and a number of similar things. Apart from that it's very good at what it does. Hush seems to start from a normal language like Python or Ruby and add the pipelining / redirection stuff which is usually very verbose and error prone to do in those kind of languages.

hnlmorg 5y ago

I couldn't agree with this statement more. As someone who has developed their own shell over the last decade I've been keeping a close eye on what other people have been building too. So often I see people writing shells that are inspired by programming languages so their syntax looks amazing in documents. But they always strike me as being hugely tedious for repetitive and often quite dull tasks. The kind of 5 minute jobs that you drop into an interactive REPL to solve an immediate problem. And then maybe save as a shell script later if you find you're doing that task frequently. People look at the problems with error handling and maintainability (which, as you highlighted yourself, are very real) but they end up throwing the baby out with the bath water when trying to solve those problems. Ultimately it doesn't matter how well a shell is for scripting if it still creates more trouble as an interactive shell compared to (for example) Bash. I say that because we already have Python, Ruby, Perl, node.js, Jua, and a plethora of other languages that can be used for scripting. We don't need more scripting languages. We need better shells.

klibertp 5y ago
> We need better shells. Obviously. I don't think this is the most important missing part, though. I would say it differently: we need way, way better REPLs. IPython is an example of a REPL that's passable as a shell. It can run in a terminal and has a GUI version based on Qt, which allows displaying images inline. You can drop into a "real" shell with a single `!` character (you get pipes, output capture, and (Python) variable interpolation), and it even has some syntactic shortcuts for the parts where Python's own syntax is irritatingly verbose. If you like Python, then IPython can be your day-to-day shell right now. You just need to remember not to start ncurses programs from within qtconsole (works ok in terminal). I used it for a few years when I was forced to work on Windows. Before my time, I heard it was popular to use tclsh as a shell on Windows. I think that it proves that almost any language can be used as a shell, as long as its REPL is as rich and featureful enough. Since you can use Python as a shell, which as a language is not exactly the epitome of terseness and expressiveness, you could definitely make do with almost any other interpreted language, too. The problem is that very, very few languages have REPLs that are anywhere near IPython. It's so bad sometimes that you're advised to use `rlwrap` just to get basic line editing and history! I've been working on a new shell based on GNU Smalltalk[1]. I really like the syntax - or lack of thereof - and being able to dump an image (ie. full state of the VM) at any time (and load it later) seemed like a good idea. The only change I needed was to add the `|>` pseudo-operator, which puts what's on the left into parens. Being able to introspect the running session was my primary motivation: I wanted to make the shell and the whole environment as discoverable as possible. I wrote some code for that and then realized that the default REPL uses readline from C, so it freezes the entire VM when waiting for input (including all background threads). My workaround was to set up a socket server and connect to it via rlwrapped telnet... Anyway, I think "do we need a new shell" is the wrong question; instead, we should focus on improving REPLs to the point where a separate shell becomes unnecessary. [1] https://github.com/piotrklibert/stshell https://github.com/piotrklibert/stshell
- hnlmorg 5y ago
  The REPL is the shell. I even used the term "REPL" (and a couple of synonyms too) in my comment. So I do agree it's critical but that doesn't make the language irrelevant. Your point about how Python shells have had to create syntactic sugar for REPl usage is a good illustration of my point about how it matters a lot. Also you can render images in quite a few terminal emulators already. Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. eg https://github.com/lmorg/murex/blob/master/config/defaults/profile_all.mx#L809-L871 https://github.com/lmorg/murex/blob/master/config/defaults/p... There's definitely room for improvement in the whole TTY / shell / terminal emulator integration space though. But that's not going to happen without breaking support for existing CLI tools. Which basically means it's never going to happen given it wouldn't ever gain traction.
  - nine_k 5y ago
    REPLs are highly interactive. Shells are also needed to run scripts. A Bourne shell implementation without Readline is a pain to use as a REPL, but is a completely adequate shell scripting tool.
    - hnlmorg 5y ago
      Scripting without a shell is just scripting. The "shell" is the UI component. Shell's don't even need to be CLI based either (eg explorer.exe, web shells, etc). https://en.wikipedia.org/wiki/Shell_(computing) https://en.wikipedia.org/wiki/Shell_(computing)
  - klibertp 5y ago
    > Your point about how Python shells have had to create syntactic sugar for REPL usage is a good illustration of my point about how it matters a lot. Of course it matters. The fact that Python is hardly suited for one-liners means that, if you want to write one-liners in it, you'll have to first extend or change its syntax and stdlib, pouring time and effort into creating a DSL for the interactive use-case. Some languages basically are DSLs for one-liners, like AWK, jq, TCL, Perl and Raku; some require minimal effort, like Lisp, Smalltalk, Ruby or Groovy; and some require a lot of work to make the syntax you want work, like Python, Lua, or JavaScript (or Java). So yes, the choice of language definitely matters, but in principle every language with a REPL can be made into a shell with more or less effort. The problem is that even when you do make that effort, you're stuck with a shell nice for scripting, but with line-editing capabilities of raw readline at best. I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell. > Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. That's an interesting bit of functionality, I will take a look. Thanks!

iamwpj 5y ago

Yes, Python comes with 100% overhead from Shell. I have packages inside either the system or virtual environment tools. Honestly I err toward sh/bash because I write it, ship it, and forget it!

deltaonefour 5y ago

>the reason shell scripts have endured is precisely because they are extremely good at that It's very easy to make something extremely good at this and much less error prone. That's the main problem. >often imposing awkward control flow on you or making things that should be simple much much more complicated. There's literally one primitive paradigm for control flow, both bash and other languages incorporate it. if... else. <----That''s it.

chasil 5y ago

The weakness of the shell is that it did not emerge as an LR-parsed language that could easily be expressed with a concise yacc grammar. The "one true awk," for example, bundled a yacc grammar as part of the build until relatively recently. The complex grammar of the POSIX shell is ambiguous in the extreme in many situations. The shell language was further constrained by POSIX that removed much Korn shell functionality, even though a public domain implementation of Korn existed. This was done to allow the shell parser to be very small, and Debian's Almquist shell compiles to under 100k on i386 (unlike bash, which trails it also in speed). This new Hush shell looks a bit too wordy and reminiscent of javascript. Anyone designing a new shell grammar should also design in such a way that Busybox could bundle this new interpreter, should popularity justify it.

nine_k 5y ago
While being extremely small is a worthy goal, I suppose the aim of Hush is to make writing larger shell scripts easier and less error-prone. It's more for the niche of Perl of old than of minimal shells like ash. For a very limited device, a very limited shell like that in Busybox is sufficient, because it likely does not need large shell scripts, or a lot of interactive work. Looking at [1], current Hush is under 700k, which is still way smaller than Python or Perl, with much of its expressiveness. [1]: https://github.com/hush-shell/hush/releases https://github.com/hush-shell/hush/releases
- chasil 5y ago
  The whole of Busybox is a megabyte. If Hush is 700k, then there are many, many places that it cannot go. As such, it cannot be a standard default shell. Use on smaller embedded platforms will not be possible. That is the trade.
  - nine_k 5y ago
    zsh is also rather large, but it does not prevent zsh from being useful and popular. I'm not going to run it on my router, though. OTOH I'm not going to run Busybox as the main shell on my laptop. Right tool for the job.

chubot 5y ago

Yes, exactly. I have a few slogans for this, but one is that the shell language should compose with the rest of the operating system, which is "processes and files". It's a situated language. So I have been making the point that the "narrow waist" of shell is still processes and files. Whereas the narrow waist of Python and Lua is functions and compound data structures like dicts/lists (tables). One thing that is not quite obvious until you design AND USE a shell language is that if you have these conflicting narrow waists, you have problems of composition, and you will offload this complexity onto your users. From a very quick look at the hush docs, it is Lua with some shell-like stuff grafted on. I believe that will cause cause such problems in programs. They will be longer and harder to write. You will have more edge cases. ---- I hit this issue many times in Oil, and I wrote many posts tagged #software-architecture about it: http://www.oilshell.org/blog/tags.html?tag=software-architecture#software-architecture http://www.oilshell.org/blog/tags.html?tag=software-architec... Unix Shell: Philosophy, Design, and FAQs http://www.oilshell.org/blog/2021/01/philosophy-design.html http://www.oilshell.org/blog/2021/01/philosophy-design.html makes several of these points. This recent post http://www.oilshell.org/blog/2022/03/backlog-arch.html http://www.oilshell.org/blog/2022/03/backlog-arch.html asks: - Should shells have two tiers? - Both external processes and internal "functions"? - Both pipelines of bytes and pipelines of structured data? - Another one I'm working on now: Both exit codes and Python-like exceptions? In Oil the answer is yes for the first 2, but a subtle point is that the former are PRIMARY and still the narrow waist; the latter are helpers. Dicts and Lists exist to help you deal with files (and produce JSON and TSV); but they're not the primary structures of the program. I've also discussed this issue pretty extensively with the authors of the NGS and Elvish shells. A related recent comment: https://old.reddit.com/r/ProgrammingLanguages/comments/ub5vij/what_was_considered_problematic_with_unsigned/i63at3x/ https://old.reddit.com/r/ProgrammingLanguages/comments/ub5vi... Having unsigned / signed and sync / async (what color is your function?) are similar problems of composition in language design. I call them "Perlis-Thompson" problems. There is no easy answer but IMO it's vital to explicitly think about and address these problems. C has a flawed solution to the combinatorial explosion (implicit conversions), and it's caused problems and been lamented for ~50 years. There's also a good quote in that thread about why unsigned sizes in STL were a language design mistake.

jerf 5y ago

I want a shell language that wonderfully encompasses both interactive usage and the creation of easy, safe, effective shell scripts. I've come to the conclusion I can want that all I want, but it's not possible. Shell scripts and interactive usage have too many fundamentally opposing forces in them for one language to bridge the gap. You can be great at one or the other, and you can be bad at both, but you can't be great at both. There simply isn't such a thing, even in theory. Closest you could get is a language that has two very closely related dialects, but the "dialects" aren't going to be "the same thing, but with a couple of little changes"; there's going to be a lot of differences, enough that people are going to be arguing seriously and with some merit that it's just two languages tied together at the hip.

klibertp 5y ago
> but you can't be great at both My latest discovery - Raku - does try, and with not bad results. Also, Scala and Ammonite work quite well. Clojure and Babashka, too. > There simply isn't such a thing, even in theory. That's a very strong claim. What makes you think it's impossible to design such a language? The fact that none exists currently doesn't mean that it can't exist. > arguing seriously and with some merit that it's just two languages tied together at the hip. Just use Racket :-) It's a bundle of tens of languages anyway, so nobody will complain if you add one more. https://rash-lang.org https://rash-lang.org does this. Due to how Racket works, you can write scripts for that shell in anything from Scheme to Typed Racket to Datalog to Brainf*ck (and more, obviously).
- jerf 5y ago
  "What makes you think it's impossible to design such a language?" 1. An interactive shell bases its error handling on the human being right there to handle the error. A programming languages bases its error handling on a human not being right there to handle it. This is a fundamental difference, and a language must privilege one or the other. Shell languages currently privilege the first, which is a non-trivial element of why they're dangerous to program in. 2. An interactive shell prioritizes keystrokes, because a human is right there in the moment tapping the keyboard trying to do something. A programming language won't necessarily ignore this, but it's definitely down the list of priorities. 3. An interactive shell bases its expectations of what state the user has in their head as being based on the live session state and the user's experience. A programming language bases its expectations of what state the user has based on the parameters to the functions and other language environment stuff. "cd" is really quite dangerous in shell scripts, especially once you start feeding it variables, but generally safe in interactive use because you see where you are (unless your PS1 is too bare, but you should fix that). Yes, obviously, we've all still messed that up at some point but I don't think it's necessarily above the baseline for any other human messup. 4. An interactive shell expects you to base your grounding of what's going to happen when you hit enter on the contents of the line when you hit enter. Again, yes, there are human messups, but that's the model. You don't wrap a whole bunch of defensive programming around every command line you type, you just use your eyes and your own understanding. A programming language does expect defensive programming. Defaults that make sense in one case are incredibly annoying or dangerous in the other. In my shell scripts, I want an empty (or perhaps "undefined") variable to be an error. In my interactive use, I may like a button to expand them all in place so I can see what I'm actually running but don't want it to necessarily be an error. And that's just one example of the sort of differences I'd expect. Another is string handling; there is a reason that almost all programming languages require all strings to be delimited via quotes (and the ones that don't generally consider it a mistake), and almost all shells do not. I wouldn't be surprised this criterion alone could be used as a razor to separate the categories with very high accuracy. I suspect that were I to take the time to examine the 6 things you cite as things that are putatively both good scripting languages and good shell languages that I could tell you which category I put them in quite easily and why I wouldn't consider using them for the other thing, but that's rather more than I can take on for an HN post.

pmontra 5y ago

Agreed. The problem with bash & Co. (if that's a problem) is that it's the result of a process of adding clunk after clunk of stuff in the only few ways that wouldn't break the language. So we have things like i=$(( i + 1 )) and [ $? -eq 0 ] and [ "$?" = "0" ]. Add to it the differences between "$@" "$" and $ and a number of similar things. Apart from that it's very good at what it does. Hush seems to start from a normal language like Python or Ruby and add the pipelining / redirection stuff which is usually very verbose and error prone to do in those kind of languages.