5 ms·

I couldn't agree with this statement more. As someone who has developed their own shell over the last decade I've been keeping a close eye on what other people

I couldn't agree with this statement more. As someone who has developed their own shell over the last decade I've been keeping a close eye on what other people have been building too. So often I see people writing shells that are inspired by programming languages so their syntax looks amazing in documents. But they always strike me as being hugely tedious for repetitive and often quite dull tasks. The kind of 5 minute jobs that you drop into an interactive REPL to solve an immediate problem. And then maybe save as a shell script later if you find you're doing that task frequently. People look at the problems with error handling and maintainability (which, as you highlighted yourself, are very real) but they end up throwing the baby out with the bath water when trying to solve those problems. Ultimately it doesn't matter how well a shell is for scripting if it still creates more trouble as an interactive shell compared to (for example) Bash. I say that because we already have Python, Ruby, Perl, node.js, Jua, and a plethora of other languages that can be used for scripting. We don't need more scripting languages. We need better shells.

klibertp 5y ago
> We need better shells. Obviously. I don't think this is the most important missing part, though. I would say it differently: we need way, way better REPLs. IPython is an example of a REPL that's passable as a shell. It can run in a terminal and has a GUI version based on Qt, which allows displaying images inline. You can drop into a "real" shell with a single `!` character (you get pipes, output capture, and (Python) variable interpolation), and it even has some syntactic shortcuts for the parts where Python's own syntax is irritatingly verbose. If you like Python, then IPython can be your day-to-day shell right now. You just need to remember not to start ncurses programs from within qtconsole (works ok in terminal). I used it for a few years when I was forced to work on Windows. Before my time, I heard it was popular to use tclsh as a shell on Windows. I think that it proves that almost any language can be used as a shell, as long as its REPL is as rich and featureful enough. Since you can use Python as a shell, which as a language is not exactly the epitome of terseness and expressiveness, you could definitely make do with almost any other interpreted language, too. The problem is that very, very few languages have REPLs that are anywhere near IPython. It's so bad sometimes that you're advised to use `rlwrap` just to get basic line editing and history! I've been working on a new shell based on GNU Smalltalk[1]. I really like the syntax - or lack of thereof - and being able to dump an image (ie. full state of the VM) at any time (and load it later) seemed like a good idea. The only change I needed was to add the `|>` pseudo-operator, which puts what's on the left into parens. Being able to introspect the running session was my primary motivation: I wanted to make the shell and the whole environment as discoverable as possible. I wrote some code for that and then realized that the default REPL uses readline from C, so it freezes the entire VM when waiting for input (including all background threads). My workaround was to set up a socket server and connect to it via rlwrapped telnet... Anyway, I think "do we need a new shell" is the wrong question; instead, we should focus on improving REPLs to the point where a separate shell becomes unnecessary. [1] https://github.com/piotrklibert/stshell https://github.com/piotrklibert/stshell
- hnlmorg 5y ago
  The REPL is the shell. I even used the term "REPL" (and a couple of synonyms too) in my comment. So I do agree it's critical but that doesn't make the language irrelevant. Your point about how Python shells have had to create syntactic sugar for REPl usage is a good illustration of my point about how it matters a lot. Also you can render images in quite a few terminal emulators already. Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. eg https://github.com/lmorg/murex/blob/master/config/defaults/profile_all.mx#L809-L871 https://github.com/lmorg/murex/blob/master/config/defaults/p... There's definitely room for improvement in the whole TTY / shell / terminal emulator integration space though. But that's not going to happen without breaking support for existing CLI tools. Which basically means it's never going to happen given it wouldn't ever gain traction.
  - nine_k 5y ago
    REPLs are highly interactive. Shells are also needed to run scripts. A Bourne shell implementation without Readline is a pain to use as a REPL, but is a completely adequate shell scripting tool.
    - hnlmorg 5y ago
      Scripting without a shell is just scripting. The "shell" is the UI component. Shell's don't even need to be CLI based either (eg explorer.exe, web shells, etc). https://en.wikipedia.org/wiki/Shell_(computing) https://en.wikipedia.org/wiki/Shell_(computing)
  - klibertp 5y ago
    > Your point about how Python shells have had to create syntactic sugar for REPL usage is a good illustration of my point about how it matters a lot. Of course it matters. The fact that Python is hardly suited for one-liners means that, if you want to write one-liners in it, you'll have to first extend or change its syntax and stdlib, pouring time and effort into creating a DSL for the interactive use-case. Some languages basically are DSLs for one-liners, like AWK, jq, TCL, Perl and Raku; some require minimal effort, like Lisp, Smalltalk, Ruby or Groovy; and some require a lot of work to make the syntax you want work, like Python, Lua, or JavaScript (or Java). So yes, the choice of language definitely matters, but in principle every language with a REPL can be made into a shell with more or less effort. The problem is that even when you do make that effort, you're stuck with a shell nice for scripting, but with line-editing capabilities of raw readline at best. I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell. > Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. That's an interesting bit of functionality, I will take a look. Thanks!

klibertp 5y ago

> We need better shells. Obviously. I don't think this is the most important missing part, though. I would say it differently: we need way, way better REPLs. IPython is an example of a REPL that's passable as a shell. It can run in a terminal and has a GUI version based on Qt, which allows displaying images inline. You can drop into a "real" shell with a single `!` character (you get pipes, output capture, and (Python) variable interpolation), and it even has some syntactic shortcuts for the parts where Python's own syntax is irritatingly verbose. If you like Python, then IPython can be your day-to-day shell right now. You just need to remember not to start ncurses programs from within qtconsole (works ok in terminal). I used it for a few years when I was forced to work on Windows. Before my time, I heard it was popular to use tclsh as a shell on Windows. I think that it proves that almost any language can be used as a shell, as long as its REPL is as rich and featureful enough. Since you can use Python as a shell, which as a language is not exactly the epitome of terseness and expressiveness, you could definitely make do with almost any other interpreted language, too. The problem is that very, very few languages have REPLs that are anywhere near IPython. It's so bad sometimes that you're advised to use `rlwrap` just to get basic line editing and history! I've been working on a new shell based on GNU Smalltalk[1]. I really like the syntax - or lack of thereof - and being able to dump an image (ie. full state of the VM) at any time (and load it later) seemed like a good idea. The only change I needed was to add the `|>` pseudo-operator, which puts what's on the left into parens. Being able to introspect the running session was my primary motivation: I wanted to make the shell and the whole environment as discoverable as possible. I wrote some code for that and then realized that the default REPL uses readline from C, so it freezes the entire VM when waiting for input (including all background threads). My workaround was to set up a socket server and connect to it via rlwrapped telnet... Anyway, I think "do we need a new shell" is the wrong question; instead, we should focus on improving REPLs to the point where a separate shell becomes unnecessary. [1] https://github.com/piotrklibert/stshell https://github.com/piotrklibert/stshell

hnlmorg 5y ago
The REPL is the shell. I even used the term "REPL" (and a couple of synonyms too) in my comment. So I do agree it's critical but that doesn't make the language irrelevant. Your point about how Python shells have had to create syntactic sugar for REPl usage is a good illustration of my point about how it matters a lot. Also you can render images in quite a few terminal emulators already. Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. eg https://github.com/lmorg/murex/blob/master/config/defaults/profile_all.mx#L809-L871 https://github.com/lmorg/murex/blob/master/config/defaults/p... There's definitely room for improvement in the whole TTY / shell / terminal emulator integration space though. But that's not going to happen without breaking support for existing CLI tools. Which basically means it's never going to happen given it wouldn't ever gain traction.
- nine_k 5y ago
  REPLs are highly interactive. Shells are also needed to run scripts. A Bourne shell implementation without Readline is a pain to use as a REPL, but is a completely adequate shell scripting tool.
  - hnlmorg 5y ago
    Scripting without a shell is just scripting. The "shell" is the UI component. Shell's don't even need to be CLI based either (eg explorer.exe, web shells, etc). https://en.wikipedia.org/wiki/Shell_(computing) https://en.wikipedia.org/wiki/Shell_(computing)
- klibertp 5y ago
  > Your point about how Python shells have had to create syntactic sugar for REPL usage is a good illustration of my point about how it matters a lot. Of course it matters. The fact that Python is hardly suited for one-liners means that, if you want to write one-liners in it, you'll have to first extend or change its syntax and stdlib, pouring time and effort into creating a DSL for the interactive use-case. Some languages basically are DSLs for one-liners, like AWK, jq, TCL, Perl and Raku; some require minimal effort, like Lisp, Smalltalk, Ruby or Groovy; and some require a lot of work to make the syntax you want work, like Python, Lua, or JavaScript (or Java). So yes, the choice of language definitely matters, but in principle every language with a REPL can be made into a shell with more or less effort. The problem is that even when you do make that effort, you're stuck with a shell nice for scripting, but with line-editing capabilities of raw readline at best. I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell. > Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. That's an interesting bit of functionality, I will take a look. Thanks!