4 ms·

> We need better shells. Obviously. I don't think this is the most important missing part, though. I would say it differently: we need way, way better REPLs.

> We need better shells. Obviously. I don't think this is the most important missing part, though. I would say it differently: we need way, way better REPLs. IPython is an example of a REPL that's passable as a shell. It can run in a terminal and has a GUI version based on Qt, which allows displaying images inline. You can drop into a "real" shell with a single `!` character (you get pipes, output capture, and (Python) variable interpolation), and it even has some syntactic shortcuts for the parts where Python's own syntax is irritatingly verbose. If you like Python, then IPython can be your day-to-day shell right now. You just need to remember not to start ncurses programs from within qtconsole (works ok in terminal). I used it for a few years when I was forced to work on Windows. Before my time, I heard it was popular to use tclsh as a shell on Windows. I think that it proves that almost any language can be used as a shell, as long as its REPL is as rich and featureful enough. Since you can use Python as a shell, which as a language is not exactly the epitome of terseness and expressiveness, you could definitely make do with almost any other interpreted language, too. The problem is that very, very few languages have REPLs that are anywhere near IPython. It's so bad sometimes that you're advised to use `rlwrap` just to get basic line editing and history! I've been working on a new shell based on GNU Smalltalk[1]. I really like the syntax - or lack of thereof - and being able to dump an image (ie. full state of the VM) at any time (and load it later) seemed like a good idea. The only change I needed was to add the `|>` pseudo-operator, which puts what's on the left into parens. Being able to introspect the running session was my primary motivation: I wanted to make the shell and the whole environment as discoverable as possible. I wrote some code for that and then realized that the default REPL uses readline from C, so it freezes the entire VM when waiting for input (including all background threads). My workaround was to set up a socket server and connect to it via rlwrapped telnet... Anyway, I think "do we need a new shell" is the wrong question; instead, we should focus on improving REPLs to the point where a separate shell becomes unnecessary. [1] https://github.com/piotrklibert/stshell https://github.com/piotrklibert/stshell

hnlmorg 5y ago
The REPL is the shell. I even used the term "REPL" (and a couple of synonyms too) in my comment. So I do agree it's critical but that doesn't make the language irrelevant. Your point about how Python shells have had to create syntactic sugar for REPl usage is a good illustration of my point about how it matters a lot. Also you can render images in quite a few terminal emulators already. Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. eg https://github.com/lmorg/murex/blob/master/config/defaults/profile_all.mx#L809-L871 https://github.com/lmorg/murex/blob/master/config/defaults/p... There's definitely room for improvement in the whole TTY / shell / terminal emulator integration space though. But that's not going to happen without breaking support for existing CLI tools. Which basically means it's never going to happen given it wouldn't ever gain traction.
- nine_k 5y ago
  REPLs are highly interactive. Shells are also needed to run scripts. A Bourne shell implementation without Readline is a pain to use as a REPL, but is a completely adequate shell scripting tool.
  - hnlmorg 5y ago
    Scripting without a shell is just scripting. The "shell" is the UI component. Shell's don't even need to be CLI based either (eg explorer.exe, web shells, etc). https://en.wikipedia.org/wiki/Shell_(computing) https://en.wikipedia.org/wiki/Shell_(computing)
- klibertp 5y ago
  > Your point about how Python shells have had to create syntactic sugar for REPL usage is a good illustration of my point about how it matters a lot. Of course it matters. The fact that Python is hardly suited for one-liners means that, if you want to write one-liners in it, you'll have to first extend or change its syntax and stdlib, pouring time and effort into creating a DSL for the interactive use-case. Some languages basically are DSLs for one-liners, like AWK, jq, TCL, Perl and Raku; some require minimal effort, like Lisp, Smalltalk, Ruby or Groovy; and some require a lot of work to make the syntax you want work, like Python, Lua, or JavaScript (or Java). So yes, the choice of language definitely matters, but in principle every language with a REPL can be made into a shell with more or less effort. The problem is that even when you do make that effort, you're stuck with a shell nice for scripting, but with line-editing capabilities of raw readline at best. I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell. > Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. That's an interesting bit of functionality, I will take a look. Thanks!
  - hnlmorg 5y ago
    > I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell. It's doable with existing tools. You have LSP to provide the syntactical framework and there's no shortage of alternatives to readline (I'd written my own[1] to use in murex[2], and open sourced that). [1] https://github.com/lmorg/readline https://github.com/lmorg/readline [2] https://murex.rocks https://murex.rocks The problem you still face is that a good shell will offer autocompletion suggestions for strings that aren't language keywords or function names. eg - file names; and there's a lot of hidden logic in how to do this. Do you build in fzf-like support, just include fzf wholesale but increase your dependency tree, or go for basic path completion. Do you check metadata (eg hidden files and system files on Windows), include dot-prefixed files on Linux / UNIX, etc. How do you know when to return paths, or paths and files, or even know not to return disk items at all? (see next point) - flags for existing CLI tools (assuming you want compatibility with existing tools). Fish and murex will parse man pages to populate suggestions, others rely entirely on the community to write autocompletion scripts. - Are you including variables in your completion of strings. And if so are you reading the variables to spot if it's a path and then following that path. eg `cd $HOME/[tab]` should then return items inside a your home directory even though you've not actually specified your home directory as a string. That means the shell needs to expand the variables to see if it's a valid path. But that's a shell decision rather than a language feature. Some of these lists might take a while to populate so you then have another problem. Do you delay the autocompletion list (bad UX because it slows the user down) or provide the autocompletion sooner. And if the latter, how do you do that without: 1. changing the items under what you're about to select causing you to accidentally select the wrong option 2. communicate that there are update clearly 3. ensure the UI is consistent when slower loading entries might not fit the same dimensions as the space allocated for the list (if you dynamically size your completions to fit the screen real estate) 4. ensure that there's still something present while you're lazy loading the rest of the suggestions; and that those early entries on the completion list are worthwhile and accurate 5. what about sorting the list? Alphabetical? By feature? etc The REPL in murex was inspired by IDEs so I've spent a lot of time trying to consider how to provide the best UX around autocompletion. One thing I've learnt is that it's a lot harder to get right than it seems on the surface.

hnlmorg 5y ago

The REPL is the shell. I even used the term "REPL" (and a couple of synonyms too) in my comment. So I do agree it's critical but that doesn't make the language irrelevant. Your point about how Python shells have had to create syntactic sugar for REPl usage is a good illustration of my point about how it matters a lot. Also you can render images in quite a few terminal emulators already. Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. eg https://github.com/lmorg/murex/blob/master/config/defaults/profile_all.mx#L809-L871 https://github.com/lmorg/murex/blob/master/config/defaults/p... There's definitely room for improvement in the whole TTY / shell / terminal emulator integration space though. But that's not going to happen without breaking support for existing CLI tools. Which basically means it's never going to happen given it wouldn't ever gain traction.

nine_k 5y ago
REPLs are highly interactive. Shells are also needed to run scripts. A Bourne shell implementation without Readline is a pain to use as a REPL, but is a completely adequate shell scripting tool.
- hnlmorg 5y ago
  Scripting without a shell is just scripting. The "shell" is the UI component. Shell's don't even need to be CLI based either (eg explorer.exe, web shells, etc). https://en.wikipedia.org/wiki/Shell_(computing) https://en.wikipedia.org/wiki/Shell_(computing)
klibertp 5y ago
> Your point about how Python shells have had to create syntactic sugar for REPL usage is a good illustration of my point about how it matters a lot. Of course it matters. The fact that Python is hardly suited for one-liners means that, if you want to write one-liners in it, you'll have to first extend or change its syntax and stdlib, pouring time and effort into creating a DSL for the interactive use-case. Some languages basically are DSLs for one-liners, like AWK, jq, TCL, Perl and Raku; some require minimal effort, like Lisp, Smalltalk, Ruby or Groovy; and some require a lot of work to make the syntax you want work, like Python, Lua, or JavaScript (or Java). So yes, the choice of language definitely matters, but in principle every language with a REPL can be made into a shell with more or less effort. The problem is that even when you do make that effort, you're stuck with a shell nice for scripting, but with line-editing capabilities of raw readline at best. I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell. > Some shells (mine included) ship with hooks to autodetect which terminal emulator you're using and find the best method for rendering those images. That's an interesting bit of functionality, I will take a look. Thanks!
- hnlmorg 5y ago
  > I would like to see a framework for creating rich REPLs that would be language agnostic, so that I could get a state of the art auto-completion dialog no matter which language I decided to make into a shell. It's doable with existing tools. You have LSP to provide the syntactical framework and there's no shortage of alternatives to readline (I'd written my own[1] to use in murex[2], and open sourced that). [1] https://github.com/lmorg/readline https://github.com/lmorg/readline [2] https://murex.rocks https://murex.rocks The problem you still face is that a good shell will offer autocompletion suggestions for strings that aren't language keywords or function names. eg - file names; and there's a lot of hidden logic in how to do this. Do you build in fzf-like support, just include fzf wholesale but increase your dependency tree, or go for basic path completion. Do you check metadata (eg hidden files and system files on Windows), include dot-prefixed files on Linux / UNIX, etc. How do you know when to return paths, or paths and files, or even know not to return disk items at all? (see next point) - flags for existing CLI tools (assuming you want compatibility with existing tools). Fish and murex will parse man pages to populate suggestions, others rely entirely on the community to write autocompletion scripts. - Are you including variables in your completion of strings. And if so are you reading the variables to spot if it's a path and then following that path. eg `cd $HOME/[tab]` should then return items inside a your home directory even though you've not actually specified your home directory as a string. That means the shell needs to expand the variables to see if it's a valid path. But that's a shell decision rather than a language feature. Some of these lists might take a while to populate so you then have another problem. Do you delay the autocompletion list (bad UX because it slows the user down) or provide the autocompletion sooner. And if the latter, how do you do that without: 1. changing the items under what you're about to select causing you to accidentally select the wrong option 2. communicate that there are update clearly 3. ensure the UI is consistent when slower loading entries might not fit the same dimensions as the space allocated for the list (if you dynamically size your completions to fit the screen real estate) 4. ensure that there's still something present while you're lazy loading the rest of the suggestions; and that those early entries on the completion list are worthwhile and accurate 5. what about sorting the list? Alphabetical? By feature? etc The REPL in murex was inspired by IDEs so I've spent a lot of time trying to consider how to provide the best UX around autocompletion. One thing I've learnt is that it's a lot harder to get right than it seems on the surface.