5 ms·

I'm working on creating a modern terminal interface with Electron where you can use the mouse to modify the command line, each command sends output to a separat

I'm working on creating a modern terminal interface with Electron where you can use the mouse to modify the command line, each command sends output to a separate "output block" (think Jupyter notebook), you have global autocompletion from history regardless of which machine you are sshed into or how many shells deep you are, and where every time you start typing a command the man page automatically appears in a sidebar. This is surprisingly hard to implement. If you just run bash unmodified as the backend then you can't multiplex the output from different commands and therefore you can't display output from different commands in different blocks. If you run each command in a separate shell then you lose automatic tracking of environment variables and working directories... (There are other challenges but they are a little easier to work around.) Does anyone have creative ideas for how to work around this?

rogual 6y ago
Bit hacky but you could keep a long-running Bash process and use PROMPT_COMMAND to notify you to start a new block? I love this kind of thing, by the way. I'm sure terminal emulators are not the best we can do, CLI-wise.
- sime2009 6y ago
  The shell integration in [Extraterm](http://extraterm.org/features.html http://extraterm.org/features.html) and [iTerm2](https://iterm2.com/documentation-shell-integration.html https://iterm2.com/documentation-shell-integration.html) both work via 'hooks' related to the command prompt and command execution.
pletnes 6y ago
Jupyter already let you run shell so at least have a look for inspiration - and limitations that would be nice to lift (e.g faster output)
dan-robertson 6y ago
I think something like this would be good, but I don’t think I’d base it on bash. Some of the things I would have are: 1. When someone writes a command like foo | bar, instead of making a single pipe from one to the next, put your own lightweight program that works a bit like pv in the middle. In the standard case this is a pretty cheap program: it just needs to call splice to move from stdin to stdout, but you could have some mechanism to eg view the data going through a pipe or the rate of data: one could tell if the program is doing anything 2. Typical interactive shell use involves writing commands, inspecting output, then modifying the commands. A notebook format may be better for that. I think I’d want some mechanism to say “these commands will generate some output. I want to inspect it then write the next stage in the pipeline so hang onto that output instead of rerunning the command.” And why not make some “interactive grep” and interactive versions of other commands so that one can get immediate feedback when modifying the regex. 3. It’s hard to know if a command will have side effects but I feel like it matters a lot. Maybe something could be done to detect it somehow.
- effie 6y ago
  Userland has something like this. https://hisham.hm/userland/ https://hisham.hm/userland/
  - dan-robertson 6y ago
    This is super cool. Thanks!
Conlectus 6y ago
Ha! I was working in literally the same thing a while ago (a terminal emulator that used separate panes for commands, based on Electron). You could potentially pipe every command output into a separate FIFO file and then run them in the background.
chubot 6y ago
(Oil author here) Yeah I think you will run into many problems doing this in bash, and other people have. A few people are working on similar UIs for shell, which I think is a great idea. I would like you all to get together and tell me what kind of API or IPC Oil needs to support it. And maybe hack on the code too :) There is some nacsent / working code out there, linked in this issue. https://github.com/oilshell/oil/issues/738 https://github.com/oilshell/oil/issues/738 I also created this #shell-gui Zulip channel in December: https://oilshell.zulipchat.com/#narrow/stream/266977-shell-gui https://oilshell.zulipchat.com/#narrow/stream/266977-shell-g... (requires login, e.g. with Github or Google) Oil has an extremely modular architecture so it should be able to support this (within the constraints of what the kernel allows) Here I say that "Oil is to bash as LLVM/Clang is to GCC", which is accurate: http://www.oilshell.org/blog/2021/01/why-a-new-shell.html#more-ways-to-understand-oil http://www.oilshell.org/blog/2021/01/why-a-new-shell.html#mo... Many experiments in the same vein here: https://github.com/oilshell/oil/wiki/Interactive-Shell https://github.com/oilshell/oil/wiki/Interactive-Shell
sime2009 6y ago
I've been working on my own terminal for a while now, and capturing the output of separate commands and reusing it is one it its unique features. See Extraterm in [action here](http://extraterm.org/features.html#shell-integration http://extraterm.org/features.html#shell-integration) It doesn't quite do what you are asking though. There is shell integration for some fancy features and it works with the most common 'normal' shells. I don't have a custom shell or similar which deeply integrates with Extraterm to automatically run commands in separate panes or windows. Extraterm is based around the idea of a stack of 'blocks' which can hold output, images or more complex data (possiblye tabular data) etc. Although I haven't tried it directly, emulating something like a Jupyter notebook should be possible after writing some extensions via Extraterm's extension API. A session spanning 'global' history would be fairly easy to implement as an Extraterm extension. Getting what you in particular are asking for to work across ssh, containers, etc is difficult. You need to run your own code (=shell or "command runner") on the remote end somehow and you don't want a manual installation step everywhere. The only thing left is doing some kind of over the terminal delivery of code to the remote shell and then running it.
ilyash 6y ago
Hi. I'm Ilya, author of NGS.I see there is (at least) some overlap between the UI you are thinking about and the one that I'm thinking about - https://github.com/ngs-lang/ngs/wiki/UI-Design https://github.com/ngs-lang/ngs/wiki/UI-Design The plan for NGS is to support multiple UIs. I was thinking terminal and web. I would also (as the author of Oil shell) be glad to hear about APIs and protocols. > start typing a command the man page automatically appears in a sidebar. How about this one (unknown amount of work but I was thinking about this in NGS) - parse the man page in order to translate an exit code to a human-readable error message?
Per_Bothner 6y ago
DomTerm (https://domterm.org/ https://domterm.org/) solves part of the puzzle. See https://domterm.org/saved/dt-prettynested.html https://domterm.org/saved/dt-prettynested.html for a "live screenshot" - it's a capture of a bash+KawaScheme session, but with some JavaScript so it is still responsive. In addition to the highlighting of input vs prompt, note the little triangles which allow you to hide parts of the output. Try it - it's live. The triangles at the right of the screen are emitted by bash (see instructions here https://domterm.org/Shell-prompts.html https://domterm.org/Shell-prompts.html) or the Kawa Scheme REPL. The triangles elsewhere are emitted by the application (PrintNested or Kawa) to display a nested data structure. Note the latter does dynamic pretty-printing - re-size the window and the line-breaks are recalculated by JavaScript in a way that shows the logical structure of the output. (This is done by the application sending some special escape sequences with the output. The escape sequences are recorded on the DomTerm data structure, and handle re-flow for commands that are dead.) Also notice the support for nested command groups: Clicking the button by the "kawa" (shell) command hides all the non-initial lines from the Kawa command, while buttons by individual Kawa input lines just hide that Kawa command. What I'd like to add is a hook for the "job" feature of various shells. When you put a job into the background, it would hide the output from the command, which would continue to produce hidden output. The shell would emit a new prompt for a new command. All background jobs could be made visible at the same the (if you want) by clicking the show/hide triangles, and hidden by re-clicking. One way a shell could do this is by creating a new pty when creating a new job. DomTerm could be set up to interleave the output front different ptys. (DomTerm could build on its "buffer" model, which extends the xterm main/alternate buffer model. DomTerm supports an arbitrary number of buffers, which are displayed one below the other. Interleaving pty could build on this by tying each pty to a new buffer.)

rogual 6y ago

Bit hacky but you could keep a long-running Bash process and use PROMPT_COMMAND to notify you to start a new block? I love this kind of thing, by the way. I'm sure terminal emulators are not the best we can do, CLI-wise.

sime2009 6y ago
The shell integration in [Extraterm](http://extraterm.org/features.html http://extraterm.org/features.html) and [iTerm2](https://iterm2.com/documentation-shell-integration.html https://iterm2.com/documentation-shell-integration.html) both work via 'hooks' related to the command prompt and command execution.

pletnes 6y ago

Jupyter already let you run shell so at least have a look for inspiration - and limitations that would be nice to lift (e.g faster output)

dan-robertson 6y ago

I think something like this would be good, but I don’t think I’d base it on bash. Some of the things I would have are: 1. When someone writes a command like foo | bar, instead of making a single pipe from one to the next, put your own lightweight program that works a bit like pv in the middle. In the standard case this is a pretty cheap program: it just needs to call splice to move from stdin to stdout, but you could have some mechanism to eg view the data going through a pipe or the rate of data: one could tell if the program is doing anything 2. Typical interactive shell use involves writing commands, inspecting output, then modifying the commands. A notebook format may be better for that. I think I’d want some mechanism to say “these commands will generate some output. I want to inspect it then write the next stage in the pipeline so hang onto that output instead of rerunning the command.” And why not make some “interactive grep” and interactive versions of other commands so that one can get immediate feedback when modifying the regex. 3. It’s hard to know if a command will have side effects but I feel like it matters a lot. Maybe something could be done to detect it somehow.

effie 6y ago
Userland has something like this. https://hisham.hm/userland/ https://hisham.hm/userland/
- dan-robertson 6y ago
  This is super cool. Thanks!

Conlectus 6y ago

Ha! I was working in literally the same thing a while ago (a terminal emulator that used separate panes for commands, based on Electron). You could potentially pipe every command output into a separate FIFO file and then run them in the background.

chubot 6y ago

(Oil author here) Yeah I think you will run into many problems doing this in bash, and other people have. A few people are working on similar UIs for shell, which I think is a great idea. I would like you all to get together and tell me what kind of API or IPC Oil needs to support it. And maybe hack on the code too :) There is some nacsent / working code out there, linked in this issue. https://github.com/oilshell/oil/issues/738 https://github.com/oilshell/oil/issues/738 I also created this #shell-gui Zulip channel in December: https://oilshell.zulipchat.com/#narrow/stream/266977-shell-gui https://oilshell.zulipchat.com/#narrow/stream/266977-shell-g... (requires login, e.g. with Github or Google) Oil has an extremely modular architecture so it should be able to support this (within the constraints of what the kernel allows) Here I say that "Oil is to bash as LLVM/Clang is to GCC", which is accurate: http://www.oilshell.org/blog/2021/01/why-a-new-shell.html#more-ways-to-understand-oil http://www.oilshell.org/blog/2021/01/why-a-new-shell.html#mo... Many experiments in the same vein here: https://github.com/oilshell/oil/wiki/Interactive-Shell https://github.com/oilshell/oil/wiki/Interactive-Shell

sime2009 6y ago

I've been working on my own terminal for a while now, and capturing the output of separate commands and reusing it is one it its unique features. See Extraterm in [action here](http://extraterm.org/features.html#shell-integration http://extraterm.org/features.html#shell-integration) It doesn't quite do what you are asking though. There is shell integration for some fancy features and it works with the most common 'normal' shells. I don't have a custom shell or similar which deeply integrates with Extraterm to automatically run commands in separate panes or windows. Extraterm is based around the idea of a stack of 'blocks' which can hold output, images or more complex data (possiblye tabular data) etc. Although I haven't tried it directly, emulating something like a Jupyter notebook should be possible after writing some extensions via Extraterm's extension API. A session spanning 'global' history would be fairly easy to implement as an Extraterm extension. Getting what you in particular are asking for to work across ssh, containers, etc is difficult. You need to run your own code (=shell or "command runner") on the remote end somehow and you don't want a manual installation step everywhere. The only thing left is doing some kind of over the terminal delivery of code to the remote shell and then running it.

ilyash 6y ago

Hi. I'm Ilya, author of NGS.I see there is (at least) some overlap between the UI you are thinking about and the one that I'm thinking about - https://github.com/ngs-lang/ngs/wiki/UI-Design https://github.com/ngs-lang/ngs/wiki/UI-Design The plan for NGS is to support multiple UIs. I was thinking terminal and web. I would also (as the author of Oil shell) be glad to hear about APIs and protocols. > start typing a command the man page automatically appears in a sidebar. How about this one (unknown amount of work but I was thinking about this in NGS) - parse the man page in order to translate an exit code to a human-readable error message?

Per_Bothner 6y ago

DomTerm (https://domterm.org/ https://domterm.org/) solves part of the puzzle. See https://domterm.org/saved/dt-prettynested.html https://domterm.org/saved/dt-prettynested.html for a "live screenshot" - it's a capture of a bash+KawaScheme session, but with some JavaScript so it is still responsive. In addition to the highlighting of input vs prompt, note the little triangles which allow you to hide parts of the output. Try it - it's live. The triangles at the right of the screen are emitted by bash (see instructions here https://domterm.org/Shell-prompts.html https://domterm.org/Shell-prompts.html) or the Kawa Scheme REPL. The triangles elsewhere are emitted by the application (PrintNested or Kawa) to display a nested data structure. Note the latter does dynamic pretty-printing - re-size the window and the line-breaks are recalculated by JavaScript in a way that shows the logical structure of the output. (This is done by the application sending some special escape sequences with the output. The escape sequences are recorded on the DomTerm data structure, and handle re-flow for commands that are dead.) Also notice the support for nested command groups: Clicking the button by the "kawa" (shell) command hides all the non-initial lines from the Kawa command, while buttons by individual Kawa input lines just hide that Kawa command. What I'd like to add is a hook for the "job" feature of various shells. When you put a job into the background, it would hide the output from the command, which would continue to produce hidden output. The shell would emit a new prompt for a new command. All background jobs could be made visible at the same the (if you want) by clicking the show/hide triangles, and hidden by re-clicking. One way a shell could do this is by creating a new pty when creating a new job. DomTerm could be set up to interleave the output front different ptys. (DomTerm could build on its "buffer" model, which extends the xterm main/alternate buffer model. DomTerm supports an arbitrary number of buffers, which are displayed one below the other. Interleaving pty could build on this by tying each pty to a new buffer.)