11 ms·

Rich Command Shells

187 points by BruceM 12y ago

BruceM 12y ago
I will probably write a "More Rich Command Shells" to cover things I missed here that are important in some way (like Apple's MPW). The next thing I want to write though is about building something that has a command shell UI today and thinking about how to do so in a flexible way that works with multiple output devices.
- agumonkey 12y ago
  Thanks for that article. I'm often thinking about how to present/"unify" these: - command (statements validated by some token), - repl (expression, less side-effect driven than the previous one), - unary/immediate mode (each input event is interpretable right away, direct mapping between input and action), - n-ary/non-immediate mode (some parsing occurs, gathering a few tokens into a higher level construct) - non keyboard based UI/UX (some sofware like Maya have proper separation, most mouse events have direct translation into object methods)
- deleted 12y ago
  [deleted]
- rsync 12y ago
  Please do - I liked this first article very much and would look forward to a follow-up.
josephg 12y ago
I miss TermKit[1]. Its a real shame that the developer abandoned it after it got a lot of hype. There's a huge opportunity for someone to come along and make either a new terminal encoding which allows rich, interactive output or extend VT somehow to which allows the same. If you could extend the terminal protocol, you might even be able to get it to work over SSH. HTML+JS seem like the obvious way to do it - even though the web is an awful platform, its standard, cross-platform and fully featured. Its the perfect worse-is-better solution for this. I think the hardest part would be figuring out how to reconcile browser-like UI events and file streams. Maybe you'd need to make a standardized event serialization format so your process could receive serialized events via stdin or something like that. I think it'll be a really hard sell if we have to abandon our unix pipes entirely. [1] https://github.com/unconed/TermKit https://github.com/unconed/TermKit Previous hackernews discussion around termkit: https://news.ycombinator.com/item?id=2559734 https://news.ycombinator.com/item?id=2559734
- lallysingh 12y ago
  > HTML+JS seem like the obvious way to do it > I think the hardest part would be figuring out how to reconcile browser-like UI events and file streams. How about Chrome Apps? They've got a mix of file I/O capability and HTML/JS.
pjmlp 12y ago
And this is why UNIX shells feel so primitive.
mozmark 12y ago
If you found this interesting, you may also like GCLI (as featured in the Firefox dev tools command line - or you can play with it here: http://mozilla.github.io/gcli/ http://mozilla.github.io/gcli/ ).
- BruceM 12y ago
  GCLI is interesting, but I didn't mention it because I feel like it focuses more on the issue of command parsing / completion. That said, that's a great topic to cover! I think GCLI does a pretty good job of command completion. The Lisp Machine did as well. I have friends who love some of the router CLIs, but not sure which one(s).
  - lispm 12y ago
    Cisco's IOS is an example.
robbles 12y ago
CLOS and CLIM always sounded to me like the names of programs you'd expect to see competing at disc wars, or racing in a lightcycle grid.
tubelite 12y ago
If I might plug a rich command shell I'm developing: http://pigshell.com http://pigshell.com (Source at https://github.com/pigshell/pigshell https://github.com/pigshell/pigshell) It provides - A shell for the web. Runs in the browser, pure client-side. - File-like abstraction for URLs and other entities exposed by web APIs - Unix-like style of composing commands using pipes. - Visualization using HTML For instance, cp -r /gdrive/<username> /home will backup the contents of your Google Drive to your desktop (see http://pigshell.com/v/0.6.2/doc/gdrive.html http://pigshell.com/v/0.6.2/doc/gdrive.html for details). Replace /home with /dropbox/<username>, and the same command will back up GDrive to Dropbox. And so on. While it is already useful, there is still some work, especially around hardening the file abstraction/APIs (and reams of documentation) before before it can be horizontally expanded to support a bunch more APIs and cloud stores. I am actively working on these and expect they'll take ~2 months.
- qznc 12y ago
  Interesting. Why do you think that files are good abstraction for a web shell?
  - tubelite 12y ago
    It's the old "if you have a hammer, everything looks like a nail". Since I'm a filesystem engineer, everything looks like a file :) Seriously though: the file is a powerful and familiar abstraction. The primary motivation for pigshell was to provide a common minimum abstraction across different web APIs which would enable basic data movement and backup. That said, pigshell commands actually pass objects across the pipeline; files are a kind of object. For instance, cat http://pigshell.com/sample/life-expectancy.html | table2js -e "tr" foo country data | head | printf extracts data from an html table, converts them to plain Javascript objects and prints their json representation.
  - judk 12y ago
    Are you familiar with Plan9? Everything is a file. Everything.
akavel 12y ago
Given all of that, and also Oberon -- '80s too, and even the Engelbart's Demo -- '68!, I totally every day wonder and can't understand why in 2010's all of the "modern" OSes still provide the developer just with text-only consoles??... okay, maybe for end-users there was a jump (in interfaces and features), and maybe it was significant indeed (movies editing, 3d modelling/sculpting, possibility of instant communication with significant fraction of all the people in the world, to bring up some). But it still feels like even for them, some things were there, and now are not (sorry for vagueness, but I don't have much time now to think about examples, unfortunately. When I'm gonna finally start this blog thing, one day...). Any theories, anyone? I'm really curious. Still believe this can be improved, and work on some ideas in my free time, but I often wonder why I have to, and I can't already use those beautiful ancient features?
- pjmlp 12y ago
  Because many seem to be willing to go back to the 70's (vi/sh) instead of embracing the progress you mention. Since I discovered UNIX, with Xenix in 1994 followed by many other variants, I always looked for ways to replicate such workflows in UNIX. At the end of the day, Mac OS X and Windows communities are more welcoming to such ways of working.
  - akavel 12y ago
    But, again, why? this can still have many reasons: not willing because of technological obstacles? complexity? because they shown in fact to be somehow worse? (e.g. pure text easier to read, mold, and more universal?) because of some social obstacles? (if yes, of what kind?) what else could that be? (those are just a few of my random quick theories, have I described all possibilities?) edit: the comment by zorbo seems an interesting answer to my question, but I'm still not 100% convinced; also, I must still ask how can we be sure we're not victims of a text-mode "Stockholm Syndrome" in this regard?
    - pjmlp 12y ago
      Because the majority of those people never had the opportunity to work with Smalltalk, Lisp Machines, Oberon, Amiga,... So they look around and see only UNIX CLI. It is just like mankind during the middle ages that never experienced how the Roman and Greek society were developed before the empire downfall.
ivanca 12y ago
Strange he didn't mention Xiki: http://xsh.org/ http://xsh.org/ http://xiki.org/ http://xiki.org/
zorbo 12y ago
Text is the universal interface. You can do things with it. You can strip it, cut it, transform it, send it to other places. Humans can read it, programs can read it, your printer can output it. It can be sent to web APIs, it can be stored anywhere. It's compressible, can be colored and can be copy-pasted and is infinitely extendable. Thousands of protocols run over it. The command line works with text. The command line remains the best interface I've ever used. It's user friendly, composable and available everywhere. It's easy to automate and easy to extend. I wish the "command line with pictures" idea would just go away already. It adds nothing for the general public. I can already view pictures on remote machines with X forwarding. Command line with pictures never made it, because there are ten competing standards. With text, everybody just agreed on ASCII and now Unicode/UTF8. Text has hundreds of ugly clutches on top of it (Extended ASCII, ANSI, Escape codes, etc, etc). It still works. It's still simple. It has its problems, but nowhere near as many problems as GUIs. Those who don't understand Unix are doomed to reimplement it... poorly.
- aduitsis 12y ago
  Exactly as you indicate, even in "classic" terminals we have features like unicode support, color, geometry reporting, mouse reporting, window title manipulation, arbitrary cursor movement, etc. All these are in widespread use today. These features could not have been implemented in a real tty, but nowadays I think it makes sense to have them. Maybe some are a little kludge-ish, but are we really arguing about their usefulness? So why shouldn't we have a couple of extra escape sequences where the terminal could, e.g. draw an arbitrary bitmap? You are right, that would certainly require some sort of standard becoming prevalent, but why would a little extra capability (that doesn't break compatibility) be a bad thing?
  - a3n 12y ago
    I wouldn't mind it.
  - lmm 12y ago
    Like extending email, there's a a chicken-and-egg problem. What terminal emulator is going to add a command that no program uses? What program is going to use a command that no terminal implements? By all means implement this thing - I wouldn't be surprised if there were already terminals that did that. But personally I'd be surprised if it caught on. Compatibility matters.
josh-wrale 12y ago
How about a declarative and idempotent shell? Well, more declarative and idempotent than bash + the GNU core utils.
- _mhr_ 12y ago
  How would this be useful?
  - josh-wrale 12y ago
    This comment thread discusses the methodology. https://news.ycombinator.com/item?id=7378764#up_7380392 https://news.ycombinator.com/item?id=7378764#up_7380392
lispm 12y ago
If you come to Freiheit.com next week (16. Oct 2014) to the Clojure User Group Meeting in Hamburg/Germany, I'll demo the Dynamic Windows user interface of a real Symbolics Lisp Machine. Including its command shell called 'Dynamic Lisp Listener'. http://www.meetup.com/ClojureUserGroupHH/events/207314372/ http://www.meetup.com/ClojureUserGroupHH/events/207314372/
- cpach 12y ago
  Cool! Do you know if the demo will be recorded?
  - lispm 12y ago
    It won't.
    - agumonkey 12y ago
      Oh the sadness. Please harass someone with a capable smartphone to attempt at least :)
  - Scuds 12y ago
    This demo appears to be running on an emulator instead of an actual antique piece of hardware but you can get the idea. https://www.youtube.com/watch?v=o4-YnLpLgtk https://www.youtube.com/watch?v=o4-YnLpLgtk the AI winter is something you don't hear too much about. http://en.wikipedia.org/wiki/AI_winter http://en.wikipedia.org/wiki/AI_winter
vinodkd 12y ago
Slightly off-topic, but the recent surge in responsive UI had me thinking thus: If we are now driven to merge UI logic across different graphical devices, can we think of apps that span both textual and graphical devices? After all, the core functionality of the application remains the same. To use a simple example: when you search for a product online, get a search result list and then select one from the list, couldnt this flow be modeled just the same in both gui and text interfaces? I'm thinking back to the turbo-pascal style applications of the past that produced full-blown IDEs in text, or the wordstars/wordperfects of yore: the UI model that sits behind those apps cannot be much different in principle from the modern day equivalents. Even farther back, there was a time (and probably still is for college assignments) when cli applications had a prompt-read user input-respond cycle, replete with text-based choices to select from and so forth. What if we were to merge the two worlds instead of trying to get one to confirm to the other?
fiatjaf 12y ago
What about the Aurora/Eve thing? https://www.youtube.com/watch?v=L6iUm_Cqx2s https://www.youtube.com/watch?v=L6iUm_Cqx2s It is a shell, isn't it?

187 points by BruceM 12y ago

I will probably write a "More Rich Command Shells" to cover things I missed here that are important in some way (like Apple's MPW). The next thing I want to write though is about building something that has a command shell UI today and thinking about how to do so in a flexible way that works with multiple output devices.

agumonkey 12y ago
Thanks for that article. I'm often thinking about how to present/"unify" these: - command (statements validated by some token), - repl (expression, less side-effect driven than the previous one), - unary/immediate mode (each input event is interpretable right away, direct mapping between input and action), - n-ary/non-immediate mode (some parsing occurs, gathering a few tokens into a higher level construct) - non keyboard based UI/UX (some sofware like Maya have proper separation, most mouse events have direct translation into object methods)
deleted 12y ago
[deleted]
rsync 12y ago
Please do - I liked this first article very much and would look forward to a follow-up.

josephg 12y ago

I miss TermKit[1]. Its a real shame that the developer abandoned it after it got a lot of hype. There's a huge opportunity for someone to come along and make either a new terminal encoding which allows rich, interactive output or extend VT somehow to which allows the same. If you could extend the terminal protocol, you might even be able to get it to work over SSH. HTML+JS seem like the obvious way to do it - even though the web is an awful platform, its standard, cross-platform and fully featured. Its the perfect worse-is-better solution for this. I think the hardest part would be figuring out how to reconcile browser-like UI events and file streams. Maybe you'd need to make a standardized event serialization format so your process could receive serialized events via stdin or something like that. I think it'll be a really hard sell if we have to abandon our unix pipes entirely. [1] https://github.com/unconed/TermKit https://github.com/unconed/TermKit Previous hackernews discussion around termkit: https://news.ycombinator.com/item?id=2559734 https://news.ycombinator.com/item?id=2559734

lallysingh 12y ago
> HTML+JS seem like the obvious way to do it > I think the hardest part would be figuring out how to reconcile browser-like UI events and file streams. How about Chrome Apps? They've got a mix of file I/O capability and HTML/JS.

pjmlp 12y ago

And this is why UNIX shells feel so primitive.

mozmark 12y ago

If you found this interesting, you may also like GCLI (as featured in the Firefox dev tools command line - or you can play with it here: http://mozilla.github.io/gcli/ http://mozilla.github.io/gcli/ ).

BruceM 12y ago
GCLI is interesting, but I didn't mention it because I feel like it focuses more on the issue of command parsing / completion. That said, that's a great topic to cover! I think GCLI does a pretty good job of command completion. The Lisp Machine did as well. I have friends who love some of the router CLIs, but not sure which one(s).
- lispm 12y ago
  Cisco's IOS is an example.

robbles 12y ago

CLOS and CLIM always sounded to me like the names of programs you'd expect to see competing at disc wars, or racing in a lightcycle grid.

tubelite 12y ago

If I might plug a rich command shell I'm developing: http://pigshell.com http://pigshell.com (Source at https://github.com/pigshell/pigshell https://github.com/pigshell/pigshell) It provides - A shell for the web. Runs in the browser, pure client-side. - File-like abstraction for URLs and other entities exposed by web APIs - Unix-like style of composing commands using pipes. - Visualization using HTML For instance, cp -r /gdrive/<username> /home will backup the contents of your Google Drive to your desktop (see http://pigshell.com/v/0.6.2/doc/gdrive.html http://pigshell.com/v/0.6.2/doc/gdrive.html for details). Replace /home with /dropbox/<username>, and the same command will back up GDrive to Dropbox. And so on. While it is already useful, there is still some work, especially around hardening the file abstraction/APIs (and reams of documentation) before before it can be horizontally expanded to support a bunch more APIs and cloud stores. I am actively working on these and expect they'll take ~2 months.

qznc 12y ago
Interesting. Why do you think that files are good abstraction for a web shell?
- tubelite 12y ago
  It's the old "if you have a hammer, everything looks like a nail". Since I'm a filesystem engineer, everything looks like a file :) Seriously though: the file is a powerful and familiar abstraction. The primary motivation for pigshell was to provide a common minimum abstraction across different web APIs which would enable basic data movement and backup. That said, pigshell commands actually pass objects across the pipeline; files are a kind of object. For instance, cat http://pigshell.com/sample/life-expectancy.html | table2js -e "tr" foo country data | head | printf extracts data from an html table, converts them to plain Javascript objects and prints their json representation.
- judk 12y ago
  Are you familiar with Plan9? Everything is a file. Everything.

akavel 12y ago

Given all of that, and also Oberon -- '80s too, and even the Engelbart's Demo -- '68!, I totally every day wonder and can't understand why in 2010's all of the "modern" OSes still provide the developer just with text-only consoles??... okay, maybe for end-users there was a jump (in interfaces and features), and maybe it was significant indeed (movies editing, 3d modelling/sculpting, possibility of instant communication with significant fraction of all the people in the world, to bring up some). But it still feels like even for them, some things were there, and now are not (sorry for vagueness, but I don't have much time now to think about examples, unfortunately. When I'm gonna finally start this blog thing, one day...). Any theories, anyone? I'm really curious. Still believe this can be improved, and work on some ideas in my free time, but I often wonder why I have to, and I can't already use those beautiful ancient features?

pjmlp 12y ago
Because many seem to be willing to go back to the 70's (vi/sh) instead of embracing the progress you mention. Since I discovered UNIX, with Xenix in 1994 followed by many other variants, I always looked for ways to replicate such workflows in UNIX. At the end of the day, Mac OS X and Windows communities are more welcoming to such ways of working.
- akavel 12y ago
  But, again, why? this can still have many reasons: not willing because of technological obstacles? complexity? because they shown in fact to be somehow worse? (e.g. pure text easier to read, mold, and more universal?) because of some social obstacles? (if yes, of what kind?) what else could that be? (those are just a few of my random quick theories, have I described all possibilities?) edit: the comment by zorbo seems an interesting answer to my question, but I'm still not 100% convinced; also, I must still ask how can we be sure we're not victims of a text-mode "Stockholm Syndrome" in this regard?
  - pjmlp 12y ago
    Because the majority of those people never had the opportunity to work with Smalltalk, Lisp Machines, Oberon, Amiga,... So they look around and see only UNIX CLI. It is just like mankind during the middle ages that never experienced how the Roman and Greek society were developed before the empire downfall.

ivanca 12y ago

Strange he didn't mention Xiki: http://xsh.org/ http://xsh.org/ http://xiki.org/ http://xiki.org/

zorbo 12y ago

Text is the universal interface. You can do things with it. You can strip it, cut it, transform it, send it to other places. Humans can read it, programs can read it, your printer can output it. It can be sent to web APIs, it can be stored anywhere. It's compressible, can be colored and can be copy-pasted and is infinitely extendable. Thousands of protocols run over it. The command line works with text. The command line remains the best interface I've ever used. It's user friendly, composable and available everywhere. It's easy to automate and easy to extend. I wish the "command line with pictures" idea would just go away already. It adds nothing for the general public. I can already view pictures on remote machines with X forwarding. Command line with pictures never made it, because there are ten competing standards. With text, everybody just agreed on ASCII and now Unicode/UTF8. Text has hundreds of ugly clutches on top of it (Extended ASCII, ANSI, Escape codes, etc, etc). It still works. It's still simple. It has its problems, but nowhere near as many problems as GUIs. Those who don't understand Unix are doomed to reimplement it... poorly.

aduitsis 12y ago
Exactly as you indicate, even in "classic" terminals we have features like unicode support, color, geometry reporting, mouse reporting, window title manipulation, arbitrary cursor movement, etc. All these are in widespread use today. These features could not have been implemented in a real tty, but nowadays I think it makes sense to have them. Maybe some are a little kludge-ish, but are we really arguing about their usefulness? So why shouldn't we have a couple of extra escape sequences where the terminal could, e.g. draw an arbitrary bitmap? You are right, that would certainly require some sort of standard becoming prevalent, but why would a little extra capability (that doesn't break compatibility) be a bad thing?
- a3n 12y ago
  I wouldn't mind it.
- lmm 12y ago
  Like extending email, there's a a chicken-and-egg problem. What terminal emulator is going to add a command that no program uses? What program is going to use a command that no terminal implements? By all means implement this thing - I wouldn't be surprised if there were already terminals that did that. But personally I'd be surprised if it caught on. Compatibility matters.

josh-wrale 12y ago

How about a declarative and idempotent shell? Well, more declarative and idempotent than bash + the GNU core utils.

_mhr_ 12y ago
How would this be useful?
- josh-wrale 12y ago
  This comment thread discusses the methodology. https://news.ycombinator.com/item?id=7378764#up_7380392 https://news.ycombinator.com/item?id=7378764#up_7380392

lispm 12y ago

If you come to Freiheit.com next week (16. Oct 2014) to the Clojure User Group Meeting in Hamburg/Germany, I'll demo the Dynamic Windows user interface of a real Symbolics Lisp Machine. Including its command shell called 'Dynamic Lisp Listener'. http://www.meetup.com/ClojureUserGroupHH/events/207314372/ http://www.meetup.com/ClojureUserGroupHH/events/207314372/

cpach 12y ago
Cool! Do you know if the demo will be recorded?
- lispm 12y ago
  It won't.
  - agumonkey 12y ago
    Oh the sadness. Please harass someone with a capable smartphone to attempt at least :)
- Scuds 12y ago
  This demo appears to be running on an emulator instead of an actual antique piece of hardware but you can get the idea. https://www.youtube.com/watch?v=o4-YnLpLgtk https://www.youtube.com/watch?v=o4-YnLpLgtk the AI winter is something you don't hear too much about. http://en.wikipedia.org/wiki/AI_winter http://en.wikipedia.org/wiki/AI_winter

vinodkd 12y ago

Slightly off-topic, but the recent surge in responsive UI had me thinking thus: If we are now driven to merge UI logic across different graphical devices, can we think of apps that span both textual and graphical devices? After all, the core functionality of the application remains the same. To use a simple example: when you search for a product online, get a search result list and then select one from the list, couldnt this flow be modeled just the same in both gui and text interfaces? I'm thinking back to the turbo-pascal style applications of the past that produced full-blown IDEs in text, or the wordstars/wordperfects of yore: the UI model that sits behind those apps cannot be much different in principle from the modern day equivalents. Even farther back, there was a time (and probably still is for college assignments) when cli applications had a prompt-read user input-respond cycle, replete with text-based choices to select from and so forth. What if we were to merge the two worlds instead of trying to get one to confirm to the other?

fiatjaf 12y ago

What about the Aurora/Eve thing? https://www.youtube.com/watch?v=L6iUm_Cqx2s https://www.youtube.com/watch?v=L6iUm_Cqx2s It is a shell, isn't it?