9 ms·

Building the Future of the Command Line

90 points by smartblondeva 4y ago

rektide 4y ago
The command line ought hybridize some, please. Having better self describing interfaces, machine to machine capabilities... humans are awesome enouhh to whip up super wild magic on the fly ("spellcasting on the fly") but these same tools are much weirder to use as you descend into scripting, as you start bringing "real" programming languages in (which have their own alternate realities: "standard libraries"). The command line should/ought bridge & integrate better. Making it more usable from these higher (more pre-baked/automated) levels is one side. And then reciprocally, how wonderful it would be to see execution flow expressed less in terms of stack traces & more in terms of networks of communicating processes. Create boundary layers, make the cli tools visible & known operations sequenced by (but still visible within) higher level systems. Cli on and on!!
- jrm4 4y ago
  Amen. Offhand, can we start with e.g. a 2 or 3 window deal, in which there is a file manager and a document viewer that one can easily copy and paste from? There's stuff out there that sort of does this but it could be done MUCH better.
an1sotropy 4y ago
One thing not discussed are the libraries used for command-line parsing (parsing argv), and how that might get complicated by shells trying to make the command-line into something effectively more than an array of strings. Having written a non-trivial command-line parser in C, and having used a bunch of them in other languages, it seems to me that this task would benefit from some more standardization and maturation. What is the JSON of the command-line? How can we do to increase the level of interoperability between how information is encoded on different tools' command-lines? e.g. think of ImageMagick "convert" versus "find" versus "ffmpeg": totally different universes, but all of them in their own way turn command-line arguments into mini-DSLs.
- scj 4y ago
  I've always wondered about expanding stdin, stdout, stderr. Say, stdjson that doesn't get visually displayed, but can be piped (and would only be generated if it is needed on the pipe stream). ls | cat </dev/stdjson | string_proc_the_json_for_some_reason With the direct ability to process in line: ls -a | json.files[0].last_modified I'd probably want multiple output formats (including s-expressions).
  - BirAdam 4y ago
    You can always just use JQ. Additionally, as JSON is text, you can use awk/sed/grep and so on.
    - scj 4y ago
      The intent is that standard tools would output alternative formats.
      - an1sotropy 4y ago
        Right; every file is just bytes but we get a lot of mileage out of libraries like libpng that parse those bytes into usefully structured info. And I was pondering what more could evolve to parse info from the command line. I think stdjson is insane yet awesome to ponder.
        dotancohen 4y ago
        I agree that a standardized structured format would be awesome, but I'm not convinced that JSON is it. And there should _not_ be more than one. One thing that I do like about JSON is that it is ubiquitous - and that makes up for a lot of its other faults. But I would like to see proposed use cases that JSON would not support beforehand, to clearly define where the limitations are, and what limitations the community is willing to accept.
      - nishs 4y ago
        The tool should instead provide an `--output=<format>` flag, where <format> is one of text (default), json, xml, etc.
anigbrowl 4y ago
Very well written article and very impressive magazine generally.
Kukumber 4y ago
"your server needs a GPU and an i5 to use our shell, as it provides a graphical interface and some shader animation because that's what attract the money people, they want shiny stuff y'know" This trend of "new modern shells" that runs and start as slow as some javascript code (powershell) needs to stop People forgot what shells are for, and what scripting is for
- yazzku 4y ago
  Indeed. These pseudo-authoritative blogs on Github from people who have no idea what they're talking about could also stop.
- rektide 4y ago
  Doesn't seem like a real problem. Why are we grumbling about this? No one is making rxvt less an option. Why the fear? Why the negativity? Does start-up time matter? Who cares? (I have three terminals that have been open for almost a year.) Are we concerned about only first start, or primarily the faster second starts once all the dynamic libraries have loaded? Does anything actually require an i5? Whats wrong with requiring a GPU? Doesnt it get tiring, being so grumbly about other people having fun & doing cool things? Do you really think we should do as you say & just freeze time, insist on doing nothing?
  - nmz 4y ago
    I care. I care about start up time. I care because I just had a black out, and that requires booting, I care because I mess with my system and I have to reboot sometimes. I care because I expect software to improve, not get worse. I care about engineering because bad engineering ultimately wastes my time and everyone elses. Everyone can have fun and nobody should prohibit others from doing so. but valid criticism is valid criticism, even though I hate it. With that said, the counter argument should've been that although powershell does start up slow, (and many other things) it is better than bash in many cases and more performant.
    - rektide 4y ago
tambourine_man 4y ago
The future of the command line is something along the lines of what these guys are doing: https://www.textualize.io https://www.textualize.io
- ablob 4y ago
  Isn't this still confined to what a terminal gives you right now (as it is using current terminal specifications)? While certainly colorful and fancy, it doesn't really introduce new concepts, does it? I mean, for starters: -> unix introduced text as a universal interface -> bash made reusing stuff a lot easier via file descriptors, etc. (think: <(input to regard as file)) -> powershell allowed for object oriented scripting -> some older systems (name forgotten/unknown) even had interactivity in the cli: click on parts of command-output and stuff happens, even after other commands have been run already
  - tambourine_man 4y ago
    Sure, the terminal primitives are fixed unless we expand the standard. But what they are doing is way beyond colorful only. It’s smooth scrolling of text boxes within text boxes. That alone is bonkers. But they also have an easy API for smart/partial redrawing, sensible UI components, I mean, their progress bars are exquisite. It’s truly impressive.
    - deleted 4y ago
      [deleted]
- dragonelite 4y ago
  Tui example looks really nice would be cool if someone made something like that as a markdown previewer without having to jerryrig it to a browser.
e3bc54b2 4y ago
How about Arcan? [0] It seems very well reasoned, has stable API, excellent backwards compatibility and does not require GPU and i5 as this one might? Its author also has proven record and actual experience, which I'm not sure authors hpf TFA have, judging solely from their writing. 0: https://arcan-fe.com/ https://arcan-fe.com/
- carapace 4y ago
  Um, how do you use it? I've tried like five times and I keep bouncing off the project.
  - e3bc54b2 4y ago
    NixOS has a nice module that can be easily enabled. After that default DE durden can be invoked with single command and can be interacted with as a normal VM/Window alongside your existing DE. Admittedly I haven't tries making it my default yet, but seemed fairly well adaptable for tiling WM/Openbox needs.
frob 4y ago
I'm finding the text block with an overall left-to-right gradient surprisingly hard to read. Continuing from the end of one line to the beginning of the next takes more effort than it should. I'm guessing it's the abrupt change in color. I've found the gradient text trend to be interesting for titles and single lines, but I don't think it works for multi-line text.
oceanplexian 4y ago
> One of its modules attempts to translate natural language requests into the correct shell commands and syntax. For example, if you typed “compress Documents folder,” CLAI will recommend the corresponding Tar command. This is such a bad idea I don’t know where to start. Shell commands are a dangerous, but precise tool, somewhat like using a scalpel or a surgical tool. Dumbing it down so it can “guess what you want it to do” is going to result in more people (Specifically people who don’t bother to read the docs) breaking things.
- mormegil 4y ago
  Exactly. > Warren Teitelman originally wrote DWIM to fix his typos and spelling errors, so it was somewhat idiosyncratic to his style, and would often make hash of anyone else's typos if they were stylistically different. Some victims of DWIM thus claimed that the acronym stood for ‘Damn Warren’s Infernal Machine!'. > In one notorious incident, Warren added a DWIM feature to the command interpreter used at Xerox PARC. One day another hacker there typed delete *$ to free up some disk space. (The editor there named backup files by appending $ to the original file name, so he was trying to delete any backup files left over from old editing sessions.) It happened that there weren't any editor backup files, so DWIM helpfully reported *$ not found, assuming you meant 'delete *'. It then started to delete all the files on the disk! The hacker managed to stop it with a Vulcan nerve pinch after only a half dozen or so files were lost. The Jargon File http://www.catb.org/jargon/html/D/DWIM.html http://www.catb.org/jargon/html/D/DWIM.html
- none_to_remain 4y ago
  What if it tells you to type `man tar`
  - mburee 4y ago
    Then 99% will give up. I mean just try to read the man page of any modern GNU utility, it's so long and filled with options that nobody will ever read that through.
- bhedgeoser 4y ago
  I guess that's why shell programs use regexes and globs.
XorNot 4y ago
Everyone who builds one of these never fixes the fundamental problem: they don't distinguish between "space" as a character, and the next element in the argv array. Stop making me have to engage in bizarre escaping rituals and let me just toggle between "string mode" and "array mode".
gorgoiler 4y ago
I love the future that fzf has given us. So many new ideas for selecting and viewing lines of space delimited records — the bread and butter of the shell — are possible with fzf and you get to build them in the traditional of small composable tools. Junegunn Choi is really talented designer. More please.
deleted 4y ago
[deleted]
none_to_remain 4y ago
Has anyone done anything around just ... mixing images in with the terminal output? Let's say I wanted to check if I had any old memes lying around my home directory, and have a quick look so I can decide to delete or not. ~$ls *{png,jpg} oldmeme.png ~$imgcat oldmeme.png /----------------\ | oldmeme.png | | appears right | | here in the | | terminal | \----------------/ ~$rm oldmeme.png Terminfo man page shows some evidence of support for "bit_image" commands but none of the terminals in my terminfo files seem to have it. I have over 2000 terminfo files though, I like the idea that if I found some literal teletypewriter from 1973 and figured out some way to hook it up, I am probably prepared with the proper escape sequences.
- kretaceous 4y ago
  When I first installed viu[0], I did not think I'd be using it as much as I use it today. PS: I use nnn[1] with the preview plugin which make use cases like yours very easy to solve. 0: https://github.com/atanunq/viu https://github.com/atanunq/viu 1: https://github.com/jarun/nnn https://github.com/jarun/nnn
- kybernetikos 4y ago
  I use wezterm which supports sixel, so I can do exactly what you show here. I use lsix and img2sixel in the terminal. Another fun thing you can try: curl v3.wttr.in/Texas.sxl
- csande17 4y ago
  On macOS, iTerm implements this, and they even provide a command called imgcat that works like you describe: https://iterm2.com/documentation-images.html https://iterm2.com/documentation-images.html Kitty provides a similar but incompatible protocol: https://sw.kovidgoyal.net/kitty/graphics-protocol/ https://sw.kovidgoyal.net/kitty/graphics-protocol/
- kwhitefoot 4y ago
  Terminology can display images and video inline. "This is an EFL terminal emulator with some extra bells and whistles such as the ability to display in-line images, video and even play music files, background images, videos, Z-Modem like sending (e.g. SSH into a server and use tysend to send a file back to the local terminal), GPU Accelerated rendering (optional - just set the EFL Elementary toolkit engine to use OpenGL) and much more." https://github.com/borisfaure/terminology https://github.com/borisfaure/terminology
wodenokoto 4y ago
Is this a blog post _by_ GitHub or something posted by a user _on_ GitHub?
- csande17 4y ago
  The author is employed by GitHub to write/edit for the blog: https://github.com/klintron https://github.com/klintron
o-o- 4y ago
To me this is just more of the same: added convenience to what we already have. Converting individual commands to TUIs is a step away from the unix philosophy. I think the future of the command line lies in the direction of flow-based programming and spatial representation of complex commands. I would like to see a terminal that, as I type, _generates_ a flow-based view of my command. Every command would be visualised as a component: ls, awk, sed... Every |, < or > that I type would append a link and a new component to my flow, and ultimately I would be able to manipulate my flow instead of typing: click the ls component, have it output Creation Date instead of Modified Date, then click the awk component and add another output to a new sed component and so on.
bregma 4y ago
It seems to me the solution to the problem posed in the article -- and to the wishes of many of the commenters here -- is to use JavaScript as your shell. The missing piece is an app that presents a text UI with a JavaScript REPL and that renders the DOM inline. Missing file, device, and information manipulation applications that shell programmers string together would be replaced by JavaScript functions from a library. If you really want JSON, use the JavaScript Serialized Object Notation to serialize JavaScript objects in JavaScript. Why do people write "what I want is..." articles and comments when they could be writing solutions that scratch their itch and meet their needs?

90 points by smartblondeva 4y ago

The command line ought hybridize some, please. Having better self describing interfaces, machine to machine capabilities... humans are awesome enouhh to whip up super wild magic on the fly ("spellcasting on the fly") but these same tools are much weirder to use as you descend into scripting, as you start bringing "real" programming languages in (which have their own alternate realities: "standard libraries"). The command line should/ought bridge & integrate better. Making it more usable from these higher (more pre-baked/automated) levels is one side. And then reciprocally, how wonderful it would be to see execution flow expressed less in terms of stack traces & more in terms of networks of communicating processes. Create boundary layers, make the cli tools visible & known operations sequenced by (but still visible within) higher level systems. Cli on and on!!

jrm4 4y ago
Amen. Offhand, can we start with e.g. a 2 or 3 window deal, in which there is a file manager and a document viewer that one can easily copy and paste from? There's stuff out there that sort of does this but it could be done MUCH better.

an1sotropy 4y ago

One thing not discussed are the libraries used for command-line parsing (parsing argv), and how that might get complicated by shells trying to make the command-line into something effectively more than an array of strings. Having written a non-trivial command-line parser in C, and having used a bunch of them in other languages, it seems to me that this task would benefit from some more standardization and maturation. What is the JSON of the command-line? How can we do to increase the level of interoperability between how information is encoded on different tools' command-lines? e.g. think of ImageMagick "convert" versus "find" versus "ffmpeg": totally different universes, but all of them in their own way turn command-line arguments into mini-DSLs.

scj 4y ago
I've always wondered about expanding stdin, stdout, stderr. Say, stdjson that doesn't get visually displayed, but can be piped (and would only be generated if it is needed on the pipe stream). ls | cat </dev/stdjson | string_proc_the_json_for_some_reason With the direct ability to process in line: ls -a | json.files[0].last_modified I'd probably want multiple output formats (including s-expressions).
- BirAdam 4y ago
  You can always just use JQ. Additionally, as JSON is text, you can use awk/sed/grep and so on.
  - scj 4y ago
    The intent is that standard tools would output alternative formats.
    - an1sotropy 4y ago
      Right; every file is just bytes but we get a lot of mileage out of libraries like libpng that parse those bytes into usefully structured info. And I was pondering what more could evolve to parse info from the command line. I think stdjson is insane yet awesome to ponder.
      - dotancohen 4y ago
        I agree that a standardized structured format would be awesome, but I'm not convinced that JSON is it. And there should _not_ be more than one. One thing that I do like about JSON is that it is ubiquitous - and that makes up for a lot of its other faults. But I would like to see proposed use cases that JSON would not support beforehand, to clearly define where the limitations are, and what limitations the community is willing to accept.
    - nishs 4y ago
      The tool should instead provide an `--output=<format>` flag, where <format> is one of text (default), json, xml, etc.

anigbrowl 4y ago

Very well written article and very impressive magazine generally.

Kukumber 4y ago

"your server needs a GPU and an i5 to use our shell, as it provides a graphical interface and some shader animation because that's what attract the money people, they want shiny stuff y'know" This trend of "new modern shells" that runs and start as slow as some javascript code (powershell) needs to stop People forgot what shells are for, and what scripting is for

yazzku 4y ago
Indeed. These pseudo-authoritative blogs on Github from people who have no idea what they're talking about could also stop.
rektide 4y ago
Doesn't seem like a real problem. Why are we grumbling about this? No one is making rxvt less an option. Why the fear? Why the negativity? Does start-up time matter? Who cares? (I have three terminals that have been open for almost a year.) Are we concerned about only first start, or primarily the faster second starts once all the dynamic libraries have loaded? Does anything actually require an i5? Whats wrong with requiring a GPU? Doesnt it get tiring, being so grumbly about other people having fun & doing cool things? Do you really think we should do as you say & just freeze time, insist on doing nothing?
- nmz 4y ago
  I care. I care about start up time. I care because I just had a black out, and that requires booting, I care because I mess with my system and I have to reboot sometimes. I care because I expect software to improve, not get worse. I care about engineering because bad engineering ultimately wastes my time and everyone elses. Everyone can have fun and nobody should prohibit others from doing so. but valid criticism is valid criticism, even though I hate it. With that said, the counter argument should've been that although powershell does start up slow, (and many other things) it is better than bash in many cases and more performant.
  - rektide 4y ago

tambourine_man 4y ago

The future of the command line is something along the lines of what these guys are doing: https://www.textualize.io https://www.textualize.io

ablob 4y ago
Isn't this still confined to what a terminal gives you right now (as it is using current terminal specifications)? While certainly colorful and fancy, it doesn't really introduce new concepts, does it? I mean, for starters: -> unix introduced text as a universal interface -> bash made reusing stuff a lot easier via file descriptors, etc. (think: <(input to regard as file)) -> powershell allowed for object oriented scripting -> some older systems (name forgotten/unknown) even had interactivity in the cli: click on parts of command-output and stuff happens, even after other commands have been run already
- tambourine_man 4y ago
  Sure, the terminal primitives are fixed unless we expand the standard. But what they are doing is way beyond colorful only. It’s smooth scrolling of text boxes within text boxes. That alone is bonkers. But they also have an easy API for smart/partial redrawing, sensible UI components, I mean, their progress bars are exquisite. It’s truly impressive.
  - deleted 4y ago
    [deleted]
dragonelite 4y ago
Tui example looks really nice would be cool if someone made something like that as a markdown previewer without having to jerryrig it to a browser.

e3bc54b2 4y ago

How about Arcan? [0] It seems very well reasoned, has stable API, excellent backwards compatibility and does not require GPU and i5 as this one might? Its author also has proven record and actual experience, which I'm not sure authors hpf TFA have, judging solely from their writing. 0: https://arcan-fe.com/ https://arcan-fe.com/

carapace 4y ago
Um, how do you use it? I've tried like five times and I keep bouncing off the project.
- e3bc54b2 4y ago
  NixOS has a nice module that can be easily enabled. After that default DE durden can be invoked with single command and can be interacted with as a normal VM/Window alongside your existing DE. Admittedly I haven't tries making it my default yet, but seemed fairly well adaptable for tiling WM/Openbox needs.

frob 4y ago

I'm finding the text block with an overall left-to-right gradient surprisingly hard to read. Continuing from the end of one line to the beginning of the next takes more effort than it should. I'm guessing it's the abrupt change in color. I've found the gradient text trend to be interesting for titles and single lines, but I don't think it works for multi-line text.

oceanplexian 4y ago

> One of its modules attempts to translate natural language requests into the correct shell commands and syntax. For example, if you typed “compress Documents folder,” CLAI will recommend the corresponding Tar command. This is such a bad idea I don’t know where to start. Shell commands are a dangerous, but precise tool, somewhat like using a scalpel or a surgical tool. Dumbing it down so it can “guess what you want it to do” is going to result in more people (Specifically people who don’t bother to read the docs) breaking things.

mormegil 4y ago
Exactly. > Warren Teitelman originally wrote DWIM to fix his typos and spelling errors, so it was somewhat idiosyncratic to his style, and would often make hash of anyone else's typos if they were stylistically different. Some victims of DWIM thus claimed that the acronym stood for ‘Damn Warren’s Infernal Machine!'. > In one notorious incident, Warren added a DWIM feature to the command interpreter used at Xerox PARC. One day another hacker there typed delete *$ to free up some disk space. (The editor there named backup files by appending $ to the original file name, so he was trying to delete any backup files left over from old editing sessions.) It happened that there weren't any editor backup files, so DWIM helpfully reported *$ not found, assuming you meant 'delete *'. It then started to delete all the files on the disk! The hacker managed to stop it with a Vulcan nerve pinch after only a half dozen or so files were lost. The Jargon File http://www.catb.org/jargon/html/D/DWIM.html http://www.catb.org/jargon/html/D/DWIM.html
none_to_remain 4y ago
What if it tells you to type `man tar`
- mburee 4y ago
  Then 99% will give up. I mean just try to read the man page of any modern GNU utility, it's so long and filled with options that nobody will ever read that through.
bhedgeoser 4y ago
I guess that's why shell programs use regexes and globs.

XorNot 4y ago

Everyone who builds one of these never fixes the fundamental problem: they don't distinguish between "space" as a character, and the next element in the argv array. Stop making me have to engage in bizarre escaping rituals and let me just toggle between "string mode" and "array mode".

gorgoiler 4y ago

I love the future that fzf has given us. So many new ideas for selecting and viewing lines of space delimited records — the bread and butter of the shell — are possible with fzf and you get to build them in the traditional of small composable tools. Junegunn Choi is really talented designer. More please.

deleted 4y ago

[deleted]

none_to_remain 4y ago

Has anyone done anything around just ... mixing images in with the terminal output? Let's say I wanted to check if I had any old memes lying around my home directory, and have a quick look so I can decide to delete or not. ~$ls *{png,jpg} oldmeme.png ~$imgcat oldmeme.png /----------------\ | oldmeme.png | | appears right | | here in the | | terminal | \----------------/ ~$rm oldmeme.png Terminfo man page shows some evidence of support for "bit_image" commands but none of the terminals in my terminfo files seem to have it. I have over 2000 terminfo files though, I like the idea that if I found some literal teletypewriter from 1973 and figured out some way to hook it up, I am probably prepared with the proper escape sequences.

kretaceous 4y ago
When I first installed viu[0], I did not think I'd be using it as much as I use it today. PS: I use nnn[1] with the preview plugin which make use cases like yours very easy to solve. 0: https://github.com/atanunq/viu https://github.com/atanunq/viu 1: https://github.com/jarun/nnn https://github.com/jarun/nnn
kybernetikos 4y ago
I use wezterm which supports sixel, so I can do exactly what you show here. I use lsix and img2sixel in the terminal. Another fun thing you can try: curl v3.wttr.in/Texas.sxl
csande17 4y ago
On macOS, iTerm implements this, and they even provide a command called imgcat that works like you describe: https://iterm2.com/documentation-images.html https://iterm2.com/documentation-images.html Kitty provides a similar but incompatible protocol: https://sw.kovidgoyal.net/kitty/graphics-protocol/ https://sw.kovidgoyal.net/kitty/graphics-protocol/
kwhitefoot 4y ago
Terminology can display images and video inline. "This is an EFL terminal emulator with some extra bells and whistles such as the ability to display in-line images, video and even play music files, background images, videos, Z-Modem like sending (e.g. SSH into a server and use tysend to send a file back to the local terminal), GPU Accelerated rendering (optional - just set the EFL Elementary toolkit engine to use OpenGL) and much more." https://github.com/borisfaure/terminology https://github.com/borisfaure/terminology

wodenokoto 4y ago

Is this a blog post _by_ GitHub or something posted by a user _on_ GitHub?

csande17 4y ago
The author is employed by GitHub to write/edit for the blog: https://github.com/klintron https://github.com/klintron

o-o- 4y ago

To me this is just more of the same: added convenience to what we already have. Converting individual commands to TUIs is a step away from the unix philosophy. I think the future of the command line lies in the direction of flow-based programming and spatial representation of complex commands. I would like to see a terminal that, as I type, _generates_ a flow-based view of my command. Every command would be visualised as a component: ls, awk, sed... Every |, < or > that I type would append a link and a new component to my flow, and ultimately I would be able to manipulate my flow instead of typing: click the ls component, have it output Creation Date instead of Modified Date, then click the awk component and add another output to a new sed component and so on.

bregma 4y ago

It seems to me the solution to the problem posed in the article -- and to the wishes of many of the commenters here -- is to use JavaScript as your shell. The missing piece is an app that presents a text UI with a JavaScript REPL and that renders the DOM inline. Missing file, device, and information manipulation applications that shell programmers string together would be replaced by JavaScript functions from a library. If you really want JSON, use the JavaScript Serialized Object Notation to serialize JavaScript objects in JavaScript. Why do people write "what I want is..." articles and comments when they could be writing solutions that scratch their itch and meet their needs?