5 ms·

I was at Bell during the options “debate.” I think something that this otherwise wonderful article misses is that some believed that commands were never intende

I was at Bell during the options “debate.” I think something that this otherwise wonderful article misses is that some believed that commands were never intended to be the only way to use the system as the shell was intended to be just one of the many user interfaces that Research Unix would provide. From that perspective, it was entirely reasonable to believe that if a command was so complex that it needed options than it was likely more appropriate for one of the other user interfaces. I believe this is unfortunately now understood by some as it shouldn’t exist. However, because of the seemingly instantaneous popularity of the shell and pipelining text it became synonymous with Unix. It’s a shame because McIlroy had a lot of interesting ideas around pipelining audio and graphics that, as far as I know, never materialized.

Tijdreiziger 4y ago
Interesting, any chance you could expand on these 'other user interfaces'? I'm not really familiar with Unix itself, but I've always considered Linux a shell-first OS (as opposed to Windows (NT), which I consider a GUI-first OS).
- imwillofficial 4y ago
  A shell is just how you interact with the underlying system. The gui is also a shell. Confusing I know!
  - daptaq 4y ago
    I like to see it this way: A shell /wraps/ the kernel. You cannot issue system calls directly, but a program that handles user input generically (and ideally dynamically), can do this for you. A desktop environment, Emacs, and to an increasing degree web browsers are all different "shells".
    - imwillofficial 4y ago
      Like valence shells on atoms. I dig it
    - tenebrisalietum 4y ago
      A shell is a program dedicated to allowing an operator to launch other programs. It can be as simple as a menu or as complex as a COM-interfaceable GUI with graphical desktop metaphor. It's often configured to but not strictly required to automatically launch on user login. Any user-space executable can issue system calls, the shell isn't special there.
  - edgyquant 4y ago
    E.g. the Windows GUI is the “Windows Shell,” not be confused with the Windows Command Line which is one (well, many) application within the shell.
- bfrankline 4y ago
  The other environment that is still popular today is the “statistical environment” that Rick Becker, Allan Wilks, and John Chambers created. It eventually became “S” and John would essentially recreate it as “R.” It’s a very nice environment for performing statistics and graphics together.
- lousyd 4y ago
  Maybe something like Jupyter? Text documents with sections of code or commands?
MontyCarloHall 4y ago
>From that perspective, it was entirely reasonable to believe that if a command was so complex that it needed options than it was likely more appropriate for one of the other user interfaces What would the non-shell interface to commands for text processing pipelining (e.g. sort, cut, grep, etc., all of which absolutely need options to function) have looked like? Some people to this day believe that any text processing more complicated than a simple grep or cut should be done in a stand-alone script written in a domain-specific language (e.g. Perl or awk), rather than piping shell commands to each other. Personally, I’m glad we got the best of both worlds—having command line tools with a dizzying array of options doesn’t preclude being able to accomplish the same tasks with a more verbose stand-alone script. It is often far faster to write shell pipelines to accomplish fairly involved tasks than to write a stand-alone script. The classic example is the (in)famous McIlroy vs. Knuth: six lines of McIlroy’s shell pipeline accomplished the same thing as dozens of lines of Knuth’s “literate program,” while being equally as understandable [0]. >It’s a shame because McIlroy had a lot of interesting ideas around pipelining audio and graphics that, as far as I know, never materialized. I would love to hear more about this. The UNIX shell is amazing for pipelining (most) things that are easily represented as text but really falls flat for pipelining everything else. [0] https://leancrew.com/all-this/2011/12/more-shell-less-egg/ https://leancrew.com/all-this/2011/12/more-shell-less-egg/
- svat 4y ago
  > six lines of McIlroy’s shell pipeline accomplished the same thing as Common misconception. See https://buttondown.email/hillelwayne/archive/donald-knuth-was-framed/ https://buttondown.email/hillelwayne/archive/donald-knuth-wa... ("Donald Knuth Was Framed") and discussion at https://news.ycombinator.com/item?id=22406070 https://news.ycombinator.com/item?id=22406070 (etc).
  - MontyCarloHall 4y ago
    How is it a misconception? My overall point was that shell oneliners are often much faster to quickly bang out for a one-off use case than writing a full program from the ground up to accomplish the same thing. This is demonstrated to a very exaggerated degree in the Knuth vs. McIlroy example, but it also holds true for non-exaggerated real-world use cases. (I had a coworker who was totally shell illiterate and would write a Python script every time they had to do a simple task like count the number of unique words in a file. This took at least 10 times longer than someone proficient at the shell, which one could argue is itself a highly optimized domain-specific language for text processing.) If your point is that the shell script isn't really the same thing as Knuth's program: sure, the approaches weren't algorithmically identical (assuming constant time insertions on average, Knuth's custom trie yields an O(N) solution, which is faster than McIlroy's O(N*log N) sort, though this point is moot if you use awk's hashtables to tally words rather than `sort | uniq -c`), but both approaches accomplish the exact same end result, and both fail to handle the exact same edge cases (e.g. accent marks).
tambourine_man 4y ago
Very interesting. We’d love to read more on other media pipelining and alternatives to the shell.
- harry8 4y ago
  Yeah. I wonder if it looks anything like gstreamer...
- bfrankline 4y ago
  It was very primitive. It was essentially a mechanism for composing operations on vectors. I don’t know for certain but I would guess that it was inspired by IBM and their work on APL.
jancsika 4y ago
> It’s a shame because McIlroy had a lot of interesting ideas around pipelining audio and graphics that, as far as I know, never materialized. Do you know if those ideas are documented anywhere?
- bfrankline 4y ago
  The most popular today outside of the shell environment is the statistical environment “S.” John Chambers would recreate it as “R” and I understand that it’s very popular and does a nice job of performing statistics and graphics together.
nmz 4y ago
FWIW, you can pipe graphics (sort of) with arcan[1]. So its not like the ideas have been abandoned. 1. https://arcan-fe.com/ https://arcan-fe.com/
AlphaSite 4y ago
I imagine powershell could do this quite well with it’s typed output. (Not that you can’t just base raw bytes via unix shells but it’s awfully error prone).

Tijdreiziger 4y ago

Interesting, any chance you could expand on these 'other user interfaces'? I'm not really familiar with Unix itself, but I've always considered Linux a shell-first OS (as opposed to Windows (NT), which I consider a GUI-first OS).

imwillofficial 4y ago
A shell is just how you interact with the underlying system. The gui is also a shell. Confusing I know!
- daptaq 4y ago
  I like to see it this way: A shell /wraps/ the kernel. You cannot issue system calls directly, but a program that handles user input generically (and ideally dynamically), can do this for you. A desktop environment, Emacs, and to an increasing degree web browsers are all different "shells".
  - imwillofficial 4y ago
    Like valence shells on atoms. I dig it
  - tenebrisalietum 4y ago
    A shell is a program dedicated to allowing an operator to launch other programs. It can be as simple as a menu or as complex as a COM-interfaceable GUI with graphical desktop metaphor. It's often configured to but not strictly required to automatically launch on user login. Any user-space executable can issue system calls, the shell isn't special there.
- edgyquant 4y ago
  E.g. the Windows GUI is the “Windows Shell,” not be confused with the Windows Command Line which is one (well, many) application within the shell.
bfrankline 4y ago
The other environment that is still popular today is the “statistical environment” that Rick Becker, Allan Wilks, and John Chambers created. It eventually became “S” and John would essentially recreate it as “R.” It’s a very nice environment for performing statistics and graphics together.
lousyd 4y ago
Maybe something like Jupyter? Text documents with sections of code or commands?

MontyCarloHall 4y ago

>From that perspective, it was entirely reasonable to believe that if a command was so complex that it needed options than it was likely more appropriate for one of the other user interfaces What would the non-shell interface to commands for text processing pipelining (e.g. sort, cut, grep, etc., all of which absolutely need options to function) have looked like? Some people to this day believe that any text processing more complicated than a simple grep or cut should be done in a stand-alone script written in a domain-specific language (e.g. Perl or awk), rather than piping shell commands to each other. Personally, I’m glad we got the best of both worlds—having command line tools with a dizzying array of options doesn’t preclude being able to accomplish the same tasks with a more verbose stand-alone script. It is often far faster to write shell pipelines to accomplish fairly involved tasks than to write a stand-alone script. The classic example is the (in)famous McIlroy vs. Knuth: six lines of McIlroy’s shell pipeline accomplished the same thing as dozens of lines of Knuth’s “literate program,” while being equally as understandable [0]. >It’s a shame because McIlroy had a lot of interesting ideas around pipelining audio and graphics that, as far as I know, never materialized. I would love to hear more about this. The UNIX shell is amazing for pipelining (most) things that are easily represented as text but really falls flat for pipelining everything else. [0] https://leancrew.com/all-this/2011/12/more-shell-less-egg/ https://leancrew.com/all-this/2011/12/more-shell-less-egg/

svat 4y ago
> six lines of McIlroy’s shell pipeline accomplished the same thing as Common misconception. See https://buttondown.email/hillelwayne/archive/donald-knuth-was-framed/ https://buttondown.email/hillelwayne/archive/donald-knuth-wa... ("Donald Knuth Was Framed") and discussion at https://news.ycombinator.com/item?id=22406070 https://news.ycombinator.com/item?id=22406070 (etc).
- MontyCarloHall 4y ago
  How is it a misconception? My overall point was that shell oneliners are often much faster to quickly bang out for a one-off use case than writing a full program from the ground up to accomplish the same thing. This is demonstrated to a very exaggerated degree in the Knuth vs. McIlroy example, but it also holds true for non-exaggerated real-world use cases. (I had a coworker who was totally shell illiterate and would write a Python script every time they had to do a simple task like count the number of unique words in a file. This took at least 10 times longer than someone proficient at the shell, which one could argue is itself a highly optimized domain-specific language for text processing.) If your point is that the shell script isn't really the same thing as Knuth's program: sure, the approaches weren't algorithmically identical (assuming constant time insertions on average, Knuth's custom trie yields an O(N) solution, which is faster than McIlroy's O(N*log N) sort, though this point is moot if you use awk's hashtables to tally words rather than `sort | uniq -c`), but both approaches accomplish the exact same end result, and both fail to handle the exact same edge cases (e.g. accent marks).

tambourine_man 4y ago

Very interesting. We’d love to read more on other media pipelining and alternatives to the shell.

harry8 4y ago
Yeah. I wonder if it looks anything like gstreamer...
bfrankline 4y ago
It was very primitive. It was essentially a mechanism for composing operations on vectors. I don’t know for certain but I would guess that it was inspired by IBM and their work on APL.

jancsika 4y ago

> It’s a shame because McIlroy had a lot of interesting ideas around pipelining audio and graphics that, as far as I know, never materialized. Do you know if those ideas are documented anywhere?

bfrankline 4y ago
The most popular today outside of the shell environment is the statistical environment “S.” John Chambers would recreate it as “R” and I understand that it’s very popular and does a nice job of performing statistics and graphics together.

nmz 4y ago

FWIW, you can pipe graphics (sort of) with arcan[1]. So its not like the ideas have been abandoned. 1. https://arcan-fe.com/ https://arcan-fe.com/

AlphaSite 4y ago

I imagine powershell could do this quite well with it’s typed output. (Not that you can’t just base raw bytes via unix shells but it’s awfully error prone).