6 ms·

> In the talk, he gives several demonstrations a key aspect of why unix pipelines are so practically useful: you build them interactively. The standard Unix in

> In the talk, he gives several demonstrations a key aspect of why unix pipelines are so practically useful: you build them interactively. The standard Unix interface might have been interactive in the ’70s, back when hardware and peripherals were horribly non-interactive. But I don’t know why so many so-called millenial programmers (people my age) get excited about the alleged interactivity of the Unix that most people are familiar with. It doesn’t even have the cutting edge ’90s interactivity of Plan 9, what with mouse(!) selection of arbitrary text that can be piped to commands and so on. And every time someone comes up with a Unix-hosted tool that uses some kind of fold-up menu that informs you about what key combination you can type next (you know, like what all GUI programs have with Alt+x and the file|edit|view|… toolbar), people hail it as some kind of UX innovation.

SomethingOrNot 8y ago
One interactive feature I like about Bash though (or shell or whatever) is C-x C-e to edit the current line in a text editor (FC). Great when I know how to transform some text in the editor easily but not on the command line.
jraph 8y ago
I think the interactivity you describe might be a different thing from what your parent is talking about. From what I understand, your parent talks about how the commands are built iteratively, with some kind of trial-error loop, which is a strength that is supposedly not emphasized enough. And I agree by the way. Nothing to do with how things are input.
- pdkl95 8y ago
  That's correct. Articles/tutorials or an evangelizing fan often show the end result: the cool command/pipeline that does something cool and useful. The obvious question when someone unfamiliar with unix upon seeing something like the pipeline in this article: comm -1 -3 <(ls -1 dataset-directory | \ grep '\d\d\d\d_A.csv' | \ cut -c 1-4 | \ python3 parse.py | \ uniq \ ) \ <(seq 500) is "Why would I want to write a complicated mess like that?" Just use ${FAVORITE_PROG_LANG:-Perl, Ruby, or whatever}". For many tasks, a short paragraph of code in a "normal" programming language is probably easier to write and is almost certainly a more robust, easier to maintain solution. However, this assumes that you knew what the problem was and that qualities like maintainability are a goal. Bernhardt's (and my) point is that sometimes you don't know what the goal is yet. Sometimes you just need to do a small, one-off task where a half-assed solutions might be appropriate... iff it's the right half of the ass. Unix shell gets that right for a really useful set of tasks. This works because you are free to utilize that powerful features incrementally, as needed. The interactive nature of the shell lets you explore the problem. The "better" version in a "proper" programming language doesn't exist when you don't yet know the exact nature of the problem. A half-assed bit of shell code that slowly evolved into something useful might be the step between "I have some data" and a larger "real" programming project. That said, there is also wisdom in learning to recognize when your needs have outgrown "small, half-assed" solutions. If the project is growing and adding layers of complexity, it's probably time to switch to a more appropriate tool.
  - SomethingOrNot 8y ago
    I generalized interactivity to the Unix that most people seem familiar with. “The interactive nature of the shell” isn’t that impressive in this day and age. Certainly not shells like Bash (Fish is probably better, but then again that’s very cutting edge shell (“for the ’90s”)). Irrespective of the shell this just boils down to executing code, editing text, executing code, repeat. I suspect people started doing that once they got updating displays, if not sooner.
    - TeMPOraL 8y ago
      How is that not impressive for vast majority of developers? For the past couple decades, the only other even remotely mainstream place where you could get a comparable experience was a Lisp REPL. And maaaybe Matlab, later on. Recently, projects like R, Jupyer, and (AFAIK) Julia have been introducing people to interactive development, but those are specific to scientific computing. For general programming, this approach is pretty much unknown outside of Lisp and Unix shell worlds.
      - fwip 8y ago
        The alternative is to write a program or script that does part of the job, run it (compiling if necessary), and see what happens. Then modify. This loop is definitely slower than the shell or other REPL though.
        TeMPOraL 8y ago
        It's much slower, and doesn't lend itself as well for building the program up from small, independently tested and refined pieces. The speed of that feedback loop really matters - the slower it is, the larger chunks you'll be writing before testing. I currently believe the popularity of TDD is primarily a symptom of not having a decent REPL (though REPL doesn't replace unit tests, especially in terms of regression testing). BTW. there's another nice feature of Lisp-style interactive development - you're mutating a living program. You can change the data or define and redefine functions and classes as the program is executing them, without pausing the program. The other end of your REPL essentially becomes a small OS. This matters less when you're building a terminal utility, but it's useful for server and GUI software, and leads to wonders like this: https://www.youtube.com/watch?v=gj5IzggEWKE https://www.youtube.com/watch?v=gj5IzggEWKE It's a different way of approaching programming, and I encourage everyone to try it out.
- TuringTest 8y ago
  > I think the interactivity you describe might be a different thing from what your parent is talking about. Actually no, they're not different things; both refer to the same activity of a user analyzing the information on the screen and issuing commands that refine the available information iteratively, in order to solve a problem. (I would have bought your argument had you made a distinction between "solving the problem" and "finding the right tools to solve the problem"). The thing is that the Unix shell is terribly coarse-gained in terms of what interactivity is allowed, so that the smaller refinement actions (what you call "input") must be described in terms of a formal programming language, instead of having interactive tools for those smaller trial-error steps. There are some very limited forms of interactivity (command line history, keyboard accelerators, "man" and "-h" help), but the kind of direct manipulation that would allow the user to select commands and data iteratively, are mostly absent from the Unix shell. Emacs is way better in that sense, except for the terrible discoverability of options (based on recall over recognition).
  - SomethingOrNot 8y ago
    One of the dead ends of Unix UX are all the terse DSLs. I feel that terse languages like Vi’s command language [1] get confused with interactivity. It sure can be terse, but having dozens of tiny languages with little coherence is not interactive; it’s just confusing and error-prone. One of these languages is the history expansion in Bash. At first I was taken by all the `!!^1` weirdness. But (of course) it’s better—and actually interactive—to use keybindings like `up` (previous command). Thankfully Fish had the good sense to not implement history expansion. [1] I use Emacs+Evil so I like Vi(m) myself.
  - pdkl95 8y ago
    > select commands and data iteratively ... Emacs is way better in that sense Bind up/down to history-search-backward/history-search-forward. In ~/.inputrc # your terminal might send something else for the # for the up/down keys; check with ^v<key> # UP "\e[A": history-search-backward # DOWN "\e[B": history-search-forward (note that this affects anything that uses readline, not just bash) The default (previous-history/next-history) only step through history one item at a time. The history-search- commands step through only the history entries that match the prefix you have already typed. (i.e. typing "cp<UP>" gets the last "cp ..." command; continuing to press <UP> steps through all of the "cp ..." commands in ${HISTFILE}). As your history file grows, this ends up kind of like smex[1] (ido-mode for M-x that prefers recently and most frequently used commands). For maximum effect, you might want to also significantly increase the size of the saved history: # no file size limit HISTFILESIZE="-1" # runtime limit of commands in history. default is 500! HISTSIZE="1000000" # ignoredups to make the searching more efficient HISTCONTROL="ignorespace:ignoredups" # (and make sure HISTFILE is set to something sane) [1] https://github.com/nonsequitur/smex/ https://github.com/nonsequitur/smex/
smittywerben 8y ago
I’m a lone dev that works with moderate-size data and whatever UX solution you’re thinking of is slow or doesn’t exist. Yes Bash is an untyped hell. But when I pipe 100 GB to stout, my computer’s death wish to show me the fucking data.
mnarayan01 8y ago
> mouse(!) selection of arbitrary text that can be piped to commands and so on E.g.: xclip -out | ... or do you mean something different?
- SomethingOrNot 8y ago
  Essentially selecting and operating on text in the same buffer. Saw it in Russ Cox’s demonstration of Acme.
  - pjmlp 8y ago
    The idea was taken from Oberon, which got inspired by Mesa/Cedar at Xerox PARC. Basically any function/procedure that gets exported from a module can be used either on the REPL, or from them mouse, depending on its signature. Quite powerful concept for those that like to take advantage of GUI based OSes. Powershell is the only that comes close to it. Maybe fish as well, but never used it.

SomethingOrNot 8y ago

One interactive feature I like about Bash though (or shell or whatever) is C-x C-e to edit the current line in a text editor (FC). Great when I know how to transform some text in the editor easily but not on the command line.

jraph 8y ago

I think the interactivity you describe might be a different thing from what your parent is talking about. From what I understand, your parent talks about how the commands are built iteratively, with some kind of trial-error loop, which is a strength that is supposedly not emphasized enough. And I agree by the way. Nothing to do with how things are input.

pdkl95 8y ago
That's correct. Articles/tutorials or an evangelizing fan often show the end result: the cool command/pipeline that does something cool and useful. The obvious question when someone unfamiliar with unix upon seeing something like the pipeline in this article: comm -1 -3 <(ls -1 dataset-directory | \ grep '\d\d\d\d_A.csv' | \ cut -c 1-4 | \ python3 parse.py | \ uniq \ ) \ <(seq 500) is "Why would I want to write a complicated mess like that?" Just use ${FAVORITE_PROG_LANG:-Perl, Ruby, or whatever}". For many tasks, a short paragraph of code in a "normal" programming language is probably easier to write and is almost certainly a more robust, easier to maintain solution. However, this assumes that you knew what the problem was and that qualities like maintainability are a goal. Bernhardt's (and my) point is that sometimes you don't know what the goal is yet. Sometimes you just need to do a small, one-off task where a half-assed solutions might be appropriate... iff it's the right half of the ass. Unix shell gets that right for a really useful set of tasks. This works because you are free to utilize that powerful features incrementally, as needed. The interactive nature of the shell lets you explore the problem. The "better" version in a "proper" programming language doesn't exist when you don't yet know the exact nature of the problem. A half-assed bit of shell code that slowly evolved into something useful might be the step between "I have some data" and a larger "real" programming project. That said, there is also wisdom in learning to recognize when your needs have outgrown "small, half-assed" solutions. If the project is growing and adding layers of complexity, it's probably time to switch to a more appropriate tool.
- SomethingOrNot 8y ago
  I generalized interactivity to the Unix that most people seem familiar with. “The interactive nature of the shell” isn’t that impressive in this day and age. Certainly not shells like Bash (Fish is probably better, but then again that’s very cutting edge shell (“for the ’90s”)). Irrespective of the shell this just boils down to executing code, editing text, executing code, repeat. I suspect people started doing that once they got updating displays, if not sooner.
  - TeMPOraL 8y ago
    How is that not impressive for vast majority of developers? For the past couple decades, the only other even remotely mainstream place where you could get a comparable experience was a Lisp REPL. And maaaybe Matlab, later on. Recently, projects like R, Jupyer, and (AFAIK) Julia have been introducing people to interactive development, but those are specific to scientific computing. For general programming, this approach is pretty much unknown outside of Lisp and Unix shell worlds.
    - fwip 8y ago
      The alternative is to write a program or script that does part of the job, run it (compiling if necessary), and see what happens. Then modify. This loop is definitely slower than the shell or other REPL though.
      - TeMPOraL 8y ago
        It's much slower, and doesn't lend itself as well for building the program up from small, independently tested and refined pieces. The speed of that feedback loop really matters - the slower it is, the larger chunks you'll be writing before testing. I currently believe the popularity of TDD is primarily a symptom of not having a decent REPL (though REPL doesn't replace unit tests, especially in terms of regression testing). BTW. there's another nice feature of Lisp-style interactive development - you're mutating a living program. You can change the data or define and redefine functions and classes as the program is executing them, without pausing the program. The other end of your REPL essentially becomes a small OS. This matters less when you're building a terminal utility, but it's useful for server and GUI software, and leads to wonders like this: https://www.youtube.com/watch?v=gj5IzggEWKE https://www.youtube.com/watch?v=gj5IzggEWKE It's a different way of approaching programming, and I encourage everyone to try it out.
TuringTest 8y ago
> I think the interactivity you describe might be a different thing from what your parent is talking about. Actually no, they're not different things; both refer to the same activity of a user analyzing the information on the screen and issuing commands that refine the available information iteratively, in order to solve a problem. (I would have bought your argument had you made a distinction between "solving the problem" and "finding the right tools to solve the problem"). The thing is that the Unix shell is terribly coarse-gained in terms of what interactivity is allowed, so that the smaller refinement actions (what you call "input") must be described in terms of a formal programming language, instead of having interactive tools for those smaller trial-error steps. There are some very limited forms of interactivity (command line history, keyboard accelerators, "man" and "-h" help), but the kind of direct manipulation that would allow the user to select commands and data iteratively, are mostly absent from the Unix shell. Emacs is way better in that sense, except for the terrible discoverability of options (based on recall over recognition).
- SomethingOrNot 8y ago
  One of the dead ends of Unix UX are all the terse DSLs. I feel that terse languages like Vi’s command language [1] get confused with interactivity. It sure can be terse, but having dozens of tiny languages with little coherence is not interactive; it’s just confusing and error-prone. One of these languages is the history expansion in Bash. At first I was taken by all the `!!^1` weirdness. But (of course) it’s better—and actually interactive—to use keybindings like `up` (previous command). Thankfully Fish had the good sense to not implement history expansion. [1] I use Emacs+Evil so I like Vi(m) myself.
- pdkl95 8y ago
  > select commands and data iteratively ... Emacs is way better in that sense Bind up/down to history-search-backward/history-search-forward. In ~/.inputrc # your terminal might send something else for the # for the up/down keys; check with ^v<key> # UP "\e[A": history-search-backward # DOWN "\e[B": history-search-forward (note that this affects anything that uses readline, not just bash) The default (previous-history/next-history) only step through history one item at a time. The history-search- commands step through only the history entries that match the prefix you have already typed. (i.e. typing "cp<UP>" gets the last "cp ..." command; continuing to press <UP> steps through all of the "cp ..." commands in ${HISTFILE}). As your history file grows, this ends up kind of like smex[1] (ido-mode for M-x that prefers recently and most frequently used commands). For maximum effect, you might want to also significantly increase the size of the saved history: # no file size limit HISTFILESIZE="-1" # runtime limit of commands in history. default is 500! HISTSIZE="1000000" # ignoredups to make the searching more efficient HISTCONTROL="ignorespace:ignoredups" # (and make sure HISTFILE is set to something sane) [1] https://github.com/nonsequitur/smex/ https://github.com/nonsequitur/smex/

smittywerben 8y ago

I’m a lone dev that works with moderate-size data and whatever UX solution you’re thinking of is slow or doesn’t exist. Yes Bash is an untyped hell. But when I pipe 100 GB to stout, my computer’s death wish to show me the fucking data.

mnarayan01 8y ago

> mouse(!) selection of arbitrary text that can be piped to commands and so on E.g.: xclip -out | ... or do you mean something different?

SomethingOrNot 8y ago
Essentially selecting and operating on text in the same buffer. Saw it in Russ Cox’s demonstration of Acme.
- pjmlp 8y ago
  The idea was taken from Oberon, which got inspired by Mesa/Cedar at Xerox PARC. Basically any function/procedure that gets exported from a module can be used either on the REPL, or from them mouse, depending on its signature. Quite powerful concept for those that like to take advantage of GUI based OSes. Powershell is the only that comes close to it. Maybe fish as well, but never used it.