9 ms·

Gary Bernhardt[1] gave a great talk about practical problem solving with the unix shell: "The Unix Chainsaw"[2]. "Half-assed is OK when you only need half of a

by pdkl95 8y ago

Gary Bernhardt[1] gave a great talk about practical problem solving with the unix shell: "The Unix Chainsaw"[2]. "Half-assed is OK when you only need half of an ass." In the talk, he gives several demonstrations a key aspect of why unix pipelines are so practically useful: you build them interactively. A complicated 4 line pipeline started as a single command that was gradually refined into something that actually solves a complicated problem. This talk demonstrates the part that isn't included in the the usual tutorials or "cool 1-line command" lists: the cycle of "Try something. Hit up to get the command back. Make one iterative change and try again." [1] You might know him from his other hilarious talks like "The Birth & Death of JavaScript" or "Wat". [2] https://www.youtube.com/watch?v=sCZJblyT_XM https://www.youtube.com/watch?v=sCZJblyT_XM

SomethingOrNot 8y ago
> In the talk, he gives several demonstrations a key aspect of why unix pipelines are so practically useful: you build them interactively. The standard Unix interface might have been interactive in the ’70s, back when hardware and peripherals were horribly non-interactive. But I don’t know why so many so-called millenial programmers (people my age) get excited about the alleged interactivity of the Unix that most people are familiar with. It doesn’t even have the cutting edge ’90s interactivity of Plan 9, what with mouse(!) selection of arbitrary text that can be piped to commands and so on. And every time someone comes up with a Unix-hosted tool that uses some kind of fold-up menu that informs you about what key combination you can type next (you know, like what all GUI programs have with Alt+x and the file|edit|view|… toolbar), people hail it as some kind of UX innovation.
- SomethingOrNot 8y ago
  One interactive feature I like about Bash though (or shell or whatever) is C-x C-e to edit the current line in a text editor (FC). Great when I know how to transform some text in the editor easily but not on the command line.
- jraph 8y ago
  I think the interactivity you describe might be a different thing from what your parent is talking about. From what I understand, your parent talks about how the commands are built iteratively, with some kind of trial-error loop, which is a strength that is supposedly not emphasized enough. And I agree by the way. Nothing to do with how things are input.
  - pdkl95 8y ago
    That's correct. Articles/tutorials or an evangelizing fan often show the end result: the cool command/pipeline that does something cool and useful. The obvious question when someone unfamiliar with unix upon seeing something like the pipeline in this article: comm -1 -3 <(ls -1 dataset-directory | \ grep '\d\d\d\d_A.csv' | \ cut -c 1-4 | \ python3 parse.py | \ uniq \ ) \ <(seq 500) is "Why would I want to write a complicated mess like that?" Just use ${FAVORITE_PROG_LANG:-Perl, Ruby, or whatever}". For many tasks, a short paragraph of code in a "normal" programming language is probably easier to write and is almost certainly a more robust, easier to maintain solution. However, this assumes that you knew what the problem was and that qualities like maintainability are a goal. Bernhardt's (and my) point is that sometimes you don't know what the goal is yet. Sometimes you just need to do a small, one-off task where a half-assed solutions might be appropriate... iff it's the right half of the ass. Unix shell gets that right for a really useful set of tasks. This works because you are free to utilize that powerful features incrementally, as needed. The interactive nature of the shell lets you explore the problem. The "better" version in a "proper" programming language doesn't exist when you don't yet know the exact nature of the problem. A half-assed bit of shell code that slowly evolved into something useful might be the step between "I have some data" and a larger "real" programming project. That said, there is also wisdom in learning to recognize when your needs have outgrown "small, half-assed" solutions. If the project is growing and adding layers of complexity, it's probably time to switch to a more appropriate tool.
    - SomethingOrNot 8y ago
      I generalized interactivity to the Unix that most people seem familiar with. “The interactive nature of the shell” isn’t that impressive in this day and age. Certainly not shells like Bash (Fish is probably better, but then again that’s very cutting edge shell (“for the ’90s”)). Irrespective of the shell this just boils down to executing code, editing text, executing code, repeat. I suspect people started doing that once they got updating displays, if not sooner.
- smittywerben 8y ago
  I’m a lone dev that works with moderate-size data and whatever UX solution you’re thinking of is slow or doesn’t exist. Yes Bash is an untyped hell. But when I pipe 100 GB to stout, my computer’s death wish to show me the fucking data.
- mnarayan01 8y ago
  > mouse(!) selection of arbitrary text that can be piped to commands and so on E.g.: xclip -out | ... or do you mean something different?
  - SomethingOrNot 8y ago
    Essentially selecting and operating on text in the same buffer. Saw it in Russ Cox’s demonstration of Acme.
    - pjmlp 8y ago
      The idea was taken from Oberon, which got inspired by Mesa/Cedar at Xerox PARC. Basically any function/procedure that gets exported from a module can be used either on the REPL, or from them mouse, depending on its signature. Quite powerful concept for those that like to take advantage of GUI based OSes. Powershell is the only that comes close to it. Maybe fish as well, but never used it.
mercer 8y ago
This is a reason why I like languages that have REPLs, and perhaps an advantage of dynamically typed, and especially functional languages that I feel is often not emphasized enough (judging by the recent discussions on the issue). I love how I can just open up IEx (Elixir) and work my way through various functions in my app interactively. If something doesn't work as expected, I change the code and run the 'recompile' command. Then, once things work and eventually stabilize, I add typespecs for at least some of the benefits of an explicitly typed language. In some cases I might make (unit) tests part of the process, but that depends on the situation.
yesenadam 8y ago
AWK makes most of that grep/cut/count-type stuff so much easier, 1-liners need something like 3 parts instead of 10. (My grep doesn't work like in the video, but..) for f in *.rb;do awk '$1 ~ /class|module/ {print $2}' $f;done | awk '{a[$1]++} END{for (q in a) print a[q],q}' | sort -n This seems to do what all the sed, cut, regrepping, wc etc do. AWK (in default mode, anyway) removes leading spaces, makes cutting and counting words easy. It took about 30 seconds to write, too.
- voltagex_ 8y ago
  I have never been able to get my head around awk - how did you learn it?
  - FelipeCortez 8y ago
    The AWK programming language book is excellent! https://archive.org/download/pdfy-MgN0H1joIoDVoIC7/The_AWK_Programming_Language.pdf https://archive.org/download/pdfy-MgN0H1joIoDVoIC7/The_AWK_P... You can find some discussion about it here: https://news.ycombinator.com/item?id=13451454 https://news.ycombinator.com/item?id=13451454
    - voltagex_ 8y ago
      >This book was typeset in Times Roman and Courier by the authors, using an Autologic APS-5 phototypesetter and a DEC VAX 8550 running the 9th Edition of the UNIX operating system. Delightful!
  - patrec 8y ago
    "The AWK programming language" has already been suggested and is great. But awk is actually pretty simple. Imagine everything having an implicit loop like this around it like so: # BEGIN{ ... } would go here for $0 in input.split('\n'): $1, $2, ... = line.split() # if TEST {ACTION}; pairs ... # END { ... } would go here so the first TEST in GP's awk would be `$1 ~ /class|module/` and the corresponding ACTION to take `{print $2}`. if TEST is omitted it's True, if ACTION is it's print the whole line.

by pdkl95 8y ago

SomethingOrNot 8y ago

> In the talk, he gives several demonstrations a key aspect of why unix pipelines are so practically useful: you build them interactively. The standard Unix interface might have been interactive in the ’70s, back when hardware and peripherals were horribly non-interactive. But I don’t know why so many so-called millenial programmers (people my age) get excited about the alleged interactivity of the Unix that most people are familiar with. It doesn’t even have the cutting edge ’90s interactivity of Plan 9, what with mouse(!) selection of arbitrary text that can be piped to commands and so on. And every time someone comes up with a Unix-hosted tool that uses some kind of fold-up menu that informs you about what key combination you can type next (you know, like what all GUI programs have with Alt+x and the file|edit|view|… toolbar), people hail it as some kind of UX innovation.

SomethingOrNot 8y ago
One interactive feature I like about Bash though (or shell or whatever) is C-x C-e to edit the current line in a text editor (FC). Great when I know how to transform some text in the editor easily but not on the command line.
jraph 8y ago
I think the interactivity you describe might be a different thing from what your parent is talking about. From what I understand, your parent talks about how the commands are built iteratively, with some kind of trial-error loop, which is a strength that is supposedly not emphasized enough. And I agree by the way. Nothing to do with how things are input.
- pdkl95 8y ago
  That's correct. Articles/tutorials or an evangelizing fan often show the end result: the cool command/pipeline that does something cool and useful. The obvious question when someone unfamiliar with unix upon seeing something like the pipeline in this article: comm -1 -3 <(ls -1 dataset-directory | \ grep '\d\d\d\d_A.csv' | \ cut -c 1-4 | \ python3 parse.py | \ uniq \ ) \ <(seq 500) is "Why would I want to write a complicated mess like that?" Just use ${FAVORITE_PROG_LANG:-Perl, Ruby, or whatever}". For many tasks, a short paragraph of code in a "normal" programming language is probably easier to write and is almost certainly a more robust, easier to maintain solution. However, this assumes that you knew what the problem was and that qualities like maintainability are a goal. Bernhardt's (and my) point is that sometimes you don't know what the goal is yet. Sometimes you just need to do a small, one-off task where a half-assed solutions might be appropriate... iff it's the right half of the ass. Unix shell gets that right for a really useful set of tasks. This works because you are free to utilize that powerful features incrementally, as needed. The interactive nature of the shell lets you explore the problem. The "better" version in a "proper" programming language doesn't exist when you don't yet know the exact nature of the problem. A half-assed bit of shell code that slowly evolved into something useful might be the step between "I have some data" and a larger "real" programming project. That said, there is also wisdom in learning to recognize when your needs have outgrown "small, half-assed" solutions. If the project is growing and adding layers of complexity, it's probably time to switch to a more appropriate tool.
  - SomethingOrNot 8y ago
    I generalized interactivity to the Unix that most people seem familiar with. “The interactive nature of the shell” isn’t that impressive in this day and age. Certainly not shells like Bash (Fish is probably better, but then again that’s very cutting edge shell (“for the ’90s”)). Irrespective of the shell this just boils down to executing code, editing text, executing code, repeat. I suspect people started doing that once they got updating displays, if not sooner.
smittywerben 8y ago
I’m a lone dev that works with moderate-size data and whatever UX solution you’re thinking of is slow or doesn’t exist. Yes Bash is an untyped hell. But when I pipe 100 GB to stout, my computer’s death wish to show me the fucking data.
mnarayan01 8y ago
> mouse(!) selection of arbitrary text that can be piped to commands and so on E.g.: xclip -out | ... or do you mean something different?
- SomethingOrNot 8y ago
  Essentially selecting and operating on text in the same buffer. Saw it in Russ Cox’s demonstration of Acme.
  - pjmlp 8y ago
    The idea was taken from Oberon, which got inspired by Mesa/Cedar at Xerox PARC. Basically any function/procedure that gets exported from a module can be used either on the REPL, or from them mouse, depending on its signature. Quite powerful concept for those that like to take advantage of GUI based OSes. Powershell is the only that comes close to it. Maybe fish as well, but never used it.

mercer 8y ago

This is a reason why I like languages that have REPLs, and perhaps an advantage of dynamically typed, and especially functional languages that I feel is often not emphasized enough (judging by the recent discussions on the issue). I love how I can just open up IEx (Elixir) and work my way through various functions in my app interactively. If something doesn't work as expected, I change the code and run the 'recompile' command. Then, once things work and eventually stabilize, I add typespecs for at least some of the benefits of an explicitly typed language. In some cases I might make (unit) tests part of the process, but that depends on the situation.

yesenadam 8y ago

AWK makes most of that grep/cut/count-type stuff so much easier, 1-liners need something like 3 parts instead of 10. (My grep doesn't work like in the video, but..) for f in *.rb;do awk '$1 ~ /class|module/ {print $2}' $f;done | awk '{a[$1]++} END{for (q in a) print a[q],q}' | sort -n This seems to do what all the sed, cut, regrepping, wc etc do. AWK (in default mode, anyway) removes leading spaces, makes cutting and counting words easy. It took about 30 seconds to write, too.

voltagex_ 8y ago
I have never been able to get my head around awk - how did you learn it?
- FelipeCortez 8y ago
  The AWK programming language book is excellent! https://archive.org/download/pdfy-MgN0H1joIoDVoIC7/The_AWK_Programming_Language.pdf https://archive.org/download/pdfy-MgN0H1joIoDVoIC7/The_AWK_P... You can find some discussion about it here: https://news.ycombinator.com/item?id=13451454 https://news.ycombinator.com/item?id=13451454
  - voltagex_ 8y ago
    >This book was typeset in Times Roman and Courier by the authors, using an Autologic APS-5 phototypesetter and a DEC VAX 8550 running the 9th Edition of the UNIX operating system. Delightful!
- patrec 8y ago
  "The AWK programming language" has already been suggested and is great. But awk is actually pretty simple. Imagine everything having an implicit loop like this around it like so: # BEGIN{ ... } would go here for $0 in input.split('\n'): $1, $2, ... = line.split() # if TEST {ACTION}; pairs ... # END { ... } would go here so the first TEST in GP's awk would be `$1 ~ /class|module/` and the corresponding ACTION to take `{print $2}`. if TEST is omitted it's True, if ACTION is it's print the whole line.