3 ms·

More shell, less egg (2011)

45 points by jjgreen 3y ago

CrazyStat 3y ago
This has been posted a number of times on HN, I'll just point to the must recent one that spawned a fairly large number of comments (which in turn has links to various other previous discussions courtesy of dang): https://news.ycombinator.com/item?id=30017311 https://news.ycombinator.com/item?id=30017311
- metadat 3y ago
  Thanks! FWIW, this is the only prior submission with more than 2-3 comments.
  - CrazyStat 3y ago
    Also https://news.ycombinator.com/item?id=10081539 https://news.ycombinator.com/item?id=10081539 and https://news.ycombinator.com/item?id=3329668 https://news.ycombinator.com/item?id=3329668
    - metadat 3y ago
      Oh shit, nice. I went through every page of results on https://news.ycombinator.com/from?site=leancrew.com https://news.ycombinator.com/from?site=leancrew.com but somehow missed these. Thank you!
PeterWhittaker 3y ago
And that’s why I love pipelines. McIlroy’s solution and explanation are succinct and clear, and the solution itself easily and readily expandable and repurposable. Bravo.
tyingq 3y ago
It is elegant, and easy to understand and modify. But it's less efficient and significantly slower than even a naïve python or perl script since it sorts a lot more than it needs to.
- taeric 3y ago
  Me thinks we see different naive python on the regular. Because I wouldn't be shocked to see many flat out failures. That said https://www.amazon.com/Exercises-Programming-Style-Cristina-Videira/dp/0367350203 https://www.amazon.com/Exercises-Programming-Style-Cristina-... is a good sized book that is nothing but takes on this program from different styles. It is amusing how far you can take that. Which... is kind of the point of the Knuth solution here. It was a pedagogical exercise. As much to show how literate programs can be presented as it was to "solve" the stated problem. Edit: It occurs to me you could be referring to the pipeline version. Which, yeah, I don't think that really changes my post. :D. The final paragraph would be changed to the point of the Milroy's code is to get the answer, not to have a library function that is expected to be used again and again on input.
  - tyingq 3y ago
    Yes, I was talking about the pipeline version. Last time this came up, I tried a simple perl script[1] and it was quite a lot faster than the pipeline. Even without using a heap to further limit the sorting. [1] https://news.ycombinator.com/item?id=30038415 https://news.ycombinator.com/item?id=30038415
- im3w1l 3y ago
  I think it will be faster than python or perl, because the actual work is done by optimized c-code. However if you want to extend the program from a toy to something that can handle words like "won't" "c-code" "a.i." or "naïve" then it will be quite difficult to do so.
  - tyingq 3y ago
    I tried a perl script versus the shell pipeline, see my other comments in the thread. It's significantly faster, because using sort and uniq -c is a pretty high effort way to count words, especially for big lists. Your words seem to work fine with it, it's just splitting on whitespace.
- tuatoru 3y ago
adityaathalye 3y ago
The ACM has kindly published the original source along with the rest of their archives. cf. "Programming Pearls: A literate program" https://dl.acm.org/doi/10.1145/5948.315654 https://dl.acm.org/doi/10.1145/5948.315654 Aside: I received a masterclass from Gradmaster McIlroy, in context of this selfsame shell pipeline: https://www.evalapply.org/posts/shell-aint-a-bad-place-to-fp-part-1-doug-mcilroys-pipeline/index.html#appendix-an-unexpected-masterclass https://www.evalapply.org/posts/shell-aint-a-bad-place-to-fp... I was floating in the air all week :D (edit: typo)
heavypixels 3y ago
I think most of us are used to seeing literate programming in old articles, often by Knuth himself. One excellent, modern and practical example of literate programming can be found in Physically Based Rendering, by Pharr, Jakob and Humphreys. It's not just the industry-standard overview of rendering theory and contemporary techniques in a big ol' hardcover book, it's also the source for an advanced ray tracer incorporating these techniques. And written in modern C++, not ancient Pascal. The 3rd edition can be read online here: https://www.pbr-book.org/ https://www.pbr-book.org/

45 points by jjgreen 3y ago

This has been posted a number of times on HN, I'll just point to the must recent one that spawned a fairly large number of comments (which in turn has links to various other previous discussions courtesy of dang): https://news.ycombinator.com/item?id=30017311 https://news.ycombinator.com/item?id=30017311

metadat 3y ago
Thanks! FWIW, this is the only prior submission with more than 2-3 comments.
- CrazyStat 3y ago
  Also https://news.ycombinator.com/item?id=10081539 https://news.ycombinator.com/item?id=10081539 and https://news.ycombinator.com/item?id=3329668 https://news.ycombinator.com/item?id=3329668
  - metadat 3y ago
    Oh shit, nice. I went through every page of results on https://news.ycombinator.com/from?site=leancrew.com https://news.ycombinator.com/from?site=leancrew.com but somehow missed these. Thank you!

PeterWhittaker 3y ago

And that’s why I love pipelines. McIlroy’s solution and explanation are succinct and clear, and the solution itself easily and readily expandable and repurposable. Bravo.

tyingq 3y ago

It is elegant, and easy to understand and modify. But it's less efficient and significantly slower than even a naïve python or perl script since it sorts a lot more than it needs to.

taeric 3y ago
Me thinks we see different naive python on the regular. Because I wouldn't be shocked to see many flat out failures. That said https://www.amazon.com/Exercises-Programming-Style-Cristina-Videira/dp/0367350203 https://www.amazon.com/Exercises-Programming-Style-Cristina-... is a good sized book that is nothing but takes on this program from different styles. It is amusing how far you can take that. Which... is kind of the point of the Knuth solution here. It was a pedagogical exercise. As much to show how literate programs can be presented as it was to "solve" the stated problem. Edit: It occurs to me you could be referring to the pipeline version. Which, yeah, I don't think that really changes my post. :D. The final paragraph would be changed to the point of the Milroy's code is to get the answer, not to have a library function that is expected to be used again and again on input.
- tyingq 3y ago
  Yes, I was talking about the pipeline version. Last time this came up, I tried a simple perl script[1] and it was quite a lot faster than the pipeline. Even without using a heap to further limit the sorting. [1] https://news.ycombinator.com/item?id=30038415 https://news.ycombinator.com/item?id=30038415
im3w1l 3y ago
I think it will be faster than python or perl, because the actual work is done by optimized c-code. However if you want to extend the program from a toy to something that can handle words like "won't" "c-code" "a.i." or "naïve" then it will be quite difficult to do so.
- tyingq 3y ago
  I tried a perl script versus the shell pipeline, see my other comments in the thread. It's significantly faster, because using sort and uniq -c is a pretty high effort way to count words, especially for big lists. Your words seem to work fine with it, it's just splitting on whitespace.
tuatoru 3y ago

adityaathalye 3y ago

The ACM has kindly published the original source along with the rest of their archives. cf. "Programming Pearls: A literate program" https://dl.acm.org/doi/10.1145/5948.315654 https://dl.acm.org/doi/10.1145/5948.315654 Aside: I received a masterclass from Gradmaster McIlroy, in context of this selfsame shell pipeline: https://www.evalapply.org/posts/shell-aint-a-bad-place-to-fp-part-1-doug-mcilroys-pipeline/index.html#appendix-an-unexpected-masterclass https://www.evalapply.org/posts/shell-aint-a-bad-place-to-fp... I was floating in the air all week :D (edit: typo)

heavypixels 3y ago

I think most of us are used to seeing literate programming in old articles, often by Knuth himself. One excellent, modern and practical example of literate programming can be found in Physically Based Rendering, by Pharr, Jakob and Humphreys. It's not just the industry-standard overview of rendering theory and contemporary techniques in a big ol' hardcover book, it's also the source for an advanced ray tracer incorporating these techniques. And written in modern C++, not ancient Pascal. The 3rd edition can be read online here: https://www.pbr-book.org/ https://www.pbr-book.org/