6 ms·

I love that story, although I think the takeaway needs to be slightly updated. It's quite possible that a sorted word count was easiest to do with bash and unix

I love that story, although I think the takeaway needs to be slightly updated. It's quite possible that a sorted word count was easiest to do with bash and unix utilities in 1992. Now, it's easier and more comprehensible in your favorite scripting language (which I'm assuming isn't bash). The real lesson is to program using the most powerful tools at your disposal.

gmfawcett 8y ago
That's a good lesson, but we shouldn't stop there. Knuth is a computer scientist, but also an artist: writer, musician, typographer, and creative programmer. His insights into programming-as-literature have infused an often-soulless industry with something soulful. He began with the popular notion that programming should be a creative, playful act, and tried to elevate mere playfulness into the realm of fine art. An incredible detour in an amazing career, and an enriching lesson of a different kind.
- billfruit 8y ago
  And a theologian, he has has written works on divinity, etc..
mschaef 8y ago
> It's quite possible that a sorted word count was easiest to do with bash and unix utilities in 1992. Now, it's easier and more comprehensible in your favorite scripting language It'd be interesting to see how true this is in reality. I'm not at all convinced that most scripting languages can do this quite as concisely as bash and the Unix userland. (But, being honest, this problem seems very well aligned to that tooling's strenghts.)
- sobellian 8y ago
  Well, what is easy or comprehensible isn't necessarily concise. Bash is great for code golf but I know few people who prefer to code in it.
  - sobellian 8y ago
    And for completeness, a Ruby implementation: puts STDIN.read .split(/[^A-Za-z]+/) .group_by(&:downcase).transform_values(&:size) .sort_by(&:last).reverse .take(ARGV[0].to_i) .map{ |l| l.join("\t") }.join("\n") Wordier than the bash script, but still pretty short and fairly trivial. Anyone could make this in five minutes.
- btilly 8y ago
  It is easy to do in your favorite scripting language. But is unlikely to be as short. Comprehensibility is in the eye of the beholder. Here is a Perl solution for comparison. use strict; use warnings; my $limit = shift @ARGV; my %count; for my $line (<>) { while ($line =~ /(\w+)/g) { $count{lc $1}++; } } @words = sort {$count{$b} <=> $count{$a} or $a cmp $b} keys %count; $limit = @words if @words < $limit; print "$_\t$count{$_}\n" for @words[0..($limit-1)];
  - mkl 8y ago
    Python is relatively short, batteries included and all that: import re import sys import collections words = re.findall('[a-z]+', sys.stdin.read().lower()) c = collections.Counter(words) for word, count in c.most_common(int(sys.argv[1])): print(f'{count:>7} {word}') Three times the character count of the Unix pipe version, but IMHO a lot more readable (and generalisable).
    - mkesper 8y ago
      You don't want to read all input into memory first.
      - mkl 8y ago
        Fair enough. import re import sys import collections c = collections.Counter() for line in sys.stdin: words = re.findall('[a-z]+', line.lower()) c.update(words) for word, count in c.most_common(int(sys.argv[1])): print(f'{count:>7} {word}')
  - lizmat 8y ago
    Just for reference: a Perl 6 version: sub MAIN($limit = Inf) { my %bag is Bag = words.map(&lc); say "{.key}\t{.value}" for %bag.sort( { $^b.value cmp $^a.value || $^a.key cmp $^b.key } )[^$limit] } And for what it's worth, the `words` function is lazy, so it won't read all of the words into memory first.
flukus 8y ago
There is no bash (or any scripting) involved, the complete command from the article is: tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q The real benefit is that each is a single command that is easy to test in isolation and it's multi-process. That's not possible in most scripting languages.
- sobellian 8y ago
  McIlroy literally calls it a script in his review (notice the ${1}). Nothing prevents you from unit testing in scripting languages. Multi-process, sure, but most people aren't looking for that.
  - flukus 8y ago
    > McIlroy literally calls it a script in his review (notice the ${1}). Ok, there's a single shell substitution, if it was fixed would you still call it a script? Technically the result of that is itself a sed script "3q", but if you count either of those then there isn't a lot of wiggle room between script and command, the arguments to tr are by far the most complex "script" involved. > Nothing prevents you from unit testing in scripting languages. That is a world away from what I'm talking about. Each line of that command can be executed on the CLI in isolation, you'd be replicating a lot more in nearly any scripting language, except maybe perl and awk. > Multi-process, sure, but most people aren't looking for that. Neither am I generally, still quite nice when you get it for free though.
    - sobellian 8y ago
      You're really splitting hairs. You can execute a Python command in a REPL. There's little material difference between scripts and commands for our purposes. And scripting languages provide facilities to test functions in isolation.
- zbentley 8y ago
  > There is no bash (or any scripting) involved What provides the pipe functionality?

gmfawcett 8y ago

That's a good lesson, but we shouldn't stop there. Knuth is a computer scientist, but also an artist: writer, musician, typographer, and creative programmer. His insights into programming-as-literature have infused an often-soulless industry with something soulful. He began with the popular notion that programming should be a creative, playful act, and tried to elevate mere playfulness into the realm of fine art. An incredible detour in an amazing career, and an enriching lesson of a different kind.

billfruit 8y ago
And a theologian, he has has written works on divinity, etc..

mschaef 8y ago

> It's quite possible that a sorted word count was easiest to do with bash and unix utilities in 1992. Now, it's easier and more comprehensible in your favorite scripting language It'd be interesting to see how true this is in reality. I'm not at all convinced that most scripting languages can do this quite as concisely as bash and the Unix userland. (But, being honest, this problem seems very well aligned to that tooling's strenghts.)

sobellian 8y ago
Well, what is easy or comprehensible isn't necessarily concise. Bash is great for code golf but I know few people who prefer to code in it.
- sobellian 8y ago
  And for completeness, a Ruby implementation: puts STDIN.read .split(/[^A-Za-z]+/) .group_by(&:downcase).transform_values(&:size) .sort_by(&:last).reverse .take(ARGV[0].to_i) .map{ |l| l.join("\t") }.join("\n") Wordier than the bash script, but still pretty short and fairly trivial. Anyone could make this in five minutes.
btilly 8y ago
It is easy to do in your favorite scripting language. But is unlikely to be as short. Comprehensibility is in the eye of the beholder. Here is a Perl solution for comparison. use strict; use warnings; my $limit = shift @ARGV; my %count; for my $line (<>) { while ($line =~ /(\w+)/g) { $count{lc $1}++; } } @words = sort {$count{$b} <=> $count{$a} or $a cmp $b} keys %count; $limit = @words if @words < $limit; print "$_\t$count{$_}\n" for @words[0..($limit-1)];
- mkl 8y ago
  Python is relatively short, batteries included and all that: import re import sys import collections words = re.findall('[a-z]+', sys.stdin.read().lower()) c = collections.Counter(words) for word, count in c.most_common(int(sys.argv[1])): print(f'{count:>7} {word}') Three times the character count of the Unix pipe version, but IMHO a lot more readable (and generalisable).
  - mkesper 8y ago
    You don't want to read all input into memory first.
    - mkl 8y ago
      Fair enough. import re import sys import collections c = collections.Counter() for line in sys.stdin: words = re.findall('[a-z]+', line.lower()) c.update(words) for word, count in c.most_common(int(sys.argv[1])): print(f'{count:>7} {word}')
- lizmat 8y ago
  Just for reference: a Perl 6 version: sub MAIN($limit = Inf) { my %bag is Bag = words.map(&lc); say "{.key}\t{.value}" for %bag.sort( { $^b.value cmp $^a.value || $^a.key cmp $^b.key } )[^$limit] } And for what it's worth, the `words` function is lazy, so it won't read all of the words into memory first.

flukus 8y ago

There is no bash (or any scripting) involved, the complete command from the article is: tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q The real benefit is that each is a single command that is easy to test in isolation and it's multi-process. That's not possible in most scripting languages.

sobellian 8y ago
McIlroy literally calls it a script in his review (notice the ${1}). Nothing prevents you from unit testing in scripting languages. Multi-process, sure, but most people aren't looking for that.
- flukus 8y ago
  > McIlroy literally calls it a script in his review (notice the ${1}). Ok, there's a single shell substitution, if it was fixed would you still call it a script? Technically the result of that is itself a sed script "3q", but if you count either of those then there isn't a lot of wiggle room between script and command, the arguments to tr are by far the most complex "script" involved. > Nothing prevents you from unit testing in scripting languages. That is a world away from what I'm talking about. Each line of that command can be executed on the CLI in isolation, you'd be replicating a lot more in nearly any scripting language, except maybe perl and awk. > Multi-process, sure, but most people aren't looking for that. Neither am I generally, still quite nice when you get it for free though.
  - sobellian 8y ago
    You're really splitting hairs. You can execute a Python command in a REPL. There's little material difference between scripts and commands for our purposes. And scripting languages provide facilities to test functions in isolation.
zbentley 8y ago
> There is no bash (or any scripting) involved What provides the pipe functionality?