3 ms·

> But it's also not obvious, i.e. not something a "normal user" would "know": > - even if you knew that "tr command translates... "sed ${1}q" how many people e

> But it's also not obvious, i.e. not something a "normal user" would "know": > - even if you knew that "tr command translates... "sed ${1}q" how many people even today would know that one? Are you suggesting it's ever been more likely for people to understand how to manage a trie structure in Pascal than use Unix command line tools? Or look flags up in the manpages? Personally speaking, I'm comfortable doing both, but can't imagine many scenarios where I'd rather have ten pages of a custom datastructure than six lines of shell. (And they all involve either high volumes of data or situations where I can't easily get to the process-level tools.) > The fair comparison would then include the source of all the sort, uniq, tr etc. programs which McIlroy used. If you're including the code that provides the surface abstractions, where do you draw that line? If the code for sort, uniq, etc. is fair game, why not the code for the shell that provides the pipelining, the underlying OS, the file system, the firmware on the disk? After all, who's to say that the programs in the pipeline don't just run one after another with temporary files written to disk, rather than in parallel? (Which I've seen happen in reality.) The same is true for the other side, of course. The 'fair comparison' could easily require Knuth's solution to include the source for weave/tangle, TeX/Metafont/CMR, the OS, etc. > And once that is being done, McIlroy's code would still be both less readable, less efficient and worse overall. What definition of 'worse' are you using? * I expect sort/uniq/tr/sed to be more well tested and understood than a bespoke program. * If there are issues with the program, it'll be easier to find skills/time to maintain a shell pipeline than custom trie-balancing code written in Pascal. (Sitting aside a prose description of the same.) * The shell pipeline runs on a machine that can be found in a retail store today, rather than requiring an elaborate download/build process. * It's possible that the custom solution runs faster, but not obvious without testing. (None of which is worthwhile outside of demonstrated need.) Point being: it's very easy to find a definition of 'worse' that applies more to the custom solution than to the pipeline.

acqq 8y ago
> The shell pipeline runs on a machine that can be found in a retail store today, rather than requiring an elaborate download/build process. That argument points to the fact that your “view” of the whole topic changes the assumed definition of the problem that was given to Knuth to solve. Read once again the original text: he was supposed to illustrate how the “Literate programming” could be ised while writing a program which solves a given program. It was definitely not “write an example of calling the existing pre-written programs”. And, of course, it was all in 1986, definitely not “to target the machine which can be found in the retail store in 2018.” McIlroy already behaved as the goal had been different than it was.
- mschaef 8y ago
  > McIlroy already behaved as the goal had been different than it was. How would you feel about McIlroy's solution if it was semantically exactly the same, but written in a literate approach? (Essentially a version of 'weave/tangle', but for shell scripts.)
  - acqq 8y ago
    How would you feel if somebody would present the “six steps clicks in Excel and SQL Server” that eventually produce the same result? The starting goal was simply not “show how to use and combine external programs.” Even if you need some kind of skill to combine them. It’s exactly the same kind of missed fullfilment of the given starting task.
    - mschaef 8y ago
      The reason I asked about a literate programming version of the shell script is that it speaks directly to Knuth's original stated goal: "I’ll try to prove the merits of literate programming by finding the best possible solution to whatever problem you pose" In the context of that requirement, the it's the use of literate programming that's more of a concern than the specific implementation. (Which is why I asked about a literate version of the shell pipeline.) Earlier in the thread, you also mention this concern around data volumes: > If you have hundreds of gigabytes of input, you'd have to have at least that much more just to sort it. McIlroy's approach is a good one for one-off program or not too big input processing, There, your concern is not justified by the stated requirements of the problem: "I did impose an efficiency constraint: a user should be able to find the 100 most frequent words in a twenty-page technical paper" I do think McIlroy failed to solve the problem of demonstrating the value of literate programming, but I'm not sympathetic to arguments that he should've used more complex algorithms or relied on less library code. This is particularly the case when the additional complexity is only relevant in cases that don't fall into the given requirements. (A literate program that uses SQL server or Excel might be an interesting read....)
      - 8y ago

acqq 8y ago

> The shell pipeline runs on a machine that can be found in a retail store today, rather than requiring an elaborate download/build process. That argument points to the fact that your “view” of the whole topic changes the assumed definition of the problem that was given to Knuth to solve. Read once again the original text: he was supposed to illustrate how the “Literate programming” could be ised while writing a program which solves a given program. It was definitely not “write an example of calling the existing pre-written programs”. And, of course, it was all in 1986, definitely not “to target the machine which can be found in the retail store in 2018.” McIlroy already behaved as the goal had been different than it was.

mschaef 8y ago
> McIlroy already behaved as the goal had been different than it was. How would you feel about McIlroy's solution if it was semantically exactly the same, but written in a literate approach? (Essentially a version of 'weave/tangle', but for shell scripts.)
- acqq 8y ago
  How would you feel if somebody would present the “six steps clicks in Excel and SQL Server” that eventually produce the same result? The starting goal was simply not “show how to use and combine external programs.” Even if you need some kind of skill to combine them. It’s exactly the same kind of missed fullfilment of the given starting task.
  - mschaef 8y ago
    The reason I asked about a literate programming version of the shell script is that it speaks directly to Knuth's original stated goal: "I’ll try to prove the merits of literate programming by finding the best possible solution to whatever problem you pose" In the context of that requirement, the it's the use of literate programming that's more of a concern than the specific implementation. (Which is why I asked about a literate version of the shell pipeline.) Earlier in the thread, you also mention this concern around data volumes: > If you have hundreds of gigabytes of input, you'd have to have at least that much more just to sort it. McIlroy's approach is a good one for one-off program or not too big input processing, There, your concern is not justified by the stated requirements of the problem: "I did impose an efficiency constraint: a user should be able to find the 100 most frequent words in a twenty-page technical paper" I do think McIlroy failed to solve the problem of demonstrating the value of literate programming, but I'm not sympathetic to arguments that he should've used more complex algorithms or relied on less library code. This is particularly the case when the additional complexity is only relevant in cases that don't fall into the given requirements. (A literate program that uses SQL server or Excel might be an interesting read....)
    - 8y ago