4 ms·

Somebody, somewhere, wrote the Text::LevenshteinXS module. Somebody, somewhere, had to write sed, awk, head, sort, tr... It's all fine and dandy to say "look a

by pdpi 7y ago

Somebody, somewhere, wrote the Text::LevenshteinXS module. Somebody, somewhere, had to write sed, awk, head, sort, tr... It's all fine and dandy to say "look at these awesome tools that make tasks like these trivially easy. See how powerful Unix is". But this fails to consider that somebody, somewhere, has to be a tool writer, not just a tool user. Knuth's code was a tool writer's code, exemplifying a technique (Literate Programming) that is aimed at long form code writers in general. As with others before, the author fails to grasp that this is an apples to oranges comparison.

crystaldev 7y ago
Just like apples and oranges can be compared as foods (and fruits), bespoke vs. reused solutions can be compared so software. The reused parts are of higher quality than their corresponding bespoke implementations and can be composed to accomplish the same task as well as many others. It's a powerful lesson.
- JadeNB 7y ago
  > The reused parts are of higher quality than their corresponding bespoke implementations …. I think that stating this as an inevitable and uncontested fact might be pushing the point.
  - crystaldev 7y ago
    Most of the components are multi-platform, partly or fully POSIX-standardized, battle-tested, blazing, fast, etc. It's some of the most widely-used and arguably greatest software ever written.
    - nyberg 7y ago
      And some of it is absolutely terrible with several tools overlapping in use cases along with unclear perfornance hints. Even worse is the combination performing worse which may not matter on small inputs but has a huge impact when you crank up the size. Then there's the issue of understanding why performance suffers and in what cases it's better to roll your own custom solution that better fits the problem. Reuse saves work when your problem maps perfectly to existing tools while what may seem like a minor difference will propagate down your program to end in countless issues from a codebase you don't know.
- ncmncm 7y ago
  It takes real stones to suggest Unix utility code is better quality than Knuth's handicraft. And, not having looked into much of it. Unix code usually works well enough on unchallenging input.
  - zbentley 7y ago
    > Unix utility code is better quality What's your index of "better quality"? By some indices (simplicity, documented-ness, accessibility {as in: I can read and understand it without learning more than one language's behavior vs. bash/C/several others}), Knuth wins hands-down. By others (generality, speed of implementation) he does not.
    - ncmncm 7y ago
      It is normal to point out at this point that if correctness is not important to you, an overwhelmingly more quickly produced implementation is possible. But my comment was on remarks claiming superior reliability for the Unix utilities, which you have not addressed.
- michaelmrose 7y ago
  I believe another poster earlier reported that knuths solution was 140 times faster besides being applicable to building tools like the Unix tools.
JadeNB 7y ago
> As with others before, the author fails to grasp that this is an apples to oranges comparison. I think comparing apples to oranges would be comparing Knuth as a programmer to Spinellis (the author of the blog post) as a programmer; they obviously have different approaches to programming, which one can't compare without bringing in some externally imposed value judgements. The post itself isn't a comparison at all: it says "this task, which was claimed to be very difficult using only a certain set of tools, is actually easier than one would expect even when restricted to those tools." (OK, I guess 'easier' is a comparison, technically, but only to expectations.) This says nothing about the quality of Knuth, or of Spinellis, or of the ideologies of tool use and tool creation; it says only that the existing Unix toolkit is very rich, much more so than some might expect.
vilhelm_s 7y ago
I don't think there is such a sharp distinction between writing and using tools. You could take the 9-command pipeline, paste it into a shellscript, and now you have a new tool. And isn't this how most programming is done? Typically your program relies on a set of libraries, but the methodology of writing the library looks very similar to the methodology of writing the client code; at the end of the day, it's all programming. If literate programming is great, then I would expect it would be great everywhere. And if decomposing your program into a few steps and piping together already-written software for those steps is great, then I would expect you should try to use that style of programming as widely as possible.
- taeric 7y ago
  When the task was to make a tool, the criticism should not be that there are other tools to do this. So, yeah. If you want a chair, you just go buy a chair. But if you are curious if a method of furniture making is good, you don't take as evidence someone that orders a chair from a catalog.
  - Izkata 7y ago
    > you don't take as evidence someone that orders a chair from a catalog. That's misrepresenting what's going on. A better parallel is that the person curious about furniture-making still isn't doing everything from scratch. They're gonna go buy wood and tools - they're not creating an axe, hammer, screwdriver, screws, nails, chopping down a tree, and shaping the wood themselves. The furniture-maker is still following the unix philosophy of using and building on prior work.
    - taeric 7y ago
      Depends how you read it. He offered to demonstrate a tool, literate programming. The task he was asked to demonstrate against was counting words. He was not asked to count words, that is different. It was a purposely pedagogical exercise to demonstrate literate programming. Which is itself a pedagogical method to write a program. To that end, reading his program you can learn about how he wrote his program. Reading the shell script can really just learn how the program was written. If you don't know what those commands do, you will not learn it from the script. I don't know Pascal, but most of the other still makes sense to me. Such that I could probably port it. The shell script? Now, again, if the task is just to count the words, you are probably fine with whatever crosses the finish line. If the task is to demonstrate literate programming? How does that help?
ssalazar 7y ago
Yeah. Honestly, this whole back-and-forth is exhaustingly trite and pedantic.
- crispinb 7y ago
  This is one of those stories I glance at briefly out of a kind of mildly masochistic fascination ("are people really going to argue about this" "you know damn well they are").
themodelplumber 7y ago
Those authors (of the Perl module and other software) may have worked from existing source material from other authors, too. In fact I'd guess it's pretty likely. Research into this could even promote a much more inclusive and informed approach to technology use and development. (Until then though, perhaps we'll keep having this use-your-own-brain vs. don't-reinvent-the-wheel discussion)
smoyer 7y ago
I came here to basically say the same thing ... if Knuth weren't demonstrating Literate Programming (LP), he would have used a library version of the Trie function that took up most of those eight pages. At that point, who cares whether those six pages were done in (LP) or not ... it might be easier to maintain in the long-term but either way it's code you don't have to write. If he was finding the Levenshtein distance as with this post, maybe his LP system could use the Perl library too?

by pdpi 7y ago

crystaldev 7y ago

Just like apples and oranges can be compared as foods (and fruits), bespoke vs. reused solutions can be compared so software. The reused parts are of higher quality than their corresponding bespoke implementations and can be composed to accomplish the same task as well as many others. It's a powerful lesson.

JadeNB 7y ago
> The reused parts are of higher quality than their corresponding bespoke implementations …. I think that stating this as an inevitable and uncontested fact might be pushing the point.
- crystaldev 7y ago
  Most of the components are multi-platform, partly or fully POSIX-standardized, battle-tested, blazing, fast, etc. It's some of the most widely-used and arguably greatest software ever written.
  - nyberg 7y ago
    And some of it is absolutely terrible with several tools overlapping in use cases along with unclear perfornance hints. Even worse is the combination performing worse which may not matter on small inputs but has a huge impact when you crank up the size. Then there's the issue of understanding why performance suffers and in what cases it's better to roll your own custom solution that better fits the problem. Reuse saves work when your problem maps perfectly to existing tools while what may seem like a minor difference will propagate down your program to end in countless issues from a codebase you don't know.
ncmncm 7y ago
It takes real stones to suggest Unix utility code is better quality than Knuth's handicraft. And, not having looked into much of it. Unix code usually works well enough on unchallenging input.
- zbentley 7y ago
  > Unix utility code is better quality What's your index of "better quality"? By some indices (simplicity, documented-ness, accessibility {as in: I can read and understand it without learning more than one language's behavior vs. bash/C/several others}), Knuth wins hands-down. By others (generality, speed of implementation) he does not.
  - ncmncm 7y ago
    It is normal to point out at this point that if correctness is not important to you, an overwhelmingly more quickly produced implementation is possible. But my comment was on remarks claiming superior reliability for the Unix utilities, which you have not addressed.
michaelmrose 7y ago
I believe another poster earlier reported that knuths solution was 140 times faster besides being applicable to building tools like the Unix tools.

JadeNB 7y ago

> As with others before, the author fails to grasp that this is an apples to oranges comparison. I think comparing apples to oranges would be comparing Knuth as a programmer to Spinellis (the author of the blog post) as a programmer; they obviously have different approaches to programming, which one can't compare without bringing in some externally imposed value judgements. The post itself isn't a comparison at all: it says "this task, which was claimed to be very difficult using only a certain set of tools, is actually easier than one would expect even when restricted to those tools." (OK, I guess 'easier' is a comparison, technically, but only to expectations.) This says nothing about the quality of Knuth, or of Spinellis, or of the ideologies of tool use and tool creation; it says only that the existing Unix toolkit is very rich, much more so than some might expect.

vilhelm_s 7y ago

I don't think there is such a sharp distinction between writing and using tools. You could take the 9-command pipeline, paste it into a shellscript, and now you have a new tool. And isn't this how most programming is done? Typically your program relies on a set of libraries, but the methodology of writing the library looks very similar to the methodology of writing the client code; at the end of the day, it's all programming. If literate programming is great, then I would expect it would be great everywhere. And if decomposing your program into a few steps and piping together already-written software for those steps is great, then I would expect you should try to use that style of programming as widely as possible.

taeric 7y ago
When the task was to make a tool, the criticism should not be that there are other tools to do this. So, yeah. If you want a chair, you just go buy a chair. But if you are curious if a method of furniture making is good, you don't take as evidence someone that orders a chair from a catalog.
- Izkata 7y ago
  > you don't take as evidence someone that orders a chair from a catalog. That's misrepresenting what's going on. A better parallel is that the person curious about furniture-making still isn't doing everything from scratch. They're gonna go buy wood and tools - they're not creating an axe, hammer, screwdriver, screws, nails, chopping down a tree, and shaping the wood themselves. The furniture-maker is still following the unix philosophy of using and building on prior work.
  - taeric 7y ago
    Depends how you read it. He offered to demonstrate a tool, literate programming. The task he was asked to demonstrate against was counting words. He was not asked to count words, that is different. It was a purposely pedagogical exercise to demonstrate literate programming. Which is itself a pedagogical method to write a program. To that end, reading his program you can learn about how he wrote his program. Reading the shell script can really just learn how the program was written. If you don't know what those commands do, you will not learn it from the script. I don't know Pascal, but most of the other still makes sense to me. Such that I could probably port it. The shell script? Now, again, if the task is just to count the words, you are probably fine with whatever crosses the finish line. If the task is to demonstrate literate programming? How does that help?

ssalazar 7y ago

Yeah. Honestly, this whole back-and-forth is exhaustingly trite and pedantic.

crispinb 7y ago
This is one of those stories I glance at briefly out of a kind of mildly masochistic fascination ("are people really going to argue about this" "you know damn well they are").

themodelplumber 7y ago

Those authors (of the Perl module and other software) may have worked from existing source material from other authors, too. In fact I'd guess it's pretty likely. Research into this could even promote a much more inclusive and informed approach to technology use and development. (Until then though, perhaps we'll keep having this use-your-own-brain vs. don't-reinvent-the-wheel discussion)

smoyer 7y ago

I came here to basically say the same thing ... if Knuth weren't demonstrating Literate Programming (LP), he would have used a library version of the Trie function that took up most of those eight pages. At that point, who cares whether those six pages were done in (LP) or not ... it might be easier to maintain in the long-term but either way it's code you don't have to write. If he was finding the Levenshtein distance as with this post, maybe his LP system could use the Perl library too?