4 ms·

Having never heard of literate programming, I started poking around. "Physically Based Rendering"'s intro to literate programming can be found here: http://www

by hoten 6y ago

Having never heard of literate programming, I started poking around. "Physically Based Rendering"'s intro to literate programming can be found here: http://www.pbr-book.org/3ed-2018/contents.html http://www.pbr-book.org/3ed-2018/contents.html (chapter 1.1) And then answering my own first question about this: how is this different from copious comments? (Source: Wikipedia) > Literate programming is very often misunderstood[12] to refer only to formatted documentation produced from a common file with both source code and comments – which is properly called documentation generation – or to voluminous commentaries included with code. This is the converse of literate programming: well-documented code or documentation extracted from code follows the structure of the code, with documentation embedded in the code; while in literate programming, code is embedded in documentation, with the code following the structure of the documentation. > This misconception has led to claims that comment-extraction tools, such as the Perl Plain Old Documentation or Java Javadoc systems, are "literate programming tools". However, because these tools do not implement the "web of abstract concepts" hiding behind the system of natural-language macros, or provide an ability to change the order of the source code from a machine-imposed sequence to one convenient to the human mind, they cannot properly be called literate programming tools in the sense intended by Knuth.[12][13] I was then curious if any tooling has been created for literate programming in TypeScript. I haven't given a close look yet, but here are some relevant links: - https://github.com/microsoft/TypeScript/issues/31364 https://github.com/microsoft/TypeScript/issues/31364 - https://johtela.github.io/litscript/ https://johtela.github.io/litscript/ - https://mistlog.github.io/ray-tracing/index.html https://mistlog.github.io/ray-tracing/index.html

AndrewOMartin 6y ago
I've done a few projects using literate programming, a couple of which were fairly large for a single developer. If you use noweb, it doesn't care about the language you use. You can even have a section in the document which contains chunks of code in different languages and how they interact. For example, one chunk could be a part of the render function of a React component, the next chunk could be the Flask API endpoint which generates the data displayed by the component. Literate programming has its problems, and it is made less necessary with modern languages and tools, but I've repeatedly had the experience where it was LP that allowed me to return to a project after a few months and know exactly what was going on and where to continue.
- AndrewOMartin 6y ago
  My earlier post inspired me to push one of my Literate Programming, if inauspicious, projects to my public github. https://github.com/AndrewOwenMartin/nestor https://github.com/AndrewOwenMartin/nestor This repo implements an all-but-unknown ANN called NESTOR, but you can use it as an example of how it would be relatively easy for someone who knows nothing (or has forgotten everything) about NESTOR to contribute to the project. I make no claims that it is a good example of such.
cxr 6y ago
I've been deeply fascinated by the idea of literate programming, but every time I dive into the programs themselves, they leave something to be desired. First, though, I glossed over the PBRT shoutout and thought the Academy Award mention was about lkesteloot's team's award, until I scanned back and saw that it was about PBRT. To qualify that, here's one of lkesteloot's comments from his Oscars page on his personal site: > Our renderer and lighting tool were individually good, but lots of renderers and lighting tools are good. What made ours unusual (and it’s unusual in this respect to this day) is that they were designed and built at the same time, with each other in mind. <https://www.teamten.com/lawrence/oscar/ https://www.teamten.com/lawrence/oscar/> It might not be obvious, but this really touches on a part of the philosophy underpinning literate programming. If you look at any of the recent discussions about Knuth being "framed" in relation to the seminal Programming Pearls guest column, one of the recurring themes is programs as monoliths. In a really spectacular post "An app can be a home-cooked meal" <https://www.robinsloan.com/notes/home-cooked-app/ https://www.robinsloan.com/notes/home-cooked-app/>, the author explains his own work and likens it to Shirky's description of "situated software". Shirky writes, "Situated software isn't a technological strategy so much as an attitude about closeness of fit between software and its group of users" <https://web.archive.org/web/20040411202042/http://www.shirky.com/writings/situated_software.html https://web.archive.org/web/20040411202042/http://www.shirky...>. This obviously sounds a lot like PDI's renderer and very much characterizes Knuth's monoliths. I always liked this quote from literateprogramming.com: > The fundamental logic of the WEB system encourages "top-down" programming and "structured" design. Quoting from Kernighan and Plauger, 'Top-down design and successive refinement attack a programming task by specifying it in the most general terms, then expanding these into more and more specific and detailed actions, until the whole program is complete. [...] The WEB system encourages you to work top-down by giving you the ability to break up your code into independent segments (called "sections"). Coincidentally, in addition to lkesteloot's Oscar comments, he's got a pretty great essay called "Write code top-down" which really delves into the matter <https://www.teamten.com/lawrence/programming/write-code-top-down.html https://www.teamten.com/lawrence/programming/write-code-top-...>. Opinionated and well worth the read. As for this interview, PBRT, and Knuth's flavor of literate programming: the PBRT book is available for free online <http://www.pbr-book.org/3ed-2018/Introduction/Literate_Programming.html http://www.pbr-book.org/3ed-2018/Introduction/Literate_Progr...>, and the intro starts out with a section on literate programming and the WEB/noweb family in particular <http://www.pbr-book.org/3ed-2018/Introduction/Literate_Programming.html http://www.pbr-book.org/3ed-2018/Introduction/Literate_Progr...>. This, I think, is already adequate to highlight some of the issues with the WEB/noweb implementation of literate programming. It's still too mired by constraints of C (or maybe since the language isn't technically C, and Knuth is also known for literate programming with Pascal, we should chalk it up to something akin to Sapir–Whorf?). akkartik has a good essay on the matter of Knuth's programs <http://akkartik.name/post/literate-programming http://akkartik.name/post/literate-programming>. akkartik: > just about every literate program out there begins with cruft like this: `// Some #includes` or: `-- Don't mind these imports.` I used to think people just didn't understand Knuth's vision. But then I went and looked at his literate programs. Indeed, I've also looked at Knuth's programs and felt a similar twinge. There's the seminal column, available from literateprogramming.com and linked from akkartik's post. Having tried to read it, a big issue there is that the problem just isn't very interesting in my opinion, at least on practical grounds. I've mentioned before that I think a literate programming advocate and true believer would do well to do something like publish a piece that focuses on implementing support for DEFLATE and the ZIP file format, which strikes me as a much stronger hook than a program to "print the k most common words in the file in decreasing frequency" or "Dijkstra's program [that] prints a table of the first thou-sand prime numbers". No matter, though. There are "real" programs published by practitioners of the style, right? How about Knuth's rendition of Colossal Cave Adventure? <http://literateprogramming.com/adventure.pdf http://literateprogramming.com/adventure.pdf> Once again, we're really running into sharp edges of C that WEB—or at least this particular text—doesn't do a good job of softening up, for the reasons akkartik rightly points out. Just look at the second page and right away with `main` we've got unexplained a priori structure, including stuff like local variables j, k, and p. Other programs I've looked into are lcc, and the glimpses I've gotten of Inform 7. On my first encounter with the former, I was motivated to jot down in my notes a proposed law that said, roughly, a poor/failed attempt to write a literate program will produce a piece of text that's harder to understand than if it had just been written "straight". Still, I'm hopeful that there's something there to be found in literate programming. In fact, I'm pretty sure there is something there. I wrote a recent comment soliciting info and critiques/criticism about literate programming for pedagogy <https://news.ycombinator.com/item?id=25498685 https://news.ycombinator.com/item?id=25498685>. And it's funny that you mention TypeScript. As I said, I think part of the inadequancy of CWEB comes down to C. C is fundamentally opposed to one's attempts to practice top-down programming—one of the first things that "every" programmer learned, back when "every programmer" meant programmers who learn C, is that when you read a C program, you're supposed to start at the bottom and kind-of-sort-of read it backwards. Although I suppose there's an argument to be made that were it not the case for C (or Pascal) inflicting this upon you, then Knuth may never have been sufficiently motivated to undertake his literate programming crusade in the first place. I've had niggling thoughts that a language like JS with support for forward declarations in the form of function hoisting and its dynamism at runtime, including allowance for function redefinition (or at least if not JS, then something with similar properties) would be a more perfect match for the top-down style associated with literate programming. TypeScript, though, might be a little too strict that causes it to suffer for similar reasons as C. Also the fact that JS's roots lie in a (much maligned) system for conveying documents (as opposed to an application platform) makes the fit even better. It's just that nobody seems to be noticing this feature-not-a-bug, because they've been focused on trying to recreate mobile app conventions in the browser. And I've actually sat down to experiment and play around with the affordances that could prove TBL's Web (as opposed to Knuth's WEB) a more attractive substrate for a literate programming system. More recently, I've gotten more serious about actually testing the limits of the medium while thinking of things to put together that are meant to be consumed by other people as pieces of written works to be read from top to bottom in Knuthian style, but it comes down to a matter of having enough time. A final note is that in Bob Nystrom's post on 'Crafting "Crafting Interpreters"' <http://journal.stuffwithstuff.com/2020/04/05/crafting-crafting-interpreters/ http://journal.stuffwithstuff.com/2020/04/05/crafting-crafti...>, he more or less lays out the case for a literate programming system to help with actual book authoring. I hope that if you or anyone else decides to take an interest and try hashing something out that you're successful.
jacquesm 6y ago
Look into the Leo programming editor, if this interests you. It's an elegant solution, it doesn't scale well to teams and it's a bit quirky but I think it embodies the concept very well.
cafard 6y ago
Something over 25 years ago, I downloaded and installed some program for Solaris 2.x that was written in CWEB. I wish I could remember what the program was. Oh, and thanks for the link to the PBR book.

by hoten 6y ago

AndrewOMartin 6y ago

I've done a few projects using literate programming, a couple of which were fairly large for a single developer. If you use noweb, it doesn't care about the language you use. You can even have a section in the document which contains chunks of code in different languages and how they interact. For example, one chunk could be a part of the render function of a React component, the next chunk could be the Flask API endpoint which generates the data displayed by the component. Literate programming has its problems, and it is made less necessary with modern languages and tools, but I've repeatedly had the experience where it was LP that allowed me to return to a project after a few months and know exactly what was going on and where to continue.

AndrewOMartin 6y ago
My earlier post inspired me to push one of my Literate Programming, if inauspicious, projects to my public github. https://github.com/AndrewOwenMartin/nestor https://github.com/AndrewOwenMartin/nestor This repo implements an all-but-unknown ANN called NESTOR, but you can use it as an example of how it would be relatively easy for someone who knows nothing (or has forgotten everything) about NESTOR to contribute to the project. I make no claims that it is a good example of such.

cxr 6y ago

I've been deeply fascinated by the idea of literate programming, but every time I dive into the programs themselves, they leave something to be desired. First, though, I glossed over the PBRT shoutout and thought the Academy Award mention was about lkesteloot's team's award, until I scanned back and saw that it was about PBRT. To qualify that, here's one of lkesteloot's comments from his Oscars page on his personal site: > Our renderer and lighting tool were individually good, but lots of renderers and lighting tools are good. What made ours unusual (and it’s unusual in this respect to this day) is that they were designed and built at the same time, with each other in mind. <https://www.teamten.com/lawrence/oscar/ https://www.teamten.com/lawrence/oscar/> It might not be obvious, but this really touches on a part of the philosophy underpinning literate programming. If you look at any of the recent discussions about Knuth being "framed" in relation to the seminal Programming Pearls guest column, one of the recurring themes is programs as monoliths. In a really spectacular post "An app can be a home-cooked meal" <https://www.robinsloan.com/notes/home-cooked-app/ https://www.robinsloan.com/notes/home-cooked-app/>, the author explains his own work and likens it to Shirky's description of "situated software". Shirky writes, "Situated software isn't a technological strategy so much as an attitude about closeness of fit between software and its group of users" <https://web.archive.org/web/20040411202042/http://www.shirky.com/writings/situated_software.html https://web.archive.org/web/20040411202042/http://www.shirky...>. This obviously sounds a lot like PDI's renderer and very much characterizes Knuth's monoliths. I always liked this quote from literateprogramming.com: > The fundamental logic of the WEB system encourages "top-down" programming and "structured" design. Quoting from Kernighan and Plauger, 'Top-down design and successive refinement attack a programming task by specifying it in the most general terms, then expanding these into more and more specific and detailed actions, until the whole program is complete. [...] The WEB system encourages you to work top-down by giving you the ability to break up your code into independent segments (called "sections"). Coincidentally, in addition to lkesteloot's Oscar comments, he's got a pretty great essay called "Write code top-down" which really delves into the matter <https://www.teamten.com/lawrence/programming/write-code-top-down.html https://www.teamten.com/lawrence/programming/write-code-top-...>. Opinionated and well worth the read. As for this interview, PBRT, and Knuth's flavor of literate programming: the PBRT book is available for free online <http://www.pbr-book.org/3ed-2018/Introduction/Literate_Programming.html http://www.pbr-book.org/3ed-2018/Introduction/Literate_Progr...>, and the intro starts out with a section on literate programming and the WEB/noweb family in particular <http://www.pbr-book.org/3ed-2018/Introduction/Literate_Programming.html http://www.pbr-book.org/3ed-2018/Introduction/Literate_Progr...>. This, I think, is already adequate to highlight some of the issues with the WEB/noweb implementation of literate programming. It's still too mired by constraints of C (or maybe since the language isn't technically C, and Knuth is also known for literate programming with Pascal, we should chalk it up to something akin to Sapir–Whorf?). akkartik has a good essay on the matter of Knuth's programs <http://akkartik.name/post/literate-programming http://akkartik.name/post/literate-programming>. akkartik: > just about every literate program out there begins with cruft like this: `// Some #includes` or: `-- Don't mind these imports.` I used to think people just didn't understand Knuth's vision. But then I went and looked at his literate programs. Indeed, I've also looked at Knuth's programs and felt a similar twinge. There's the seminal column, available from literateprogramming.com and linked from akkartik's post. Having tried to read it, a big issue there is that the problem just isn't very interesting in my opinion, at least on practical grounds. I've mentioned before that I think a literate programming advocate and true believer would do well to do something like publish a piece that focuses on implementing support for DEFLATE and the ZIP file format, which strikes me as a much stronger hook than a program to "print the k most common words in the file in decreasing frequency" or "Dijkstra's program [that] prints a table of the first thou-sand prime numbers". No matter, though. There are "real" programs published by practitioners of the style, right? How about Knuth's rendition of Colossal Cave Adventure? <http://literateprogramming.com/adventure.pdf http://literateprogramming.com/adventure.pdf> Once again, we're really running into sharp edges of C that WEB—or at least this particular text—doesn't do a good job of softening up, for the reasons akkartik rightly points out. Just look at the second page and right away with `main` we've got unexplained a priori structure, including stuff like local variables j, k, and p. Other programs I've looked into are lcc, and the glimpses I've gotten of Inform 7. On my first encounter with the former, I was motivated to jot down in my notes a proposed law that said, roughly, a poor/failed attempt to write a literate program will produce a piece of text that's harder to understand than if it had just been written "straight". Still, I'm hopeful that there's something there to be found in literate programming. In fact, I'm pretty sure there is something there. I wrote a recent comment soliciting info and critiques/criticism about literate programming for pedagogy <https://news.ycombinator.com/item?id=25498685 https://news.ycombinator.com/item?id=25498685>. And it's funny that you mention TypeScript. As I said, I think part of the inadequancy of CWEB comes down to C. C is fundamentally opposed to one's attempts to practice top-down programming—one of the first things that "every" programmer learned, back when "every programmer" meant programmers who learn C, is that when you read a C program, you're supposed to start at the bottom and kind-of-sort-of read it backwards. Although I suppose there's an argument to be made that were it not the case for C (or Pascal) inflicting this upon you, then Knuth may never have been sufficiently motivated to undertake his literate programming crusade in the first place. I've had niggling thoughts that a language like JS with support for forward declarations in the form of function hoisting and its dynamism at runtime, including allowance for function redefinition (or at least if not JS, then something with similar properties) would be a more perfect match for the top-down style associated with literate programming. TypeScript, though, might be a little too strict that causes it to suffer for similar reasons as C. Also the fact that JS's roots lie in a (much maligned) system for conveying documents (as opposed to an application platform) makes the fit even better. It's just that nobody seems to be noticing this feature-not-a-bug, because they've been focused on trying to recreate mobile app conventions in the browser. And I've actually sat down to experiment and play around with the affordances that could prove TBL's Web (as opposed to Knuth's WEB) a more attractive substrate for a literate programming system. More recently, I've gotten more serious about actually testing the limits of the medium while thinking of things to put together that are meant to be consumed by other people as pieces of written works to be read from top to bottom in Knuthian style, but it comes down to a matter of having enough time. A final note is that in Bob Nystrom's post on 'Crafting "Crafting Interpreters"' <http://journal.stuffwithstuff.com/2020/04/05/crafting-crafting-interpreters/ http://journal.stuffwithstuff.com/2020/04/05/crafting-crafti...>, he more or less lays out the case for a literate programming system to help with actual book authoring. I hope that if you or anyone else decides to take an interest and try hashing something out that you're successful.

jacquesm 6y ago

Look into the Leo programming editor, if this interests you. It's an elegant solution, it doesn't scale well to teams and it's a bit quirky but I think it embodies the concept very well.

cafard 6y ago

Something over 25 years ago, I downloaded and installed some program for Solaris 2.x that was written in CWEB. I wish I could remember what the program was. Oh, and thanks for the link to the PBR book.