4 ms·

Does anybody know a widely used and open source project using literate programming? (Edit: please include only programs expressed "as a web of ideas" that Knuth

Does anybody know a widely used and open source project using literate programming? (Edit: please include only programs expressed "as a web of ideas" that Knuth mentions - apparently otherwise it's not literate programming, just loads of inline documentation?) From what I've seen, literate programming is great in theory and worse than useless in practice. It's so far away from reality. I have had the misfortune of working on a FunnelWeb codebase, and it was one of the most incomprehensible codebases I've ever seen. Preprocessing the code is an awful step in any language. Somehow, the "not documentation" still goes out of date. There are several other problems, explained at [0] better than I can in a comment. Specifically about this project: While I enjoy writing blogs in markdown because of its constraints, reStructuredText (reST) is more powerful/expressive (by design). Because you can easily express semantics/relationships, the resulting links in the generated documentation make the "web" of source code easier to navigate - something Knuth was trying to do with literate programming. [0] http://akkartik.name/post/literate-programming http://akkartik.name/post/literate-programming

mzs 10y ago
https://www.ctan.org/pkg/web https://www.ctan.org/pkg/web
bpicolo 10y ago
Literate programming, to me, feels like commenting on steroids. But that's not necessarily a good thing. A big philosophy right now, and one that I share, is that your goal should be to make code that can be easily understood without comments through expressive code. Literate programming goes in the opposite direction, and I don't think that's necessarily good, though it has it's use cases. I think data science with Jupyter notebooks are a great display of when it's good (and that's likely the best / most used form of literate programming out there), because in data science a lot of what you need to justify is the "why" a calculation is done the way it's done. Pretty graphs inline are just a bonus. For ordinary code though? Comments are a fallback for complicated bits, but I feel like they shouldn't be the norm. I want people to write code that doesn't need hand holding just to understand.
- dwc 10y ago
  I once build an acceptance testing suite using literate programming. I had been handed a nice write-up by our resident physicist with all the maths for how things should work and I didn't want to lose that, so we cobbled something together that used latex math, had embedded test code, etc. These documents then served both as spec and as tests. IMO, this was a terrific use for literate programming. But we did not adopt literate programming as an overall approach, pretty much for the reasons you mention. If your end goal is to produce some kind of largely static artifact then literate programming is fine. For everyday code where you iterate and refactor all the time, literate techniques have all the downsides of prolific comments.
jasikpark 10y ago
https://witheve.com https://witheve.com ?
- guitarbill 10y ago
  connections refused on https. http works. read through http://witheve.com/deepdives/literate.html http://witheve.com/deepdives/literate.html. instead of fencing comments, code is fenced. they even admit they don't have a macro language, so isn't this just inline markdown documentation? it would seem that without macros, you don't get the webs of macros, or whatever knuth thought differentiated literal programming from documentation.
bandrami 10y ago
Isn't the original TeX literate?
- guitarbill 10y ago
  yes, you're right. it's most certainly widely used, and also makes use of the web of macros concept. source code is worth a look: https://www.tug.org/texlive/devsrc/Build/source/texk/web2c/tex.web https://www.tug.org/texlive/devsrc/Build/source/texk/web2c/t...
steveklabnik 10y ago
https://github.com/jameysharp/corrode/ https://github.com/jameysharp/corrode/
AndrewOMartin 10y ago
I've used noweb for a few projects at my work, and my PhD. Even without actually writing comments, the structure of the code has enabled me to return to a project six months after giving up on it and pick it up again really quickly. For wahtever reason, I don't recall that happening with any of my non-literate projects. I've found it really useful for a big Django project, when I add some functionality, the definition of the models, views, dependencies and URL references are all adjacent in a nicely indicated section and subsections. Navigating around it in a PDF reader has been more pleasent than scrolling up and down code, and hunting for the appropriate file, as I would often find myself doing in Eclipse. I'm also able to put output of the tests into the doc, and a full set of installation instructions from a fresh machine, and usage instructions, in the same doc with pointers to the appropriate code chunk. Much of this can be achieved in various other ways. But the actual result is that I've been much happier in my last few literate projects, than I was in the ones before that. Finally, I've also noticed that I don't have the encroaching feeling of my current project getting a bit big and unwieldy even though this is easily the most complicated thing I've ever done.
- guitarbill 10y ago
  I think Mathematica/IPython aka. Juypter notebooks are great in this respect. I used them for my BSc and MSc. However, they aren't complex programs, and portability is an issue. But according to Knuth, literate programming is about more than online code, it also includes the whole web-o'-macros thing. > I'm also able to put output of the tests into the doc, and a full set of installation instructions from a fresh machine, and usage instructions, in the same doc with pointers to the appropriate code chunk. Python has ways of doing this, and all of this just feels like overloading a document. Repos exist for this, and work fine keeping everything together.
cormacrelf 10y ago
Corrode, a C to Rust translator. I wouldn't say "widely used", but within the Rust community it's well known. Literate style seems to work well for code that's difficult to approach (how would you start building this?), and doesn't need frequent refactoring. https://github.com/jameysharp/corrode/blob/master/src/Language/Rust/Corrode/C.md https://github.com/jameysharp/corrode/blob/master/src/Langua...
- guitarbill 10y ago
  I'd say that's a good example. It doesn't use macros, however I do like this stackoverflow discussion [0] on if Literal Haskell even needs macros, because it's already a flexible language. Definitely a new perspective. [0] https://stackoverflow.com/questions/16869050/is-literate-programming-in-haskell-really-literate-programming https://stackoverflow.com/questions/16869050/is-literate-pro...
Cacti 10y ago
org-mode with babel (org-babel) can be used for just about any code base these days.
bootload 10y ago
"Does anybody know a widely used and open source project using literate programming? (Edit: please include only programs expressed "as a web of ideas" that Knuth mentions - apparently otherwise it's not literate programming, just loads of inline documentation?)" Funny you ask this, I ran across a @raganwald post mentioning a great example of ^literate programming^ (mixing code and documentation), "Literate CoffeeScript" [0]. You can read about how CoffeeScript does this at http://coffeescript.org http://coffeescript.org [0] blog post: http://raganwald.com/2013/03/27/literate-coffeescript.coffee.html http://raganwald.com/2013/03/27/literate-coffeescript.coffee... [1] raw text: https://raw.githubusercontent.com/raganwald/raganwald.github.com/master/_posts/2013-02-27-literate-coffeescript.litcoffee.md https://raw.githubusercontent.com/raganwald/raganwald.github...

mzs 10y ago

https://www.ctan.org/pkg/web https://www.ctan.org/pkg/web

bpicolo 10y ago

Literate programming, to me, feels like commenting on steroids. But that's not necessarily a good thing. A big philosophy right now, and one that I share, is that your goal should be to make code that can be easily understood without comments through expressive code. Literate programming goes in the opposite direction, and I don't think that's necessarily good, though it has it's use cases. I think data science with Jupyter notebooks are a great display of when it's good (and that's likely the best / most used form of literate programming out there), because in data science a lot of what you need to justify is the "why" a calculation is done the way it's done. Pretty graphs inline are just a bonus. For ordinary code though? Comments are a fallback for complicated bits, but I feel like they shouldn't be the norm. I want people to write code that doesn't need hand holding just to understand.

dwc 10y ago
I once build an acceptance testing suite using literate programming. I had been handed a nice write-up by our resident physicist with all the maths for how things should work and I didn't want to lose that, so we cobbled something together that used latex math, had embedded test code, etc. These documents then served both as spec and as tests. IMO, this was a terrific use for literate programming. But we did not adopt literate programming as an overall approach, pretty much for the reasons you mention. If your end goal is to produce some kind of largely static artifact then literate programming is fine. For everyday code where you iterate and refactor all the time, literate techniques have all the downsides of prolific comments.

jasikpark 10y ago

https://witheve.com https://witheve.com ?

guitarbill 10y ago
connections refused on https. http works. read through http://witheve.com/deepdives/literate.html http://witheve.com/deepdives/literate.html. instead of fencing comments, code is fenced. they even admit they don't have a macro language, so isn't this just inline markdown documentation? it would seem that without macros, you don't get the webs of macros, or whatever knuth thought differentiated literal programming from documentation.

bandrami 10y ago

Isn't the original TeX literate?

guitarbill 10y ago
yes, you're right. it's most certainly widely used, and also makes use of the web of macros concept. source code is worth a look: https://www.tug.org/texlive/devsrc/Build/source/texk/web2c/tex.web https://www.tug.org/texlive/devsrc/Build/source/texk/web2c/t...

steveklabnik 10y ago

https://github.com/jameysharp/corrode/ https://github.com/jameysharp/corrode/

AndrewOMartin 10y ago

I've used noweb for a few projects at my work, and my PhD. Even without actually writing comments, the structure of the code has enabled me to return to a project six months after giving up on it and pick it up again really quickly. For wahtever reason, I don't recall that happening with any of my non-literate projects. I've found it really useful for a big Django project, when I add some functionality, the definition of the models, views, dependencies and URL references are all adjacent in a nicely indicated section and subsections. Navigating around it in a PDF reader has been more pleasent than scrolling up and down code, and hunting for the appropriate file, as I would often find myself doing in Eclipse. I'm also able to put output of the tests into the doc, and a full set of installation instructions from a fresh machine, and usage instructions, in the same doc with pointers to the appropriate code chunk. Much of this can be achieved in various other ways. But the actual result is that I've been much happier in my last few literate projects, than I was in the ones before that. Finally, I've also noticed that I don't have the encroaching feeling of my current project getting a bit big and unwieldy even though this is easily the most complicated thing I've ever done.

guitarbill 10y ago
I think Mathematica/IPython aka. Juypter notebooks are great in this respect. I used them for my BSc and MSc. However, they aren't complex programs, and portability is an issue. But according to Knuth, literate programming is about more than online code, it also includes the whole web-o'-macros thing. > I'm also able to put output of the tests into the doc, and a full set of installation instructions from a fresh machine, and usage instructions, in the same doc with pointers to the appropriate code chunk. Python has ways of doing this, and all of this just feels like overloading a document. Repos exist for this, and work fine keeping everything together.

cormacrelf 10y ago

Corrode, a C to Rust translator. I wouldn't say "widely used", but within the Rust community it's well known. Literate style seems to work well for code that's difficult to approach (how would you start building this?), and doesn't need frequent refactoring. https://github.com/jameysharp/corrode/blob/master/src/Language/Rust/Corrode/C.md https://github.com/jameysharp/corrode/blob/master/src/Langua...

guitarbill 10y ago
I'd say that's a good example. It doesn't use macros, however I do like this stackoverflow discussion [0] on if Literal Haskell even needs macros, because it's already a flexible language. Definitely a new perspective. [0] https://stackoverflow.com/questions/16869050/is-literate-programming-in-haskell-really-literate-programming https://stackoverflow.com/questions/16869050/is-literate-pro...

Cacti 10y ago

org-mode with babel (org-babel) can be used for just about any code base these days.

bootload 10y ago

"Does anybody know a widely used and open source project using literate programming? (Edit: please include only programs expressed "as a web of ideas" that Knuth mentions - apparently otherwise it's not literate programming, just loads of inline documentation?)" Funny you ask this, I ran across a @raganwald post mentioning a great example of ^literate programming^ (mixing code and documentation), "Literate CoffeeScript" [0]. You can read about how CoffeeScript does this at http://coffeescript.org http://coffeescript.org [0] blog post: http://raganwald.com/2013/03/27/literate-coffeescript.coffee.html http://raganwald.com/2013/03/27/literate-coffeescript.coffee... [1] raw text: https://raw.githubusercontent.com/raganwald/raganwald.github.com/master/_posts/2013-02-27-literate-coffeescript.litcoffee.md https://raw.githubusercontent.com/raganwald/raganwald.github...