8 ms·

I recently tried Literate Programming and I've found that it has some downsides not commonly discussed by its advocates. (It also has upsides which are valid, t

I recently tried Literate Programming and I've found that it has some downsides not commonly discussed by its advocates. (It also has upsides which are valid, this post may come across as overly negative because I'm only covering negatives.) * It messes with tooling. If you're lucky then your editor will be smart enough to syntax highlight inside code blocks, or can be taught to do so easily. It's unlikely that more advanced IDE-style features will work. Obviously tools could be adapted with custom plugins but literate programming isn't popular enough for these to already exist as far as I'm aware. * It adds overhead to refactoring. Refactoring code now means editing and rewriting a book, doing a good job requires significantly more effort. If you have designed the program thoroughly ahead of time you can avoid this, but that's not generally the way most programmers work. * The "includes" problem. Most language require you to include/import/require packages used in a file. These are usually all placed at the start. This means a lot of chapters will start with "here are all the includes we'll need" if you're using a linear format. More complex formats that rearrange the code to generate outputs can do a better job but it's still a little clunky. Overall my conclusion was that literate programming works best if you write code like Donald Knuth: work solo, have a good idea of the entire design and a fixed scope which you can complete to declare both book and program "done forever". Unfortunately that doesn't cover most real world programming. I don't think it works well in the most common commercial/programming team style settings.

bradrn 3y ago
> Obviously tools could be adapted with custom plugins but literate programming isn't popular enough for these to already exist as far as I'm aware. I’ve heard that org-babel [https://orgmode.org/worg/org-contrib/babel/intro.html https://orgmode.org/worg/org-contrib/babel/intro.html] works pretty well for this, though I’ve never used it myself.
- tmtvl 3y ago
  Org Babel is really nice, but it doesn't really solve the problem, rather it sidesteps it my giving you the option to edit a block of code in a dedicated buffer. I don't know how well that works with LSP (I don't use LSP) but it does allow you to make full use of SLIME.
  - ParetoOptimal 3y ago
    lsp-mode does support tooling in source code blocks without opening it in a dedicated buffer by creating a virtual one backing it. See https://emacs-lsp.github.io/lsp-mode/manual-language-docs/lsp-org/ https://emacs-lsp.github.io/lsp-mode/manual-language-docs/ls...
thr-nrg 3y ago
>* The "includes" problem. Most language require you to include/import/require packages used in a file. These are usually all placed at the start. This means a lot of chapters will start with "here are all the includes we'll need" if you're using a linear format. More complex formats that rearrange the code to generate outputs can do a better job but it's still a little clunky. Real literate programming, rather than rich text comments, can order the code blocks in any order. You can add then at the very end of the book/chapter/section if you feel like it. I did some literary programs about a previous generation of our program. Every new hire reads them and asks me for copies.
- cxr 3y ago
  > Real literate programming, rather than rich text comments, can order the code blocks in any order. You can add then at the very end of the book/chapter/section if you feel like it. And yet Knuth still does it the other way. <http://akkartik.name/post/literate-programming http://akkartik.name/post/literate-programming>
  - taeric 3y ago
    I think the idea of a general preamble that would have the same explanation, every time, is fine? This is literally the boilerplate concept. The idea is more to break the code into parts you would explain and grok easily. Not every atomic part of the code.
    - cxr 3y ago
      See my response to Jtsummers. <https://news.ycombinator.com/item?id=35989931 https://news.ycombinator.com/item?id=35989931>
  - Jtsummers 3y ago
    Does it matter though? The primary purpose of using literate programming is expository. If Knuth feels that putting the includes at the start makes sense, then that's what he does. If you don't, then fine you move it to the end or the middle. That's the benefit of WEB (and WEB-derived systems). The presentation order is not dependent on the code order, it's dependent on what makes the most expository sense. When I've written literate programs, I almost always shuffle long lists of includes to the end. They add little or no value at the top and distract from the material I want to present. It's also a simple cut and paste to move them back to the top if I wanted to. The only time I leave them at the top is if there's something actually informative about having them at the top, or it's a short program, or I've not actually finished working on it (a lot of my literate programs start as traditional "live in source files" programs that I slurp into org files).
myaccountonhn 3y ago
I use it in a commercial team to write technical documentation. I write it while, for example, exploring an API that I am integrating with. It produces a nice little document with important integration apis, limitations, open questions, integration decisions as well as interactive ways to check certain parts of the integration. I do this with Typescript (deno) and a neovim plugin called sniprun.
bheadmaster 3y ago
> It adds overhead to refactoring. Refactoring code now means editing and rewriting a book In my opinion, this is a good thing. The problem with ordinary refactoring is that it's too easy to just shuffle things around until they work/seem nicer/do whatever you want to. However, with literate programming, you have to think why you're refactoring, and how does it reflect in the explanation.
streakfix 3y ago
their is no tooling for it because no one has tried to build it. all you need is a markdown file with links to the code bookmarks. every modern editor can resolve those links to the actual location of the code
velcrovan 3y ago
> It messes with tooling. If you're lucky then your editor will be smart enough to syntax highlight inside code blocks, or can be taught to do so easily. It's unlikely that more advanced IDE-style features will work. Probably a niche example and the exception that proves the rule, but in Racket, the included literate programming (Scribble/LP2) is itself a language implemented in Racket. Racket’s IDE and tools for exposing and inspecting syntax work just as well in that environment as in any other Racket-implemented language. https://docs.racket-lang.org/scribble/lp.html https://docs.racket-lang.org/scribble/lp.html

bradrn 3y ago

> Obviously tools could be adapted with custom plugins but literate programming isn't popular enough for these to already exist as far as I'm aware. I’ve heard that org-babel [https://orgmode.org/worg/org-contrib/babel/intro.html https://orgmode.org/worg/org-contrib/babel/intro.html] works pretty well for this, though I’ve never used it myself.

tmtvl 3y ago
Org Babel is really nice, but it doesn't really solve the problem, rather it sidesteps it my giving you the option to edit a block of code in a dedicated buffer. I don't know how well that works with LSP (I don't use LSP) but it does allow you to make full use of SLIME.
- ParetoOptimal 3y ago
  lsp-mode does support tooling in source code blocks without opening it in a dedicated buffer by creating a virtual one backing it. See https://emacs-lsp.github.io/lsp-mode/manual-language-docs/lsp-org/ https://emacs-lsp.github.io/lsp-mode/manual-language-docs/ls...

thr-nrg 3y ago

>* The "includes" problem. Most language require you to include/import/require packages used in a file. These are usually all placed at the start. This means a lot of chapters will start with "here are all the includes we'll need" if you're using a linear format. More complex formats that rearrange the code to generate outputs can do a better job but it's still a little clunky. Real literate programming, rather than rich text comments, can order the code blocks in any order. You can add then at the very end of the book/chapter/section if you feel like it. I did some literary programs about a previous generation of our program. Every new hire reads them and asks me for copies.

cxr 3y ago
> Real literate programming, rather than rich text comments, can order the code blocks in any order. You can add then at the very end of the book/chapter/section if you feel like it. And yet Knuth still does it the other way. <http://akkartik.name/post/literate-programming http://akkartik.name/post/literate-programming>
- taeric 3y ago
  I think the idea of a general preamble that would have the same explanation, every time, is fine? This is literally the boilerplate concept. The idea is more to break the code into parts you would explain and grok easily. Not every atomic part of the code.
  - cxr 3y ago
    See my response to Jtsummers. <https://news.ycombinator.com/item?id=35989931 https://news.ycombinator.com/item?id=35989931>
- Jtsummers 3y ago
  Does it matter though? The primary purpose of using literate programming is expository. If Knuth feels that putting the includes at the start makes sense, then that's what he does. If you don't, then fine you move it to the end or the middle. That's the benefit of WEB (and WEB-derived systems). The presentation order is not dependent on the code order, it's dependent on what makes the most expository sense. When I've written literate programs, I almost always shuffle long lists of includes to the end. They add little or no value at the top and distract from the material I want to present. It's also a simple cut and paste to move them back to the top if I wanted to. The only time I leave them at the top is if there's something actually informative about having them at the top, or it's a short program, or I've not actually finished working on it (a lot of my literate programs start as traditional "live in source files" programs that I slurp into org files).

myaccountonhn 3y ago

I use it in a commercial team to write technical documentation. I write it while, for example, exploring an API that I am integrating with. It produces a nice little document with important integration apis, limitations, open questions, integration decisions as well as interactive ways to check certain parts of the integration. I do this with Typescript (deno) and a neovim plugin called sniprun.

bheadmaster 3y ago

> It adds overhead to refactoring. Refactoring code now means editing and rewriting a book In my opinion, this is a good thing. The problem with ordinary refactoring is that it's too easy to just shuffle things around until they work/seem nicer/do whatever you want to. However, with literate programming, you have to think why you're refactoring, and how does it reflect in the explanation.

streakfix 3y ago

their is no tooling for it because no one has tried to build it. all you need is a markdown file with links to the code bookmarks. every modern editor can resolve those links to the actual location of the code

velcrovan 3y ago

> It messes with tooling. If you're lucky then your editor will be smart enough to syntax highlight inside code blocks, or can be taught to do so easily. It's unlikely that more advanced IDE-style features will work. Probably a niche example and the exception that proves the rule, but in Racket, the included literate programming (Scribble/LP2) is itself a language implemented in Racket. Racket’s IDE and tools for exposing and inspecting syntax work just as well in that environment as in any other Racket-implemented language. https://docs.racket-lang.org/scribble/lp.html https://docs.racket-lang.org/scribble/lp.html