4 ms·

TeX and Literate Programming (and Lisp) are my fundamental, day-to-day tools. Code it, explain it, generate a Literate PDF containing the code. The programmin

by daly 4y ago

TeX and Literate Programming (and Lisp) are my fundamental, day-to-day tools. Code it, explain it, generate a Literate PDF containing the code. The programming cycle is simple. Write the code in a latex block. Run make. The makefile extracts the code from the latex, compiles it, runs the test cases (also in the latex), and regenerates the PDF. Code and explanations are always up to date and in sync. I have found no better toolset.

muds 4y ago
I've been struggling with keeping track of research experiments and code at the same time. This seems pretty cool! I like how this method is language agnostic and uses "matured" tools. Question: I'd love to give this a try; do you have any public code snippets?
- daly 4y ago
  https://en.wikipedia.org/wiki/Axiom_(computer_algebra_system) https://en.wikipedia.org/wiki/Axiom_(computer_algebra_system... This is (most of) the source code for Axiom. The code is extracted from the latex for these pdfs.
  - dcuthbertson 4y ago
    FYI: axiom-developer.org/ seems to be down
skippp 4y ago
You may in interested to learn then, that for the past 10 months I have (slowly) been working on a LaTeX package that does exactly this [1]. It writes all the text from between come \begin{code} blocks out to a file, as well as use minted/listings to highlight it in the generated pdf. (exactly what you described but in LaTeX package form). I am currently a physics student and one of the main problems that I have day-to-day coding is being unable to include diagrams/equations/pictures in the source code. Moreover, because there is a fair amount of 'domain complexity' the comments are often longer than the code itself. The package is still very much unstable and not even close to being ready for any sort production. However, I did use it for my masters thesis [2], and it was quite an interesting (in a good way) experience. (I recommend everyone to actually try to write literate code at least once). [1] https://github.com/Skipp1/fortex https://github.com/Skipp1/fortex [2] https://github.com/Skipp1/mono-rich https://github.com/Skipp1/mono-rich
kkfx 4y ago
Not really a programmer (sysadmin) but I've tried the literate way with org-mode for my stuff and I use LaTeX for all "pretty docs" needs, however while for document production LaTeX excel literate programming AS A MERE CONCEPT for me does not work that much: - simple things became uselessly lengthy and prolix - complex things became too big to be easy read, while the equivalent "pure-code with comments" and a small new developers doc while fail to give an equally deep knowledge of the code base is FAR more quick and easy accessed. Maybe it's my style, I do not know but while I do not feel LaTeX as painful, since most my docs are of the same kind so a template/class produced once with calm get reused issueless, I can't really digest literate programming...
mirekrusin 4y ago
With pluto/jupyter/observable you can make it live.
taeric 4y ago
I'm curious what literate style you use? I have tried getting into the style a few times, but I often find I'm not taking advantage of the full scramble of the code that is often possible.
daly 4y ago
The literate macro is just standard latex. The command line extraction functions are: Lisp program to extract latex chunks: https://github.com/daly/axiom/blob/master/books/tangle.lisp https://github.com/daly/axiom/blob/master/books/tangle.lisp C program to extract latex chunks: https://github.com/daly/axiom/blob/master/books/tanglec.c https://github.com/daly/axiom/blob/master/books/tanglec.c Note that the C program is just a hand translation of the Lisp code. The lisp code has an explanation and the necessary latex macros. The idea is to scan the latex, find each named code 'chunk', and add each one to a hash table. Then the hash table is scanned to dump the requested chunk to stdout. For example: \begin{chunk}{part1} code for part 1 \end{chunk} Ordinary latex code between chunks. \begin{chunk}{part2} code for part 2 \end{chunk} \begin{chunk}{part1} this code will be appended to the prior chunk \end{chunk} \begin{chunk}{getall} \getchunk{part1} \getchunk{part2} \end{chunk} Assuming the above chunks are in the file foo.tex then tanglec foo.tex getall >getall.code will print out the named chunk (getall). For an individual chunk use tanglec foo.tex part2 >getsome.code
contrapunctus 4y ago
You may be interested in using Org [1], in conjunction with literate-lisp [2] (for Common Lisp) and literate-elisp [3] (for Emacs Lisp). Org provides various outlining commands (among other things), letting you view your program at different levels of granularity. literate-lisp/elisp advise the Lisp reader so the Org file can be loaded and/or compiled directly, without requiring tangling. Consequently, tools like xref will jump to the source block in your Org file, rather than the tangled source. I wrote a little hack/guide to extend this to errors raised in Emacs' compilation-mode. [4] (Unfortunately, the package that lets each Org source block behave as though it was using the corresponding language's Emacs major mode - poly-org-mode - has a ton of bugs. It was part of why I stopped using literate programming entirely for later projects.) [1] https://orgmode.org/ https://orgmode.org/ [2] https://github.com/jingtaozf/literate-lisp https://github.com/jingtaozf/literate-lisp [3] https://github.com/jingtaozf/literate-elisp https://github.com/jingtaozf/literate-elisp [4] https://contrapunctus.codeberg.page/blog/literate-programming-2.html https://contrapunctus.codeberg.page/blog/literate-programmin...
- mark_l_watson 4y ago
  I work with the author of your link [2], and his literate programming documents for Common Lisp are awesome. Watching him during Zoom screen-shares is fun. I am not currently a user of literate-lisp but it is on my non-urgent to-do list.
- daly 4y ago
  I should have mentioned that emacs is also a fundamental tool for all of my work. But since everyone, everywhere uses emacs that would be stating the obvious :-) I tried orgmode. I even attended a course at CMU that used it for the "live notes". It is excellent for teaching. But it has the same flaw as the "live notebook" idea. There is no generally accepted structure to the approach. Everyone "understands" books. They have a preface, chapters, an index, a bibliography, pictures, credits, and a table of contents. Literate programs leverage that shared understanding of the structure. Think of a physics textbook. If you just copy every equation out of the book then you "have the code". All the rest is explanation. They belong together so the explanation and equations (code) are intermingled. I'm a "primitivist". I work in straight text at an emacs buffer in fundamental mode. The point of my code is to "talk to the machine". The point of my literate program is to "explain to other programmers (mostly 'future' me)". Note that this is NOT DOCUMENTATION. It is explanation, best presented in book form.
mark_l_watson 4y ago
About 12 years ago, I used LaTex to write a semantic web book, but in 2 editions: Common Lisp and Java. It was very cool to have 2/3 of the manuscript common text, and about 1/3 unique to the programming language. For the last two evenings, I have been revisiting my old manuscript materials because I am thinking of updating the material and also creating additional editions for more programming languages: Python, JavaScript, and maybe Swift and Prolog. I had forgot how cool TeX and LaTex are. It was very easy to start working with again, even after a 12 year gap.

by daly 4y ago

muds 4y ago

I've been struggling with keeping track of research experiments and code at the same time. This seems pretty cool! I like how this method is language agnostic and uses "matured" tools. Question: I'd love to give this a try; do you have any public code snippets?

daly 4y ago
https://en.wikipedia.org/wiki/Axiom_(computer_algebra_system) https://en.wikipedia.org/wiki/Axiom_(computer_algebra_system... This is (most of) the source code for Axiom. The code is extracted from the latex for these pdfs.
- dcuthbertson 4y ago
  FYI: axiom-developer.org/ seems to be down

skippp 4y ago

You may in interested to learn then, that for the past 10 months I have (slowly) been working on a LaTeX package that does exactly this [1]. It writes all the text from between come \begin{code} blocks out to a file, as well as use minted/listings to highlight it in the generated pdf. (exactly what you described but in LaTeX package form). I am currently a physics student and one of the main problems that I have day-to-day coding is being unable to include diagrams/equations/pictures in the source code. Moreover, because there is a fair amount of 'domain complexity' the comments are often longer than the code itself. The package is still very much unstable and not even close to being ready for any sort production. However, I did use it for my masters thesis [2], and it was quite an interesting (in a good way) experience. (I recommend everyone to actually try to write literate code at least once). [1] https://github.com/Skipp1/fortex https://github.com/Skipp1/fortex [2] https://github.com/Skipp1/mono-rich https://github.com/Skipp1/mono-rich

kkfx 4y ago

Not really a programmer (sysadmin) but I've tried the literate way with org-mode for my stuff and I use LaTeX for all "pretty docs" needs, however while for document production LaTeX excel literate programming AS A MERE CONCEPT for me does not work that much: - simple things became uselessly lengthy and prolix - complex things became too big to be easy read, while the equivalent "pure-code with comments" and a small new developers doc while fail to give an equally deep knowledge of the code base is FAR more quick and easy accessed. Maybe it's my style, I do not know but while I do not feel LaTeX as painful, since most my docs are of the same kind so a template/class produced once with calm get reused issueless, I can't really digest literate programming...

mirekrusin 4y ago

With pluto/jupyter/observable you can make it live.

taeric 4y ago

I'm curious what literate style you use? I have tried getting into the style a few times, but I often find I'm not taking advantage of the full scramble of the code that is often possible.

daly 4y ago

The literate macro is just standard latex. The command line extraction functions are: Lisp program to extract latex chunks: https://github.com/daly/axiom/blob/master/books/tangle.lisp https://github.com/daly/axiom/blob/master/books/tangle.lisp C program to extract latex chunks: https://github.com/daly/axiom/blob/master/books/tanglec.c https://github.com/daly/axiom/blob/master/books/tanglec.c Note that the C program is just a hand translation of the Lisp code. The lisp code has an explanation and the necessary latex macros. The idea is to scan the latex, find each named code 'chunk', and add each one to a hash table. Then the hash table is scanned to dump the requested chunk to stdout. For example: \begin{chunk}{part1} code for part 1 \end{chunk} Ordinary latex code between chunks. \begin{chunk}{part2} code for part 2 \end{chunk} \begin{chunk}{part1} this code will be appended to the prior chunk \end{chunk} \begin{chunk}{getall} \getchunk{part1} \getchunk{part2} \end{chunk} Assuming the above chunks are in the file foo.tex then tanglec foo.tex getall >getall.code will print out the named chunk (getall). For an individual chunk use tanglec foo.tex part2 >getsome.code

contrapunctus 4y ago

You may be interested in using Org [1], in conjunction with literate-lisp [2] (for Common Lisp) and literate-elisp [3] (for Emacs Lisp). Org provides various outlining commands (among other things), letting you view your program at different levels of granularity. literate-lisp/elisp advise the Lisp reader so the Org file can be loaded and/or compiled directly, without requiring tangling. Consequently, tools like xref will jump to the source block in your Org file, rather than the tangled source. I wrote a little hack/guide to extend this to errors raised in Emacs' compilation-mode. [4] (Unfortunately, the package that lets each Org source block behave as though it was using the corresponding language's Emacs major mode - poly-org-mode - has a ton of bugs. It was part of why I stopped using literate programming entirely for later projects.) [1] https://orgmode.org/ https://orgmode.org/ [2] https://github.com/jingtaozf/literate-lisp https://github.com/jingtaozf/literate-lisp [3] https://github.com/jingtaozf/literate-elisp https://github.com/jingtaozf/literate-elisp [4] https://contrapunctus.codeberg.page/blog/literate-programming-2.html https://contrapunctus.codeberg.page/blog/literate-programmin...

mark_l_watson 4y ago
I work with the author of your link [2], and his literate programming documents for Common Lisp are awesome. Watching him during Zoom screen-shares is fun. I am not currently a user of literate-lisp but it is on my non-urgent to-do list.
daly 4y ago
I should have mentioned that emacs is also a fundamental tool for all of my work. But since everyone, everywhere uses emacs that would be stating the obvious :-) I tried orgmode. I even attended a course at CMU that used it for the "live notes". It is excellent for teaching. But it has the same flaw as the "live notebook" idea. There is no generally accepted structure to the approach. Everyone "understands" books. They have a preface, chapters, an index, a bibliography, pictures, credits, and a table of contents. Literate programs leverage that shared understanding of the structure. Think of a physics textbook. If you just copy every equation out of the book then you "have the code". All the rest is explanation. They belong together so the explanation and equations (code) are intermingled. I'm a "primitivist". I work in straight text at an emacs buffer in fundamental mode. The point of my code is to "talk to the machine". The point of my literate program is to "explain to other programmers (mostly 'future' me)". Note that this is NOT DOCUMENTATION. It is explanation, best presented in book form.

mark_l_watson 4y ago

About 12 years ago, I used LaTex to write a semantic web book, but in 2 editions: Common Lisp and Java. It was very cool to have 2/3 of the manuscript common text, and about 1/3 unique to the programming language. For the last two evenings, I have been revisiting my old manuscript materials because I am thinking of updating the material and also creating additional editions for more programming languages: Python, JavaScript, and maybe Swift and Prolog. I had forgot how cool TeX and LaTex are. It was very easy to start working with again, even after a 12 year gap.