4 ms·

I'v always liked the idea of literate programming, but I was always wondering how it actually feels in practice. Do they usual editor/IDE features still work li

by renke1 6y ago

I'v always liked the idea of literate programming, but I was always wondering how it actually feels in practice. Do they usual editor/IDE features still work like in a non-literate environment? – I assume it might work in Emacs, but are there other editors that support this approach? Thinking about it, is Jupyter Notebooks (and the like) a form of literate programming? Thinking even more about it, I think I actually used (or tried to) literate programming in a course about Machine Learning using R and – if I remember correctly – LaTeX to generate the actual application and a PDF.

LeanderK 6y ago
In my view Jupiter notebooks can be, but must not be, a form of literate programming. Many tutorials take the form of a single juypter-notebook exaplaining concepts with code interleaved, which i would argue is a form of literate programming. But my data-exploration notebooks, mostly code and an abundance of plots with sparse comments interleaved, don't really resemble anything i would call literate programming.
- mkl 6y ago
  Do you mean "need not be" rather than "must not be"? "Must not be" means they shouldn't ever be. Most of my Jupyter notebooks are just code, but I've got some with long Latex mathematical derivations of formulas that appear in the code. I would actually like to be able to have that kind of thing in comments within the code itself. I don't do literate programming really though.
  - LeanderK 6y ago
    ah sry, i am german and frequently do this wrong since it's the exact opposite in german.
gorgoiler 6y ago
It was disappointing to me when I recently found out that I couldn’t create Jupyter notebooks with vim. The on-disk format is JSON. It sounds silly, but when there’s a lot of work do in terms of writing the literature part, and then adding the actual code part, I’m much happier with vim than I am with the browser based editor. (This is for producing literate programs for the purpose of teaching others, not for my own benefit.)
- akdor1154 6y ago
  You should look into Jupytext, it might be just what you're after.
d0mine 6y ago
You could use https://github.com/dzop/emacs-jupyter https://github.com/dzop/emacs-jupyter to combine Org mode + Jupyter. Here's an example emacs configuration: (use-package jupyter :after org :config (org-babel-do-load-languages 'org-babel-load-languages '((jupyter . t))) ;; default args for jupyter-python (setq org-babel-default-header-args:jupyter-python '((:results . "replace") (:async . "yes") (:session . "py") (:kernel . "python3")))) After that, you could execute jupyter code block (Org Babel): #+begin_src jupyter-python 1+1 #+end_src #+RESULTS: : 2 You can use latex in org-mode, generate pictures, get nice git diff, export it to pdf, etc.
- amake 6y ago
  Just curious: That code could be run by just plain `#+begin_src python`. What does using Jupyter gain you?
  - d0mine 6y ago
    Nothing inherent to Jupyter (it is not like Jupyter notebooks vs. Python REPL), just some niceties: - low latency (it feels more responsive) - interactive output (tqdm works) - how Python objects are displayed by default Essentially a good implementation of Python code blocks would reimplement half of Jupyter anyway.
BeetleB 6y ago
> I'v always liked the idea of literate programming, but I was always wondering how it actually feels in practice. Do they usual editor/IDE features still work like in a non-literate environment? – I assume it might work in Emacs, but are there other editors that support this approach? In Org Mode, you get all the syntax highlightings and keybindings for whatever language's mode if you do literate programming with it. What you won't get are things like being able to integrate your compilation failures, etc. As an example, if you write something that fails to compile, Emacs will not quickly be able to take you to the line that caused the failure. Similarly, things like linting, etc typically break. Frustratingly, if you get an exception (e.g. "KeyError" in Python), it will not show the result of any print statements, so debugging is a bit painful. I'm sure someone can solve these problems, but I don't know that anyone has.
- sachac 6y ago
  I figured out that if you use the : session keyword, you can still see the prints before the exception. I tend to use something like :session "argh".
jcheng 6y ago
It’s extremely widely used in the R world. The RStudio IDE has extensive tooling for the R Markdown format. https://rmarkdown.rstudio.com/ https://rmarkdown.rstudio.com/ We don’t use the term “literate programming” much anymore, the concept of “reproducible reports” resonates much more with R users. But the experience is much the same. (Disclosure: employed at RStudio)
michaelmrose 6y ago
They can in theory but making it work in practice seems a little harder. For example I tried this https://github.com/jingtaozf/literate-clojure https://github.com/jingtaozf/literate-clojure but ran into this https://github.com/polymode/poly-org/issues/20 https://github.com/polymode/poly-org/issues/20 Either some IDE like functionality doesn't work or syntax highlighting doesn't work after you go through additional complexity compared to simply starting a project.

by renke1 6y ago

LeanderK 6y ago

In my view Jupiter notebooks can be, but must not be, a form of literate programming. Many tutorials take the form of a single juypter-notebook exaplaining concepts with code interleaved, which i would argue is a form of literate programming. But my data-exploration notebooks, mostly code and an abundance of plots with sparse comments interleaved, don't really resemble anything i would call literate programming.

mkl 6y ago
Do you mean "need not be" rather than "must not be"? "Must not be" means they shouldn't ever be. Most of my Jupyter notebooks are just code, but I've got some with long Latex mathematical derivations of formulas that appear in the code. I would actually like to be able to have that kind of thing in comments within the code itself. I don't do literate programming really though.
- LeanderK 6y ago
  ah sry, i am german and frequently do this wrong since it's the exact opposite in german.

gorgoiler 6y ago

It was disappointing to me when I recently found out that I couldn’t create Jupyter notebooks with vim. The on-disk format is JSON. It sounds silly, but when there’s a lot of work do in terms of writing the literature part, and then adding the actual code part, I’m much happier with vim than I am with the browser based editor. (This is for producing literate programs for the purpose of teaching others, not for my own benefit.)

akdor1154 6y ago
You should look into Jupytext, it might be just what you're after.

d0mine 6y ago

You could use https://github.com/dzop/emacs-jupyter https://github.com/dzop/emacs-jupyter to combine Org mode + Jupyter. Here's an example emacs configuration: (use-package jupyter :after org :config (org-babel-do-load-languages 'org-babel-load-languages '((jupyter . t))) ;; default args for jupyter-python (setq org-babel-default-header-args:jupyter-python '((:results . "replace") (:async . "yes") (:session . "py") (:kernel . "python3")))) After that, you could execute jupyter code block (Org Babel): #+begin_src jupyter-python 1+1 #+end_src #+RESULTS: : 2 You can use latex in org-mode, generate pictures, get nice git diff, export it to pdf, etc.

amake 6y ago
Just curious: That code could be run by just plain `#+begin_src python`. What does using Jupyter gain you?
- d0mine 6y ago
  Nothing inherent to Jupyter (it is not like Jupyter notebooks vs. Python REPL), just some niceties: - low latency (it feels more responsive) - interactive output (tqdm works) - how Python objects are displayed by default Essentially a good implementation of Python code blocks would reimplement half of Jupyter anyway.

BeetleB 6y ago

> I'v always liked the idea of literate programming, but I was always wondering how it actually feels in practice. Do they usual editor/IDE features still work like in a non-literate environment? – I assume it might work in Emacs, but are there other editors that support this approach? In Org Mode, you get all the syntax highlightings and keybindings for whatever language's mode if you do literate programming with it. What you won't get are things like being able to integrate your compilation failures, etc. As an example, if you write something that fails to compile, Emacs will not quickly be able to take you to the line that caused the failure. Similarly, things like linting, etc typically break. Frustratingly, if you get an exception (e.g. "KeyError" in Python), it will not show the result of any print statements, so debugging is a bit painful. I'm sure someone can solve these problems, but I don't know that anyone has.

sachac 6y ago
I figured out that if you use the : session keyword, you can still see the prints before the exception. I tend to use something like :session "argh".

jcheng 6y ago

It’s extremely widely used in the R world. The RStudio IDE has extensive tooling for the R Markdown format. https://rmarkdown.rstudio.com/ https://rmarkdown.rstudio.com/ We don’t use the term “literate programming” much anymore, the concept of “reproducible reports” resonates much more with R users. But the experience is much the same. (Disclosure: employed at RStudio)

michaelmrose 6y ago

They can in theory but making it work in practice seems a little harder. For example I tried this https://github.com/jingtaozf/literate-clojure https://github.com/jingtaozf/literate-clojure but ran into this https://github.com/polymode/poly-org/issues/20 https://github.com/polymode/poly-org/issues/20 Either some IDE like functionality doesn't work or syntax highlighting doesn't work after you go through additional complexity compared to simply starting a project.