5 ms·

To me as an org mode and org babel user, notebooks seem to be a very half-assed form or literate programming for the following reasons: (1) No linking from on

To me as an org mode and org babel user, notebooks seem to be a very half-assed form or literate programming for the following reasons: (1) No linking from on cell to another. A sequential execution is assumed or otherwise you need to manually run cells in different order. There might be extensions to help with that. It is not out of the box though. (2) The documentation language is markdown This does not allow for great technical documents with arbitrary links to arbitrary other parts in the document or inside another notebook. ReStructuredText would have that using link sources and link sinks. org mode can also link to parts in other documents. (3) I cannot include other notebooks to have it all exported as one whole document or notebook as I can in org mode. In org mode I can even specify at what level some heading should be imported. (4) The editor in notebooks is not that great compared to any IDE or normal code editor or Emacs. (5) In a typical JupyterLab environment there is only one programming language per notebook. I know there are other kinds JupyterLab environments, which allow for multiple langs inside the same notebook. (6) The base format of notebooms is JSON instead of plain text like in org mode. This means diffs are harder to read and version control. Again you need extra tooling for dealing with it well (which does exist). The literate programming stuff of notebooks has all long been there before notebooks came into existence. It is not like that was the advent of a new form of literate programming. In JupyterLab environments it would take lots of fumbling and tweaking to get something comparable to org + org babel.

funklute 5y ago
R notebooks (which are not jupyter-based) addresses (to varying degrees) issues 3-6. But then you are of course stuck inside the R ecosystem, where only Rstudio and emacs are what I would consider fully mature editors. As a casual rather than power user, I especially dislike the JSON format of jupyter notebooks.
- jonnycomputer 5y ago
  For all the popularity of Jupyter notebooks, I never did understand the appeal. I agree that R notebooks and Emac's org-mode are far more sensible ways of doing things. Not perfect, by any means, but sensible.
  - asoneth 5y ago
    I cannot speak to R, but the Emacs learning curve is the primary reason org-mode is not more popular. There may be a break-even point past which learning Emacs is a net benefit, but few people have the time, skill, and inclination to reach that point.
    - zelphirkalt 5y ago
      It is a good thing, that slowly other editors gain support for org files. I've seen them at least highlighted in VS code recently, when someone else shared their screen, even if that is but a tiny fraction of what support for org files can mean.
ogogmad 5y ago
Org mode and everything Emacs has a crazy steep learning curve. I tried using Org for note-taking with formulas, and I remember having to learn a ton of arcane things like Doom Emacs and endless keyboard bindings (Org + Emacs + Vi). I then just switched back to something easy, where I could fluently get my thoughts down without wasting endless time. Maybe I should give it another go with Spacemacs because I like the idea of Org mode. That might make the learning curve gentler. I hadn't considered that possibility at the time. I don't understand why Emacs's features aren't exposed via context menus. You can always switch to the keyboard later as you get good.
- dangom 5y ago
  Context-menus will land in the next Emacs release and are already available on master. Expect more and more functionality to be exposed via them. Being emacs, context-menus will also be configurable. https://github.com/emacs-mirror/emacs/blob/emacs-28/etc/NEWS#L227 https://github.com/emacs-mirror/emacs/blob/emacs-28/etc/NEWS... https://ruzkuku.com/texts/emacs-mouse.html https://ruzkuku.com/texts/emacs-mouse.html
- funklute 5y ago
  You can activate a menu, but it only covers very basic things. If you are confined to using only the menu, you might equally well use notepad. For base emacs, the learning curve is not as steep as vim. I wonder if your problem was trying to do everything at once? Regardless, emacs really is more of a long-term investment. It will take you years to be fully proficient. Personally, I have gone away from using emacs (including org-mode and vim emulation), because it took too much time to keep the configuration shiny. My main issue with org-mode is that it's only really a first class citizen within emacs. I've not found any vim/vscode/etc. plugins that emulates the experience to satisfactory degree, and that lock-in means I've given up on org-mode.
- zelphirkalt 5y ago
  The learning curve is steep, that is true. I guess one has to invest some learning to get the good stuff. I myself needed 2 attempts at getting into Emacs. One very short one, where I found myself struggling with even saving a buffer, that ended quickly. The other one more serious, investing my free time out of curiosity, what all the Emacs fuss is about. At some later point I discovered org mode and from then on, there was simply no way back for me to something less. Similar for Magit. Or running shells inside Emacs and being able to simply copy anything from a shell buffer, because it is a buffer and not some readonly thingy in a separate window. Very useful. Now I use org mode almost every day and definitely for all kinds of documentation and notes. It might not be perfect out of the box for formula rendering, but even that can be arranged/configured, so that you can switch to rendered formulas inside your buffer and I have done so in the past. I have not tried to use Doom Emacs or Spacemacs or whatever flavour. I started with standard Emacs and went with that and still use that. If Doom Emacs or Spacemacs introduce too many new concepts or make the learning curve too steep, perhaps trying standard Emacs will be a better experience. Not sure what use context menus would have in my personal workflow. Perhaps if they could be opened via keyboard shortcut and then I could select menu items quickly, it would be useful.
leephillips 5y ago
Pluto solves¹ many of these problems, but it’s Julia only. Most importantly: - cells are linked through a dependency graph, so the output you see is independent of the order in which the cells appear; - each notebook is backed by a plain Julia module, so you can use version control, edit in a real editor instead of only in the browser, and import other notebooks; - You can create interaction² using HTML inputs, Julia wrappers over those, or go crazy with your own custom JavaScript controls; - It’s easy to embed LaTeX into the MarkDown, and you can use HTML, so any hyperlinking works. 1 ‘An introduction to pluto’. LWN. Available from: https://lwn.net/Articles/835930/ 2 ‘PlutoUI demonstration’. Available from: https://pluton.lee-phillips.org/sliders/uiDemo.html
- Royi 5y ago
  But unfortunately it doesn't have option to turn off reactivity which makes it not appropriate for intensive computation. I wish it had a sequential mode.
musingsole 5y ago
1) Yeah, the execution count is there, but there isn't a built in way to visualize it other than the default sequential cells. I really wonder if there's a useful way to show a notebook as a nodegraph. 2) I can't understand the desire for RST over Markdown. Between simplicity and adoption, Markdown beats almost all mark-up languages -- maybe even html. 3) You can use the `run` command to link notebook executions and use Markdown linking to link the documentation. 4) This isn't true. I use vim in Jupyter. There's plugins for emacs too. Also, many editors like VSCode can edit notebooks directly. 5) This is a weird complaint. I can imagine a few corner cases where mixing languages in a notebook would be nice...but the concept of a multi-language Jupyter kernel doesn't make much sense from a practical standpoint. 6) There are diff tools such as `nbdime`. Between those and a few git commit hooks to leave your notebooks in a consistent execution state, merging versions notebooks isn't any worse than other files.
- hiptobecubic 5y ago
  I lot of these rebuttals are "I can't imagine your use case so it's probably bad/weird/not worth it."
protoduction 5y ago
You might like Starboard Notebook (https://starboard.gg https://starboard.gg), it's in-browser and mixed multi-language, as well as diffable/version control friendly. (I'm building it). https://starboard.gg https://starboard.gg
sdenton4 5y ago
As others have said, a big draw for notebooks is keeping the bar low. Sequential ordering is intuitive and doesn't require a complicated additional ui for managing the execution graph. In fact, the execution graph logic is handled by the user. When it gets too complex, it's a sign that the notebook itself is too complex, and it's time to move a lot of code into libraries. Notebooks /should/ be conceptually simple. Then again, I bounced off emacs after deciding the complexity didn't actually justify the endpoint: I was already able to write and maintain code with much simpler tools, so why bother? I can use the party of my brain that would otherwise be used for key bindings for other things. So, mileage varies.
- zelphirkalt 5y ago
  To each their own. I want to question though, whether those simpler tools are truly simpler. Lets say you want to use notebooks instead of org mode. You will need to have the following set up, before you can start: - python3 (OK, mostly a given, though not on Windows) - pip and depending on your requirements you might need Poetry or Pipenv or whatever else - Jupyter ecosystem (JupyterLab, Jupyter/notebook server and all their dependencies) - some extensions for JupyterLab. Those could be frontend and backend. If frontend, then you will also need to have NodeJS on your machine, to have NPM, to have tsc to be able to build JupyterLab with the extension installed. Quite a few parts there as well.
  - sdenton4 5y ago
    Especially for analysis tools (as I tend to use colab/Jupiter), the important part of complexity is cognitive load. Installation is a constant, upfront cost in that regard: I have literally never worried about the NPM or nodejs, as they're handled automatically by standard installers... Or already running in shared environments. (And if I'm deeply worried about binary size and dependency depth, I probably shouldn't be working python in the first place, and have other tools I can go to.) Cognitive load when using the notebook is very low compared to emacs. As has been remarked on often, emacs is probably fine if you've been using it for thirty years, but having to keep track of different key strokes for copy paste depending on what window you're in (emacs vs the rest of the world) is pointless cognitive load. The idioms are so sprawling and out of sync with everything else that the editor is an obstacle in the way to doing what I want. And while (historically) that perspective may just paint me as a noob, I'll just say that no amount of memorizing editor functionality or micro optimizing configurations will ever help me prove a theorem. If there's another tool that gets the job done and doesn't want to fight me before/while I use it, then it's the better tool.
mgraczyk 5y ago
These are interesting to me as a notebook user, because a lot of these I see as advantages of the notebook design. (1) This is better to me, because it makes notebooks more consistent and easier to jump into. I read english prose sequentially, why not code? (2) You can do this in all environments I've used with <a> tags. (3) this would be nice, but I'm not sure it's worth the tradeoff because the editing environment wouldn't correspond as well to the exported document. (4) true, but I have a vim plugin that make it better (5) also true, but this is a good thing for simplicity. I don't want to have to keep track of the lexical scope of multiple language environments (6) There are tools for this, but yeah it's not great. The main issue is saving outputs It sounds like the goals of notebooks vs org mode are slightly different, at least the way I use notebooks. For me, notebooks are a tool for sacrificing control and programming flexibility in exchange for easier collaboration and visualization. It's harder to write structured code, but it's much easier to share with a large number of semi-technical people.
- huijzer 5y ago
  > (1) This is better to me, because it makes notebooks more consistent and easier to jump into. I read english prose sequentially, why not code? As a counterexample, some things are not important but have to be mentioned. In English, these things end up in the appendix. In Pluto notebooks, these things can be moved to the bottom. Another benefit of less strict ordering is when debugging. Sometimes cell A fails because of cell B. It can be useful for debugging to temporarily put these cells next to each other
- zelphirkalt 5y ago
  When you can define a cell depending on another cell and can expect that other cell to be run after its dependencies have run, then you can make sure, that a cell will always give the correct result, not a different one, because other cells have already run, which mess up the state. It would allow you to scroll to the end and see the last layer of abstraction, just run that, and implicitly also run all its dependencies (other cells). In a top to bottom typical notebook, you will have to run the notebook from the top. There is a menu item for that, of course. However, what if you also have some cells in there, which do not have to be run? They will waste time unnecessarily or even worse, will change the result. So you will need to take a look at each cell to make sure you really can run everything sequentially. Either the notebook has to be created to make sure, that running top to bottom always works and gives the same result, or you will have to look at everything. When code segments can define their dependence on other code segments, they can be designed to that it does not matter which one you run, it will always give the intended result.

funklute 5y ago

R notebooks (which are not jupyter-based) addresses (to varying degrees) issues 3-6. But then you are of course stuck inside the R ecosystem, where only Rstudio and emacs are what I would consider fully mature editors. As a casual rather than power user, I especially dislike the JSON format of jupyter notebooks.

jonnycomputer 5y ago
For all the popularity of Jupyter notebooks, I never did understand the appeal. I agree that R notebooks and Emac's org-mode are far more sensible ways of doing things. Not perfect, by any means, but sensible.
- asoneth 5y ago
  I cannot speak to R, but the Emacs learning curve is the primary reason org-mode is not more popular. There may be a break-even point past which learning Emacs is a net benefit, but few people have the time, skill, and inclination to reach that point.
  - zelphirkalt 5y ago
    It is a good thing, that slowly other editors gain support for org files. I've seen them at least highlighted in VS code recently, when someone else shared their screen, even if that is but a tiny fraction of what support for org files can mean.

ogogmad 5y ago

Org mode and everything Emacs has a crazy steep learning curve. I tried using Org for note-taking with formulas, and I remember having to learn a ton of arcane things like Doom Emacs and endless keyboard bindings (Org + Emacs + Vi). I then just switched back to something easy, where I could fluently get my thoughts down without wasting endless time. Maybe I should give it another go with Spacemacs because I like the idea of Org mode. That might make the learning curve gentler. I hadn't considered that possibility at the time. I don't understand why Emacs's features aren't exposed via context menus. You can always switch to the keyboard later as you get good.

dangom 5y ago
Context-menus will land in the next Emacs release and are already available on master. Expect more and more functionality to be exposed via them. Being emacs, context-menus will also be configurable. https://github.com/emacs-mirror/emacs/blob/emacs-28/etc/NEWS#L227 https://github.com/emacs-mirror/emacs/blob/emacs-28/etc/NEWS... https://ruzkuku.com/texts/emacs-mouse.html https://ruzkuku.com/texts/emacs-mouse.html
funklute 5y ago
You can activate a menu, but it only covers very basic things. If you are confined to using only the menu, you might equally well use notepad. For base emacs, the learning curve is not as steep as vim. I wonder if your problem was trying to do everything at once? Regardless, emacs really is more of a long-term investment. It will take you years to be fully proficient. Personally, I have gone away from using emacs (including org-mode and vim emulation), because it took too much time to keep the configuration shiny. My main issue with org-mode is that it's only really a first class citizen within emacs. I've not found any vim/vscode/etc. plugins that emulates the experience to satisfactory degree, and that lock-in means I've given up on org-mode.
zelphirkalt 5y ago
The learning curve is steep, that is true. I guess one has to invest some learning to get the good stuff. I myself needed 2 attempts at getting into Emacs. One very short one, where I found myself struggling with even saving a buffer, that ended quickly. The other one more serious, investing my free time out of curiosity, what all the Emacs fuss is about. At some later point I discovered org mode and from then on, there was simply no way back for me to something less. Similar for Magit. Or running shells inside Emacs and being able to simply copy anything from a shell buffer, because it is a buffer and not some readonly thingy in a separate window. Very useful. Now I use org mode almost every day and definitely for all kinds of documentation and notes. It might not be perfect out of the box for formula rendering, but even that can be arranged/configured, so that you can switch to rendered formulas inside your buffer and I have done so in the past. I have not tried to use Doom Emacs or Spacemacs or whatever flavour. I started with standard Emacs and went with that and still use that. If Doom Emacs or Spacemacs introduce too many new concepts or make the learning curve too steep, perhaps trying standard Emacs will be a better experience. Not sure what use context menus would have in my personal workflow. Perhaps if they could be opened via keyboard shortcut and then I could select menu items quickly, it would be useful.

leephillips 5y ago

Pluto solves¹ many of these problems, but it’s Julia only. Most importantly: - cells are linked through a dependency graph, so the output you see is independent of the order in which the cells appear; - each notebook is backed by a plain Julia module, so you can use version control, edit in a real editor instead of only in the browser, and import other notebooks; - You can create interaction² using HTML inputs, Julia wrappers over those, or go crazy with your own custom JavaScript controls; - It’s easy to embed LaTeX into the MarkDown, and you can use HTML, so any hyperlinking works. 1 ‘An introduction to pluto’. LWN. Available from: https://lwn.net/Articles/835930/ 2 ‘PlutoUI demonstration’. Available from: https://pluton.lee-phillips.org/sliders/uiDemo.html

Royi 5y ago
But unfortunately it doesn't have option to turn off reactivity which makes it not appropriate for intensive computation. I wish it had a sequential mode.

musingsole 5y ago

1) Yeah, the execution count is there, but there isn't a built in way to visualize it other than the default sequential cells. I really wonder if there's a useful way to show a notebook as a nodegraph. 2) I can't understand the desire for RST over Markdown. Between simplicity and adoption, Markdown beats almost all mark-up languages -- maybe even html. 3) You can use the `run` command to link notebook executions and use Markdown linking to link the documentation. 4) This isn't true. I use vim in Jupyter. There's plugins for emacs too. Also, many editors like VSCode can edit notebooks directly. 5) This is a weird complaint. I can imagine a few corner cases where mixing languages in a notebook would be nice...but the concept of a multi-language Jupyter kernel doesn't make much sense from a practical standpoint. 6) There are diff tools such as `nbdime`. Between those and a few git commit hooks to leave your notebooks in a consistent execution state, merging versions notebooks isn't any worse than other files.

hiptobecubic 5y ago
I lot of these rebuttals are "I can't imagine your use case so it's probably bad/weird/not worth it."

protoduction 5y ago

You might like Starboard Notebook (https://starboard.gg https://starboard.gg), it's in-browser and mixed multi-language, as well as diffable/version control friendly. (I'm building it). https://starboard.gg https://starboard.gg

sdenton4 5y ago

As others have said, a big draw for notebooks is keeping the bar low. Sequential ordering is intuitive and doesn't require a complicated additional ui for managing the execution graph. In fact, the execution graph logic is handled by the user. When it gets too complex, it's a sign that the notebook itself is too complex, and it's time to move a lot of code into libraries. Notebooks /should/ be conceptually simple. Then again, I bounced off emacs after deciding the complexity didn't actually justify the endpoint: I was already able to write and maintain code with much simpler tools, so why bother? I can use the party of my brain that would otherwise be used for key bindings for other things. So, mileage varies.

zelphirkalt 5y ago
To each their own. I want to question though, whether those simpler tools are truly simpler. Lets say you want to use notebooks instead of org mode. You will need to have the following set up, before you can start: - python3 (OK, mostly a given, though not on Windows) - pip and depending on your requirements you might need Poetry or Pipenv or whatever else - Jupyter ecosystem (JupyterLab, Jupyter/notebook server and all their dependencies) - some extensions for JupyterLab. Those could be frontend and backend. If frontend, then you will also need to have NodeJS on your machine, to have NPM, to have tsc to be able to build JupyterLab with the extension installed. Quite a few parts there as well.
- sdenton4 5y ago
  Especially for analysis tools (as I tend to use colab/Jupiter), the important part of complexity is cognitive load. Installation is a constant, upfront cost in that regard: I have literally never worried about the NPM or nodejs, as they're handled automatically by standard installers... Or already running in shared environments. (And if I'm deeply worried about binary size and dependency depth, I probably shouldn't be working python in the first place, and have other tools I can go to.) Cognitive load when using the notebook is very low compared to emacs. As has been remarked on often, emacs is probably fine if you've been using it for thirty years, but having to keep track of different key strokes for copy paste depending on what window you're in (emacs vs the rest of the world) is pointless cognitive load. The idioms are so sprawling and out of sync with everything else that the editor is an obstacle in the way to doing what I want. And while (historically) that perspective may just paint me as a noob, I'll just say that no amount of memorizing editor functionality or micro optimizing configurations will ever help me prove a theorem. If there's another tool that gets the job done and doesn't want to fight me before/while I use it, then it's the better tool.

mgraczyk 5y ago

These are interesting to me as a notebook user, because a lot of these I see as advantages of the notebook design. (1) This is better to me, because it makes notebooks more consistent and easier to jump into. I read english prose sequentially, why not code? (2) You can do this in all environments I've used with <a> tags. (3) this would be nice, but I'm not sure it's worth the tradeoff because the editing environment wouldn't correspond as well to the exported document. (4) true, but I have a vim plugin that make it better (5) also true, but this is a good thing for simplicity. I don't want to have to keep track of the lexical scope of multiple language environments (6) There are tools for this, but yeah it's not great. The main issue is saving outputs It sounds like the goals of notebooks vs org mode are slightly different, at least the way I use notebooks. For me, notebooks are a tool for sacrificing control and programming flexibility in exchange for easier collaboration and visualization. It's harder to write structured code, but it's much easier to share with a large number of semi-technical people.

huijzer 5y ago
> (1) This is better to me, because it makes notebooks more consistent and easier to jump into. I read english prose sequentially, why not code? As a counterexample, some things are not important but have to be mentioned. In English, these things end up in the appendix. In Pluto notebooks, these things can be moved to the bottom. Another benefit of less strict ordering is when debugging. Sometimes cell A fails because of cell B. It can be useful for debugging to temporarily put these cells next to each other
zelphirkalt 5y ago
When you can define a cell depending on another cell and can expect that other cell to be run after its dependencies have run, then you can make sure, that a cell will always give the correct result, not a different one, because other cells have already run, which mess up the state. It would allow you to scroll to the end and see the last layer of abstraction, just run that, and implicitly also run all its dependencies (other cells). In a top to bottom typical notebook, you will have to run the notebook from the top. There is a menu item for that, of course. However, what if you also have some cells in there, which do not have to be run? They will waste time unnecessarily or even worse, will change the result. So you will need to take a look at each cell to make sure you really can run everything sequentially. Either the notebook has to be created to make sure, that running top to bottom always works and gives the same result, or you will have to look at everything. When code segments can define their dependence on other code segments, they can be designed to that it does not matter which one you run, it will always give the intended result.