4 ms·

I just happened to be reading Peter Naur's "Programming as theory building" recently. It strikes me that taking its theme even a little seriously helps understa

I just happened to be reading Peter Naur's "Programming as theory building" recently. It strikes me that taking its theme even a little seriously helps understand why notebooks are so popular. Notebooks happen to be convenient tools for exploring a new domain (interactively). Irrespective of how much software purists might complain, conventional software engineering provides very few tools/solutions/practices for that process. The wretched state of interactive debugging (in most languages) is a simple example. As someone who spends a substantial amount of time working with both modes (writing research code in Jupyter notebooks, and writing production code as python modules), notebooks scratch certain itches that IDEs typically don't even come close to. (Some recent progress on add-ons in Javascript-based editors is potentially interesting, because that might help marry the strengths of the two) In my experience, in the evolution of code from Jupyter notebooks to repositories of production code as part of any project, there comes a "right time" to switch from the former to the latter. And this can typically only be learned with experience.

MiroF 7y ago
I just refactor into a module that I import into my notebook as I go along. This lets me use the notebook for quick prototyping, but also productionize faster if need be.
- ssivark 7y ago
  That only works after the code in the module is largely "frozen". It doesn't work well if you're experimenting with ideas inside the module. OTOH, if the algorithm is largely frozen, and you're trying to experiment with its performance on a bunch of examples, the workflow of putting the algorithm in a module and using a notebook to interface with data and visualize results is quite useful. That is basically what I meant by knowing when to transition from one mode to the other. Here's a concrete example (maybe somebody considers this an inspiring challenge?), to illustrate how notebooks are infuriating in their primitiveness, but still better compared to using an editor on source files: Imagine a beginner trying to write/learn a sorting algorithm, and who would like to keep experimenting with their code and observing what happens on examples, possibly profiling space/time complexity along the way. To expand on my point above, there are actually three distinct computational use cases, not just two: Interactive learning -> Sharing insights with others -> Productionizing code.
  - MiroF 7y ago
    Why doesn't work with experimenting with the module? In jupyter, if you're using auto reload then the module will refresh every time you use it.
    - SiempreViernes 7y ago
      I guess the objection is that if you what you are experimenting is inside a module, you've moved the "active" code out of the notebook, and then given up the interactiveness.

MiroF 7y ago

I just refactor into a module that I import into my notebook as I go along. This lets me use the notebook for quick prototyping, but also productionize faster if need be.

ssivark 7y ago
That only works after the code in the module is largely "frozen". It doesn't work well if you're experimenting with ideas inside the module. OTOH, if the algorithm is largely frozen, and you're trying to experiment with its performance on a bunch of examples, the workflow of putting the algorithm in a module and using a notebook to interface with data and visualize results is quite useful. That is basically what I meant by knowing when to transition from one mode to the other. Here's a concrete example (maybe somebody considers this an inspiring challenge?), to illustrate how notebooks are infuriating in their primitiveness, but still better compared to using an editor on source files: Imagine a beginner trying to write/learn a sorting algorithm, and who would like to keep experimenting with their code and observing what happens on examples, possibly profiling space/time complexity along the way. To expand on my point above, there are actually three distinct computational use cases, not just two: Interactive learning -> Sharing insights with others -> Productionizing code.
- MiroF 7y ago
  Why doesn't work with experimenting with the module? In jupyter, if you're using auto reload then the module will refresh every time you use it.
  - SiempreViernes 7y ago
    I guess the objection is that if you what you are experimenting is inside a module, you've moved the "active" code out of the notebook, and then given up the interactiveness.