5 ms·

Thanks for the time to write up this thoughtful reply. I agree with all your major points about maintaining/sharing notebooks. I don't work in the same collabo

Thanks for the time to write up this thoughtful reply. I agree with all your major points about maintaining/sharing notebooks. I don't work in the same collaborative setting (only 1 or 2 other people on the project), so I don't encounter the same code review issues, but I can certainly imagine. However, I do wonder--do you think notebooks are still valuable intermediate steps towards developing the code you eventually commit, even if the notebooks themselves never see the light of day, or do you think using them at all just leads towards lower-quality code and isn't worth the productivity (in practice)?

mlthoughts2018 8y ago
Trying to use dispassionate code review is a good practice even when it’s a 1-person project. Using e.g. branches in GitHub as logical units of implementation is very important, since it allows you to test incremental changes, group changes into small, conceptually-related batches, and perform code review by inspecting the diff between the branch’s changes and the existing main implementation. The benefits of this are increased in multi-person collaborations, but it’s an important practice all the time, even when working alone on a small project. Notebooks, especially if you have to worry about artificial aspects of the diff (like whether you suppressed cell output) are a pain to work with in this type of code review workflow, even for just one person. The same is true for using automated testing. Many people don’t develop tests for code in a notebook because all along the process they imagine it’s all ad hoc, single-use code, and they arrive at components they wish to reuse in a fragmented and unplanned way. And since it can be hard to create unit tests for functions or cells from a notebook (which is not a module, the language’s standard for how to share an implementation), this means unless you manually convert to a regular source file, then testing is either very shallow (e.g. does some gross-level output of the notebook vaguely appear correct & bug-free?) or else very manually intensive (like manually executing a bunch of function calls or separate cells with different inputs to verify it doesn’t have a bug yourself). In terms of using a notebook as an integrated development environment, no, I don’t find that it’s useful for that and it also does not seem like one of the notebook’s goals (at least it’s design is antithetical to development environments). If I need to rapidly see interactive output, then the Jupyter interactive shell program (among many other interactive interpreters) would be fine, but the notebook (which forces cell-based interaction in a separate browser-driven environment) would not be. Just imagine comparing the developmemt power of using a non-windowed emacs or vim session to write code, then alt-tab to a shell with an interactive prompt to run that code. The productivity loop is much tighter and has less visual clutter than a browser notebook, while letting you simply do plain, boring text editing when writing the code (no cells, no embedded outputs, no need to remember what variable initialization or package imports have to be re-evaluated after you made a code change). And even better, testing can be automated, so you can just invoke a test command after some code changes to get feedback on whether it unexpectedly broke anything— letting you separate your design choices about comprehensiveness of testing from design choices of the code itself. For me, the notebook is like trying to use MS Word after you’ve already gotten past the learning curve of latex. Word feels restricting, visually cluttered, nasty to do proper version control or peer review, by comparison to even intermediate knowledge of latex. The same comparison is analogously useful for the notebook compared to using proper software approaches and shell-centric tools. Of course, Word has its use cases, especially for people who only occasionally write documents, or who write documents that rarely need collaborative edits or historical versioned rollbacks. But for people who are professionally writing documents, there are many much better tools. The same for the notebook.

mlthoughts2018 8y ago

Trying to use dispassionate code review is a good practice even when it’s a 1-person project. Using e.g. branches in GitHub as logical units of implementation is very important, since it allows you to test incremental changes, group changes into small, conceptually-related batches, and perform code review by inspecting the diff between the branch’s changes and the existing main implementation. The benefits of this are increased in multi-person collaborations, but it’s an important practice all the time, even when working alone on a small project. Notebooks, especially if you have to worry about artificial aspects of the diff (like whether you suppressed cell output) are a pain to work with in this type of code review workflow, even for just one person. The same is true for using automated testing. Many people don’t develop tests for code in a notebook because all along the process they imagine it’s all ad hoc, single-use code, and they arrive at components they wish to reuse in a fragmented and unplanned way. And since it can be hard to create unit tests for functions or cells from a notebook (which is not a module, the language’s standard for how to share an implementation), this means unless you manually convert to a regular source file, then testing is either very shallow (e.g. does some gross-level output of the notebook vaguely appear correct & bug-free?) or else very manually intensive (like manually executing a bunch of function calls or separate cells with different inputs to verify it doesn’t have a bug yourself). In terms of using a notebook as an integrated development environment, no, I don’t find that it’s useful for that and it also does not seem like one of the notebook’s goals (at least it’s design is antithetical to development environments). If I need to rapidly see interactive output, then the Jupyter interactive shell program (among many other interactive interpreters) would be fine, but the notebook (which forces cell-based interaction in a separate browser-driven environment) would not be. Just imagine comparing the developmemt power of using a non-windowed emacs or vim session to write code, then alt-tab to a shell with an interactive prompt to run that code. The productivity loop is much tighter and has less visual clutter than a browser notebook, while letting you simply do plain, boring text editing when writing the code (no cells, no embedded outputs, no need to remember what variable initialization or package imports have to be re-evaluated after you made a code change). And even better, testing can be automated, so you can just invoke a test command after some code changes to get feedback on whether it unexpectedly broke anything— letting you separate your design choices about comprehensiveness of testing from design choices of the code itself. For me, the notebook is like trying to use MS Word after you’ve already gotten past the learning curve of latex. Word feels restricting, visually cluttered, nasty to do proper version control or peer review, by comparison to even intermediate knowledge of latex. The same comparison is analogously useful for the notebook compared to using proper software approaches and shell-centric tools. Of course, Word has its use cases, especially for people who only occasionally write documents, or who write documents that rarely need collaborative edits or historical versioned rollbacks. But for people who are professionally writing documents, there are many much better tools. The same for the notebook.