4 ms·
My problem with notebooks is that I feel like the natural mental model for them is a spreadsheet mental model, not a REPL mental model. Under that assumption, c
by fryguy 7y ago
My problem with notebooks is that I feel like the natural mental model for them is a spreadsheet mental model, not a REPL mental model. Under that assumption, changing a calculation in the middle means that all of the cells that depend on that calculation would be updated, but instead you need to go and manually re-run the cells after it that depend on that calculation (or re-run the entire notebook) to see the effect on later things. Keeping track of the internal state of the REPL environment is tricky, and my notebooks have usually just ended up being convenient REPL blocks rather than a useful notebook since that's the workflow it emphasizes.
- WorldMaker 7y agoThat's something that I think Observable [1], in my modest usage, seems to do well. [1] https://observablehq.com/ https://observablehq.com/
- jacobolus 7y agoYep, the real complaint is “dead state”, not out of order execution. Worrying about linear flow per se turns out to be misguided based on lack of imagination for/experience with a better model: reactive re-rendering of dependent cells. Observable entirely solves the dead state problem, in a much more effective way than just guaranteeing linear flow would do. * * * More generally, Observable solves or at least ameliorates every item in the linked article’s list of complaints. (In 2020, any survey about modern notebook environments really should be discussing it.) I found the article quite superficial. More like “water cooler gripes from notebook users we polled” than fundamental problems with or opportunities for notebooks as a cognitive tool. I think you could have learned more or less the same thing from going to whatever online forum Jupyter users write their complaints at and skimming the discussion for a couple weeks. I guess this might be the best we can hope for from the results of a questionnaire like this. But it seems crazy having an article about notebook UI which makes no mention of spreadsheets, literate programming, Mathematica, REPLs, Bret Victor’s work, etc. From the title I was hoping for something more thoughtful and insightful.
- SiempreViernes 7y agoYou can get a jupyter extension[1] that allows you to add tags and dependencies and this way construct the dependency graph as you go along. Of course, you have to do it manually and the interface is a bit clunky, but it does what it says. In practice I think taking care not to accidentally shadow variables is much more important: this dependency business only makes sense once you have a clear idea of what you need and by that point you are mostly done anyway. [1] https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/execution_dependencies/README.html https://jupyter-contrib-nbextensions.readthedocs.io/en/lates...
- jacobolus 7y agoI don’t understand what you are trying to say in your second paragraph, but I highly recommend you spend a few weeks playing with http://observablehq.com http://observablehq.com instead of speculating about the differences. In practice, I find it to be dramatically better than previous notebook environments for data analysis, exploratory programming / computational research, prototyping, data visualization, and writing/reading interactive documents (blog posts, software library documentation, expository papers ...). It has a lower barrier to starting new projects, a lower-friction flow throughout I find it better at every stage of my thinking process from blank page up through final code/document, and would recommend it vs. Jupyter or Matlab or Mathematica in every case unless some specific software library is needed which is unavailable in Javascript. The only other tool I really need is pen and paper, though I also use http://desmos.com/calculator http://desmos.com/calculator and Photoshop a fair bit.
- etangent 7y agoI keep seeing the complaint about state and I am getting sick of it. JUST HIT "RESTART KERNEL AND RERUN ALL CELLS," HOW HARD CAN IT BE?
- tobmlt 7y agoThat’s what I’d always do. On more complex notebooks, though, is it possible that isn’t a solution? I wouldn’t think so but I am happy to be surprised. Then again I use notebooks only at the end of a project to present work in “executable presentation” style. Restart and Rerun all has been always been sufficient for me. More generally, I took a look at notebooks, thought, “Why develop with all the extra baggage” and left it at that until ready to experiment with presentation methods for (tight) core ideas.
- gdy 7y agoIf you have a few long data loading and preprocessing steps it's a pain to wait for them to run again, people try to avoid it. When something odd begins to happen, they don't immediately consider the possibility that it's not their bug and waste time trying to 'debug' the problem instead of just rerunning the notebook.
- playing_colours 7y agoWould it be a solution to store intermediate computations to an in-memory or disk database like Redis, SQLite? It is a matter of few minutes to run a docker instance and write simple read / write + serialize Python util functions?
- gdy 7y agoSurely, it would be a solution, but I don't think for an average data scientist it's a matter of few minutes.
- etangent 7y agoYou don't reload every time you write a line of code. Nobody's insane like that. You reload every two hours or so. This is good enough for most except most extreme data sets.
- tel 7y agoThis falls apart when computation is a factor, though. You can't recompute the whole notebook on every commit when there are 30 cells that each take 2-8 seconds to complete.