3 ms·

Notebooks were not created as a way to implement and organize software. You can do that with code files. Jupyter Notebook is a presentation software, for demon

Notebooks were not created as a way to implement and organize software. You can do that with code files. Jupyter Notebook is a presentation software, for demonstrating something to yourself or others. There is hardly anything comparable to build interactive demonstrations. You'd have to implement a (multi-paged?) GUI application, or a web application. Just plain html output may cut it for certain use cases, but still more painful. And I've done more stupid stuff than that. For example I have a demo recording and showing EEG data. And a snake game implemented as a widget with Bokeh output. And live-update graphs from FlightGear. Lately I've started experimenting with controlling my 3D printer from the notebook.

hatmatrix 8y ago
Exactly - it's a classic case of a tool being used for a different purpose ("IDE") than its original intention ("presentation software") and getting criticized for not holding up to misguided expectations. His criticism is valid but the main lesson is to use the appropriate tool for the job. He himself uses VSCode + iPython, which is a better tool more generally for the desired use case of code development.
- kbumsik 8y ago
  Although the OP criticizes missing IDE features but that is not his main points.
  - hatmatrix 8y ago
    But his main points aren't really showstoppers if you're using it as presentation software.
kbumsik 8y ago
Yeah notebooks are fantastic tool to show your works to others. But the problem OP staes is that notebooks encourages unreproducible demonstrations. Markdown + code cells makes data scientists to think a single notebook file is enough to upload to GitHub. Missing dependancy information and etc as OP states make others very hard to reproduce the notebooks. You can see tons of GitHub repos that have notebooks only and impossible to reproduce out of the box.
WorldMaker 8y ago
> Notebooks were not created as a way to implement and organize software. You can do that with code files. It doesn't have to be an either/or. The granddaddy of the modern Notebook, Literate Programming [0] _was_ about implementing and organizing software. Just because today's Notebooks like Jupyter aren't currently sufficient to implement and organize software doesn't mean that they cannot be (again) in the future. A lot of the problems pointed out in the slides are solvable. The DevOps of Notebooks is stuff we can absolutely sink our teeth into as an industry. We can make good Notebook formats that source control well. (Some folks have filters already for Jupyter.) We can make better bridges to (incrementally, per user interest) move Notebooks into source control, CI, testing, etc. Jupyter should already have some idea of the environment it is running in, it could certainly build things like requirements.txt or even full Docker containers. We can build beyond the single cell or single Notebook page and ask deeper questions about how do we organize Notebooks, how do we organize software in Notebooks, how do we interoperate with maybe some code that has a strong narrative to live in a Notebook alongside code that doesn't have a strong narrative or doesn't need one (or wasn't written with one in mind and is legacy code in the project). It could be great to take an existing Python codebase and say "this feature is best explained in a Notebook" and just build it that way. Similarly it could be great to say "this Notebook I found is already a great module, I'm going to build a more traditional app around it" and getting the Notebook's own help in bootstrapping that effort. We certainly have the technology and the opportunities to do interesting software development in/with/alongside Notebooks and Notebook-like tools. The questions are certainly more ones of what are our priorities? Knuth argued in the 80s that all software development was best when embedded in a human-focused narrative. I'm not that extreme myself, but I certainly see some great opportunities for pragmatic middle grounds where you can mix-and-match as opportunity/interest/need warrants. Having Notebooks as a tool in software development _can_ make us better software developers. It's cool to have a lot of tools in your toolbelt so you can pick and choose the best ones for the jobs at hand. [0] https://en.wikipedia.org/wiki/Literate_programming https://en.wikipedia.org/wiki/Literate_programming

hatmatrix 8y ago

Exactly - it's a classic case of a tool being used for a different purpose ("IDE") than its original intention ("presentation software") and getting criticized for not holding up to misguided expectations. His criticism is valid but the main lesson is to use the appropriate tool for the job. He himself uses VSCode + iPython, which is a better tool more generally for the desired use case of code development.

kbumsik 8y ago
Although the OP criticizes missing IDE features but that is not his main points.
- hatmatrix 8y ago
  But his main points aren't really showstoppers if you're using it as presentation software.

kbumsik 8y ago

Yeah notebooks are fantastic tool to show your works to others. But the problem OP staes is that notebooks encourages unreproducible demonstrations. Markdown + code cells makes data scientists to think a single notebook file is enough to upload to GitHub. Missing dependancy information and etc as OP states make others very hard to reproduce the notebooks. You can see tons of GitHub repos that have notebooks only and impossible to reproduce out of the box.

WorldMaker 8y ago

> Notebooks were not created as a way to implement and organize software. You can do that with code files. It doesn't have to be an either/or. The granddaddy of the modern Notebook, Literate Programming [0] _was_ about implementing and organizing software. Just because today's Notebooks like Jupyter aren't currently sufficient to implement and organize software doesn't mean that they cannot be (again) in the future. A lot of the problems pointed out in the slides are solvable. The DevOps of Notebooks is stuff we can absolutely sink our teeth into as an industry. We can make good Notebook formats that source control well. (Some folks have filters already for Jupyter.) We can make better bridges to (incrementally, per user interest) move Notebooks into source control, CI, testing, etc. Jupyter should already have some idea of the environment it is running in, it could certainly build things like requirements.txt or even full Docker containers. We can build beyond the single cell or single Notebook page and ask deeper questions about how do we organize Notebooks, how do we organize software in Notebooks, how do we interoperate with maybe some code that has a strong narrative to live in a Notebook alongside code that doesn't have a strong narrative or doesn't need one (or wasn't written with one in mind and is legacy code in the project). It could be great to take an existing Python codebase and say "this feature is best explained in a Notebook" and just build it that way. Similarly it could be great to say "this Notebook I found is already a great module, I'm going to build a more traditional app around it" and getting the Notebook's own help in bootstrapping that effort. We certainly have the technology and the opportunities to do interesting software development in/with/alongside Notebooks and Notebook-like tools. The questions are certainly more ones of what are our priorities? Knuth argued in the 80s that all software development was best when embedded in a human-focused narrative. I'm not that extreme myself, but I certainly see some great opportunities for pragmatic middle grounds where you can mix-and-match as opportunity/interest/need warrants. Having Notebooks as a tool in software development _can_ make us better software developers. It's cool to have a lot of tools in your toolbelt so you can pick and choose the best ones for the jobs at hand. [0] https://en.wikipedia.org/wiki/Literate_programming https://en.wikipedia.org/wiki/Literate_programming