14 ms·

I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. So much Python development is trying snippets of code in

I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE. All my Jupyter notebooks are like that, where my code starts as cells of a line or two, as I check each output. Then I coalesce them into a function (which avoid the problem of execution order that some people here mention). Then I may move those functions into a normal Python module and build up complex classes, add unit tests. Or if the goal is communicating with other people the results of running that code, I'll add explanatory text and mathematical formulas, break it up into sections, include references to papers and web links. I think what people miss here is many software development tasks, especially in data science and machine learning, are essentially exploratory data analysis. And the outputs are graphs, tables of data etc, designed to be read by humans. Jupyter's ease of use for iteration, and the ability to interleave documentation, formulas, graphs and tables with the code is a big win. In my daily work, I have my IDE and Jupyter open side by side. The Jupyter notebooks are version controlled like my .py modules. Over time, I expect editors like VSCode to edit Jupyter notebooks natively. And also more code editing to move from IDEs into the Jupyter (via integrating the Monaco editor). We will all benefit from that crosspollination, whether we are principally software engineers or data scientists.

seanmcdirmid 8y ago
> And also more code editing to move from IDEs into the Jupyter Jupyter, and all REPLS in general, rely on a concrete top-level execution context to provide useful feeedback. Much of the code written outside of data science doesn’t really have that, which is why you don’t see notebooks being used for software development very often.
- vosper 8y ago
  I frequently use IPython or REPL.it for software development. It's not my main editor, and I don't write large chunks of my code there, but for rapidly prototyping a small subset of some task I find them invaluable. But you are right about notebooks - I don't find them particularly useful, except as a better REPL, and no-one I know uses them for "normal" (non Data Science) software development.
  - AdamM12 8y ago
    I think my favorite non "data science" use case is web scraping. Ad-hoc nature of the task lends itself to using a notebook. A lot of times I don't want the full suite and boilerplate Scrapy gives you so I write it out in a notebook with Parsel & Requests. Once done export and clean up.
- shawn 8y ago
  I'm not sure this is as limiting as it sounds. You can import whatever modules you want from within a notebook, and most projects tend to be modular. (Those that aren't can usually be shimmed pretty quickly.)
- barrkel 8y ago
  Used all the time in Ruby (repl, not notebook). In fact I think anyone doing Ruby dev who doesn't have a pry session open is missing out on a good 30% productivity boost, unless they know their APIs inside out.
  - canhascodez 8y ago
    I would agree with this, although an IDE with really good introspection and an integrated debugger can approximate the experience. Lately I've started using spaceneovim, which is hopefully going to help with the back-and-forth workflow a bit, modulo issues of what-mode-am-I-in-again? The terminal is two keystrokes away, and having all of the normal vim commands available in both contexts is something that I think will be more and more useful.
jorgemf 8y ago
Jupyter notebooks are for data science, mostly because visualization is required. Something you do once, report it and it is done. Itsn't make sense to use Jupyter for other stuff. It doesn't make much sense to use them for training big models of deep learning because there are better tools for that.
- rmetzler 8y ago
  What tools can you recommend for machine learning?
  - jorgemf 8y ago
    IT depends what you want to do
  - cirgue 8y ago
    Scripts to train the model, Jupiter to do your analysis. This is, AFIK, what most ML shops do if they’re using python.
- mertd 8y ago
  > Something you do once, report it and it is done. I have been having a hard time getting comfortable with that idea. There is usually "a lot of" untested and unreviewed code in the notebook that produces the analysis.
  - jorgemf 8y ago
    Well, there are many things to consider: - Code is small, most of it is calling well-know libraries with the algorithms. - Every 3-4 lines of code you usually show the results of what is happening (either how the data changed, a graph or whatever) - 95% of the errors in data science come from bad practices: you either didn't clean up the data correctly, you didn't split the train/test/validation sets correctly or at the right step in the process, you chosed the wrong algorithm, etc - most of the time you don't know whether you are doing something wrong or not because you don't have enough knowledge, but the code shows something all the time. No bugs in the code. Those are the reason I don't care very much about bugs in the code because usually there are bigger problems in the data science process rather than in the code.
ivan_ah 8y ago
Yeah, I hear you about the jupyter-is-for-draft-code. I thought that's how everyone used Jupyter for software dev. I guess this is the MATLAB-style software dev---try something in the REPL, and once you figure out the parameters interactively copy-paste that line/paragraph into your program. As soon as I have one piece of functionality working as a function, I move the code to the "main" file for the project and import the function into the notebook. This means I never have to write more than a few paragraphs of code in the notebook and instead use it for testing and glue code. I don't think of the notebook as something sharable with others, it's more like WIP code... if I keep the notebook around it's usually because it can end up a useful test harness when code needs to be revisited/updated. As for global state, one thing that I find really helpful is the keyboard shortcut 00. If you focus anywhere outside of a cell and you press zero twice, this restarts the kernel and you can re-run the commands from the beginning in a "clean state" so what you see matches what is. This approach works well when you're editing the source code that gets imported into the notebook---you never have to worry about re-importing since always restarting from scratch.
- candiodari 8y ago
  I think there is a big misunderstanding here. There's 2 kinds of people working with these things, and their needs and goals are very different. You have scientists. They are trying new things, and while reproducibility is nice, most of the time it doesn't matter. Tests don't matter, to put it mildly, because we're at a stage where it isn't just "not a product", it's not worth to become product, and they are aware of this. IF there is an end product, it's a report. A report that lists in general terms what was done and what the result was, nothing more. Their goal is to fully understand the mechanics, the minutiae and being able to explore and tinker with it (without tests breaking) is valuable. I hope you can see that most tests get very much in the way of this activity. I'm not saying there isn't anything you might test (quite the contrary), but you're not going for 100% coverage. Hell, probably not even 10%. That's how you get to matlab, mathematica, python notebooks, ... That's the usecase. On the other hand you have software engineers. They're also not a homogeneous group, but let's ignore that. The key point is that they do not understand the details and don't intend to understand the details. The goal is also different. The goal is not to learn, the goal is to help a company build a product. Reliable, repeatable, controllable, and something where you can pick someone of the street and tell them to fix something and actually expect that to get fixed. Understanding everything about everything you work with, like an academic would try to do, is not practical for a software engineer. And they get presented with an excel sheet, a notebook. They don't understand, there's no docs, but remember, they don't want to learn. Worse ... likely there's problems with the notebook. Maybe it solves one case particularly badly for some reason. Maybe it needs to run in an entirely different environment, like maybe on a phone. Or faster. Or needs to start doing something new. So they need to change something. Quickly, and quickly means without understanding the domain behind the code. So they change something and ... everything else falls down. So software developers want properly designed code with comments about even banal things, explanations about every last variable, with more tests than code, that cannot reasonably be changed with less than 5 people, and where you can always and without thinking get it back to when "it worked". Without understanding the domain behind it. So notebooks are not for software developers. You'll see this difference reflected in the languages as well. Notebooks are for languages that allow one to talk extremely high level, extremely succinctly. Because that's what academics want to do. Software developer languages, the successful ones, are basic simple things, like (these days) Java, C and Go (even C++ and Python are considered too high level by most and the frameworks software devs work with don't exactly have PyTorch's flexibility). If you want to see how you give software devs what they want, study Ada (and resist the urge to gauge your own eyes out afterwards). But there is no language, imho, as strongly on the software developer side as Ada. I would even say it even throws basic sanity under the bus to get controlibility. I would say to some extent machine learning and software development are enemies. You can fix a number of things, but when push comes to shove, machine learning is about "just figure it out", which is fundamentally against the software developer mindset of controlibility. Can you guarantee that a NLP model will never be racist ? The truth is ... not really. You can test specific cases, but the whole point of using such systems is creativity. Use the right tool for the job. Although I would say, to learn programming ... even for software devs notebooks are great for this.
Ixio 8y ago
> The Jupyter notebooks are version controlled like my .py modules. That's interesting, could you give us some more details ? Last time I tried to put a Jupyter notebook under git it was as mess. Do you use another tool than git or have they made tools to help with version control ? Or is it just your workflow that helps, like emptying all cell results before saving ?
- stevesimmons 8y ago
  Our version control uses a monorepo, with non-branching workflows, development against the head version, and frequent commits/pushes to production. For Jupyter notebooks, cell results get emptied before saving, unless declared public. That is to ensure data confidentiality, not really to help version control. Of course, standard diff gets confused with ipynbs. You need a tool like nbdime (notebook diff and merge).
  - jdkram 8y ago
    Thanks for sharing nbdime! Been looking for something like this for a while.
- jdkram 8y ago
  Not OP, but I can recommend the handy https://github.com/kynan/nbstripout https://github.com/kynan/nbstripout which acts as a git filter which makes version control ignore cell outputs. With that approach, though notebooks are clean they're still fairly poor for easily evaluating diffs between versions. If code review / diffs are more important than preserving the notebook, then you could use a post save hook to convert notebook input to a .py file and output to .html: https://towardsdatascience.com/version-control-for-jupyter-notebook-3e6cef13392d https://towardsdatascience.com/version-control-for-jupyter-n...
jimhefferon 8y ago
> I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. I have a python window open on my desk right now. But the model of programming here contrasts sharply, in my mind, with the one discussed yesterday, from HtDP https://news.ycombinator.com/item?id=17826959 https://news.ycombinator.com/item?id=17826959 . I absolutely use the REPL, for instance to determine where to put what arguments into a library function. But I have also in the past had it happen that I found a routine of mine that missed an edge case and there were a good number of projects that subsequently used that routine, that now could be wrong. Some of those could have led to a publication, so there could in that sense be a permanance to that error. Oops. Those of us who have gotten bit, or who teach and routinely see students develop habits we know will lead to them getting bit, worry about systems that are not developed so much as pasted together.
blattimwind 8y ago
> So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE. This approach to development might be a lot rarer than you think – I've never seen anyone do it frequently. For tiniest snippets ("If I want to split there and then use that suffix--how do I need to index? Let's check in the REPL real quick" or "What did that stdlib function return again? [Because that's of course not documented, why would anyone need to know that, after all]" and stuff like "I have this list of tuples like that, now, how do I need to zip/unpack/star that stuff such that I get a list of X instead. Let's try in the REPL with [(1,2), (3,4)]!"), sure, for "actual code" that's going somewhere... nope.
mi_lk 8y ago
> Over time, I expect editors like VSCode to edit Jupyter notebooks natively. Please, please, please. Given the rising importance of interactive programming, I really think it deserves more than a extension to give us a seamless experience.
- mi_lk 8y ago
  Related issue: https://github.com/Microsoft/vscode/issues/34739 https://github.com/Microsoft/vscode/issues/34739
pacala 8y ago
Jupyter code is draft code. Draft code rarely makes into clean code, because it requires extra tedious effort. Most of us are too overworked to have the time or energy to perform it. Alternatively one can write exploratory code as unit tests and run them inside an IDE. When the exploration is done, three quarters of the tedious cleanup effort is already done. There are already unit tests. There are already APIs exercised by the unit tests. The code is in much better shape. The missing two bits: * No charting support. To the best of my knowledge, neither PyCharm nor VSCode have an output console with HTML support. * Caching. Some operations take a lot of time. At the extreme, fast.ai style development, where some cells train a model for hours/days. Re-running such code from scratch on every test run is impractical. An opportunity for a library with explicit save/load/cache/checkpoint support.
- nojvek 8y ago
  Vscode has decent Jupyter support with its python plugin. What I really missed my jupyter notebooks was a world class editor with keyboard shortcuts and code completion. So I went the other route, running jupyter notebooks in vscode. I made a really simple python cli module that would convert from jupyter notebook to python files with runnable cells in vscode, and vice versa. https://github.com/nojvek/vscode-ipynb-py-converter/blob/master/README.md https://github.com/nojvek/vscode-ipynb-py-converter/blob/mas... Really upped my game while doing the udacity courses. Vscode does indeed have a html output console.
  - yodon 8y ago
    Wow! This is insanely cool! You should package this up as a VS Code plugin so more people can use it. At a minimum, please post a license.txt file in your repo so folks can actually use it (MIT or similar is pretty easy and common for something like this, without a license.txt file it's not clear whether others can actually use it and many places that means they can't).
    - m_ke 8y ago
      You should check this out https://github.com/nteract/hydrogen https://github.com/nteract/hydrogen
superdaniel 8y ago
I just wish it wasn’t so tedious to version control Jupyter notebooks. I always try and clear the output before I stage it so I don’t have large binaries of plots or figures that were generated.
- alexilliamson 8y ago
  Check out nbstripout. It sets up a git filter so that your output is stripped as the notebook is committed. But you keep the output locally.
dfee 8y ago
Honestly, I just use a file called `play.py`, create a function called `test_it()` or something, and maybe use ipdb if I need to stop somewhere and explore. Then: `pytest -s play.py` I just don’t understand what the appeal of Jupyter is, but iPython is great.
bsder 8y ago
> I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. How many businesses rely on bodged up Excel spreadsheets to run their business? This is that, but on steroids. For better or worse.
spappal 8y ago
> Over time, I expect editors like VSCode to edit Jupyter notebooks natively. PyCharm has support for Jupyter notebooks [1], but for now it's not very good [2]. Two weeks ago Jetbrains released this official comment [2]: Hi, we are aware of the problems our Jupyter support has and we aren't going to fix them, because the current support is going to retire. Currently we are working on another support of Jupyter notebook (even 2!), which hopefully will be better and much more usable. We plan to make it ready for 2018.3 PyCharm release, so stay tuned. My note: The current PyCharm is version 2.2. [1] https://www.jetbrains.com/help/pycharm/using-ipython-notebook-with-product.html https://www.jetbrains.com/help/pycharm/using-ipython-noteboo... [2] https://intellij-support.jetbrains.com/hc/en-us/community/posts/360000094190-PyCharm-support-for-Jupyter-Notebooks-is-simply-terrible?page=1#community_comment_360000096419 https://intellij-support.jetbrains.com/hc/en-us/community/po...

seanmcdirmid 8y ago

> And also more code editing to move from IDEs into the Jupyter Jupyter, and all REPLS in general, rely on a concrete top-level execution context to provide useful feeedback. Much of the code written outside of data science doesn’t really have that, which is why you don’t see notebooks being used for software development very often.

vosper 8y ago
I frequently use IPython or REPL.it for software development. It's not my main editor, and I don't write large chunks of my code there, but for rapidly prototyping a small subset of some task I find them invaluable. But you are right about notebooks - I don't find them particularly useful, except as a better REPL, and no-one I know uses them for "normal" (non Data Science) software development.
- AdamM12 8y ago
  I think my favorite non "data science" use case is web scraping. Ad-hoc nature of the task lends itself to using a notebook. A lot of times I don't want the full suite and boilerplate Scrapy gives you so I write it out in a notebook with Parsel & Requests. Once done export and clean up.
shawn 8y ago
I'm not sure this is as limiting as it sounds. You can import whatever modules you want from within a notebook, and most projects tend to be modular. (Those that aren't can usually be shimmed pretty quickly.)
barrkel 8y ago
Used all the time in Ruby (repl, not notebook). In fact I think anyone doing Ruby dev who doesn't have a pry session open is missing out on a good 30% productivity boost, unless they know their APIs inside out.
- canhascodez 8y ago
  I would agree with this, although an IDE with really good introspection and an integrated debugger can approximate the experience. Lately I've started using spaceneovim, which is hopefully going to help with the back-and-forth workflow a bit, modulo issues of what-mode-am-I-in-again? The terminal is two keystrokes away, and having all of the normal vim commands available in both contexts is something that I think will be more and more useful.

jorgemf 8y ago

Jupyter notebooks are for data science, mostly because visualization is required. Something you do once, report it and it is done. Itsn't make sense to use Jupyter for other stuff. It doesn't make much sense to use them for training big models of deep learning because there are better tools for that.

rmetzler 8y ago
What tools can you recommend for machine learning?
- jorgemf 8y ago
  IT depends what you want to do
- cirgue 8y ago
  Scripts to train the model, Jupiter to do your analysis. This is, AFIK, what most ML shops do if they’re using python.
mertd 8y ago
> Something you do once, report it and it is done. I have been having a hard time getting comfortable with that idea. There is usually "a lot of" untested and unreviewed code in the notebook that produces the analysis.
- jorgemf 8y ago
  Well, there are many things to consider: - Code is small, most of it is calling well-know libraries with the algorithms. - Every 3-4 lines of code you usually show the results of what is happening (either how the data changed, a graph or whatever) - 95% of the errors in data science come from bad practices: you either didn't clean up the data correctly, you didn't split the train/test/validation sets correctly or at the right step in the process, you chosed the wrong algorithm, etc - most of the time you don't know whether you are doing something wrong or not because you don't have enough knowledge, but the code shows something all the time. No bugs in the code. Those are the reason I don't care very much about bugs in the code because usually there are bigger problems in the data science process rather than in the code.

ivan_ah 8y ago

Yeah, I hear you about the jupyter-is-for-draft-code. I thought that's how everyone used Jupyter for software dev. I guess this is the MATLAB-style software dev---try something in the REPL, and once you figure out the parameters interactively copy-paste that line/paragraph into your program. As soon as I have one piece of functionality working as a function, I move the code to the "main" file for the project and import the function into the notebook. This means I never have to write more than a few paragraphs of code in the notebook and instead use it for testing and glue code. I don't think of the notebook as something sharable with others, it's more like WIP code... if I keep the notebook around it's usually because it can end up a useful test harness when code needs to be revisited/updated. As for global state, one thing that I find really helpful is the keyboard shortcut 00. If you focus anywhere outside of a cell and you press zero twice, this restarts the kernel and you can re-run the commands from the beginning in a "clean state" so what you see matches what is. This approach works well when you're editing the source code that gets imported into the notebook---you never have to worry about re-importing since always restarting from scratch.

candiodari 8y ago
I think there is a big misunderstanding here. There's 2 kinds of people working with these things, and their needs and goals are very different. You have scientists. They are trying new things, and while reproducibility is nice, most of the time it doesn't matter. Tests don't matter, to put it mildly, because we're at a stage where it isn't just "not a product", it's not worth to become product, and they are aware of this. IF there is an end product, it's a report. A report that lists in general terms what was done and what the result was, nothing more. Their goal is to fully understand the mechanics, the minutiae and being able to explore and tinker with it (without tests breaking) is valuable. I hope you can see that most tests get very much in the way of this activity. I'm not saying there isn't anything you might test (quite the contrary), but you're not going for 100% coverage. Hell, probably not even 10%. That's how you get to matlab, mathematica, python notebooks, ... That's the usecase. On the other hand you have software engineers. They're also not a homogeneous group, but let's ignore that. The key point is that they do not understand the details and don't intend to understand the details. The goal is also different. The goal is not to learn, the goal is to help a company build a product. Reliable, repeatable, controllable, and something where you can pick someone of the street and tell them to fix something and actually expect that to get fixed. Understanding everything about everything you work with, like an academic would try to do, is not practical for a software engineer. And they get presented with an excel sheet, a notebook. They don't understand, there's no docs, but remember, they don't want to learn. Worse ... likely there's problems with the notebook. Maybe it solves one case particularly badly for some reason. Maybe it needs to run in an entirely different environment, like maybe on a phone. Or faster. Or needs to start doing something new. So they need to change something. Quickly, and quickly means without understanding the domain behind the code. So they change something and ... everything else falls down. So software developers want properly designed code with comments about even banal things, explanations about every last variable, with more tests than code, that cannot reasonably be changed with less than 5 people, and where you can always and without thinking get it back to when "it worked". Without understanding the domain behind it. So notebooks are not for software developers. You'll see this difference reflected in the languages as well. Notebooks are for languages that allow one to talk extremely high level, extremely succinctly. Because that's what academics want to do. Software developer languages, the successful ones, are basic simple things, like (these days) Java, C and Go (even C++ and Python are considered too high level by most and the frameworks software devs work with don't exactly have PyTorch's flexibility). If you want to see how you give software devs what they want, study Ada (and resist the urge to gauge your own eyes out afterwards). But there is no language, imho, as strongly on the software developer side as Ada. I would even say it even throws basic sanity under the bus to get controlibility. I would say to some extent machine learning and software development are enemies. You can fix a number of things, but when push comes to shove, machine learning is about "just figure it out", which is fundamentally against the software developer mindset of controlibility. Can you guarantee that a NLP model will never be racist ? The truth is ... not really. You can test specific cases, but the whole point of using such systems is creativity. Use the right tool for the job. Although I would say, to learn programming ... even for software devs notebooks are great for this.

Ixio 8y ago

> The Jupyter notebooks are version controlled like my .py modules. That's interesting, could you give us some more details ? Last time I tried to put a Jupyter notebook under git it was as mess. Do you use another tool than git or have they made tools to help with version control ? Or is it just your workflow that helps, like emptying all cell results before saving ?

stevesimmons 8y ago
Our version control uses a monorepo, with non-branching workflows, development against the head version, and frequent commits/pushes to production. For Jupyter notebooks, cell results get emptied before saving, unless declared public. That is to ensure data confidentiality, not really to help version control. Of course, standard diff gets confused with ipynbs. You need a tool like nbdime (notebook diff and merge).
- jdkram 8y ago
  Thanks for sharing nbdime! Been looking for something like this for a while.
jdkram 8y ago
Not OP, but I can recommend the handy https://github.com/kynan/nbstripout https://github.com/kynan/nbstripout which acts as a git filter which makes version control ignore cell outputs. With that approach, though notebooks are clean they're still fairly poor for easily evaluating diffs between versions. If code review / diffs are more important than preserving the notebook, then you could use a post save hook to convert notebook input to a .py file and output to .html: https://towardsdatascience.com/version-control-for-jupyter-notebook-3e6cef13392d https://towardsdatascience.com/version-control-for-jupyter-n...

jimhefferon 8y ago

> I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. I have a python window open on my desk right now. But the model of programming here contrasts sharply, in my mind, with the one discussed yesterday, from HtDP https://news.ycombinator.com/item?id=17826959 https://news.ycombinator.com/item?id=17826959 . I absolutely use the REPL, for instance to determine where to put what arguments into a library function. But I have also in the past had it happen that I found a routine of mine that missed an edge case and there were a good number of projects that subsequently used that routine, that now could be wrong. Some of those could have led to a publication, so there could in that sense be a permanance to that error. Oops. Those of us who have gotten bit, or who teach and routinely see students develop habits we know will lead to them getting bit, worry about systems that are not developed so much as pasted together.

blattimwind 8y ago

> So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE. This approach to development might be a lot rarer than you think – I've never seen anyone do it frequently. For tiniest snippets ("If I want to split there and then use that suffix--how do I need to index? Let's check in the REPL real quick" or "What did that stdlib function return again? [Because that's of course not documented, why would anyone need to know that, after all]" and stuff like "I have this list of tuples like that, now, how do I need to zip/unpack/star that stuff such that I get a list of X instead. Let's try in the REPL with [(1,2), (3,4)]!"), sure, for "actual code" that's going somewhere... nope.

mi_lk 8y ago

> Over time, I expect editors like VSCode to edit Jupyter notebooks natively. Please, please, please. Given the rising importance of interactive programming, I really think it deserves more than a extension to give us a seamless experience.

mi_lk 8y ago
Related issue: https://github.com/Microsoft/vscode/issues/34739 https://github.com/Microsoft/vscode/issues/34739

pacala 8y ago

Jupyter code is draft code. Draft code rarely makes into clean code, because it requires extra tedious effort. Most of us are too overworked to have the time or energy to perform it. Alternatively one can write exploratory code as unit tests and run them inside an IDE. When the exploration is done, three quarters of the tedious cleanup effort is already done. There are already unit tests. There are already APIs exercised by the unit tests. The code is in much better shape. The missing two bits: * No charting support. To the best of my knowledge, neither PyCharm nor VSCode have an output console with HTML support. * Caching. Some operations take a lot of time. At the extreme, fast.ai style development, where some cells train a model for hours/days. Re-running such code from scratch on every test run is impractical. An opportunity for a library with explicit save/load/cache/checkpoint support.

nojvek 8y ago
Vscode has decent Jupyter support with its python plugin. What I really missed my jupyter notebooks was a world class editor with keyboard shortcuts and code completion. So I went the other route, running jupyter notebooks in vscode. I made a really simple python cli module that would convert from jupyter notebook to python files with runnable cells in vscode, and vice versa. https://github.com/nojvek/vscode-ipynb-py-converter/blob/master/README.md https://github.com/nojvek/vscode-ipynb-py-converter/blob/mas... Really upped my game while doing the udacity courses. Vscode does indeed have a html output console.
- yodon 8y ago
  Wow! This is insanely cool! You should package this up as a VS Code plugin so more people can use it. At a minimum, please post a license.txt file in your repo so folks can actually use it (MIT or similar is pretty easy and common for something like this, without a license.txt file it's not clear whether others can actually use it and many places that means they can't).
  - m_ke 8y ago
    You should check this out https://github.com/nteract/hydrogen https://github.com/nteract/hydrogen

superdaniel 8y ago

I just wish it wasn’t so tedious to version control Jupyter notebooks. I always try and clear the output before I stage it so I don’t have large binaries of plots or figures that were generated.

alexilliamson 8y ago
Check out nbstripout. It sets up a git filter so that your output is stripped as the notebook is committed. But you keep the output locally.

dfee 8y ago

Honestly, I just use a file called `play.py`, create a function called `test_it()` or something, and maybe use ipdb if I need to stop somewhere and explore. Then: `pytest -s play.py` I just don’t understand what the appeal of Jupyter is, but iPython is great.

bsder 8y ago

> I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. How many businesses rely on bodged up Excel spreadsheets to run their business? This is that, but on steroids. For better or worse.

spappal 8y ago

> Over time, I expect editors like VSCode to edit Jupyter notebooks natively. PyCharm has support for Jupyter notebooks [1], but for now it's not very good [2]. Two weeks ago Jetbrains released this official comment [2]: Hi, we are aware of the problems our Jupyter support has and we aren't going to fix them, because the current support is going to retire. Currently we are working on another support of Jupyter notebook (even 2!), which hopefully will be better and much more usable. We plan to make it ready for 2018.3 PyCharm release, so stay tuned. My note: The current PyCharm is version 2.2. [1] https://www.jetbrains.com/help/pycharm/using-ipython-notebook-with-product.html https://www.jetbrains.com/help/pycharm/using-ipython-noteboo... [2] https://intellij-support.jetbrains.com/hc/en-us/community/posts/360000094190-PyCharm-support-for-Jupyter-Notebooks-is-simply-terrible?page=1#community_comment_360000096419 https://intellij-support.jetbrains.com/hc/en-us/community/po...