11 ms·

A Visual Debugger for Jupyter

197 points by sandGorgon 7y ago

westurner 7y ago
Source: jupyterlab/debugger https://github.com/jupyterlab/debugger https://github.com/jupyterlab/debugger
Gehinnn 7y ago
I don't see how this debugger is visual, but anyway it's cool to see that jupyter finally has a debugger. This is what I would call a visual debugging experience: https://github.com/hediet/vscode-debug-visualizer/blob/master/extension/README.md https://github.com/hediet/vscode-debug-visualizer/blob/maste... (Disclaimer: I'm the author of that extension)
- idoby 7y ago
  This is very cool and seems useful. I've been meaning to do something like this for years. Thanks!
- Gehinnn 7y ago
  Anyone here interested in helping me improving the Python experience of this VS Code extension? I would be very happy if you could reach out on Twitter (@hediet_dev) ;)
  - serf 7y ago
    as someone who saw the demo on the github page, and wanted to use it for Python, this comment throws up some red flags. What needs to be improved for Python use?
    - Gehinnn 7y ago
      As of now, before you can visualize your data structure, you must convert it to a compatible json string (like demonstrated in [1]). This json must match the schema of the visualizer you want to use. For TypeScript/JavaScript however, my extension injects some helpers that convert some common data structures (like number arrays) automatically to appropriate JSON. It requires a deep understanding of the target programming language to see how such helpers could be injected. I don't have this unterstanding for Python and would be very happy if someone who really knows Python could help me out. [1] https://github.com/hediet/vscode-debug-visualizer/blob/master/demos/python/graph.py#L26 https://github.com/hediet/vscode-debug-visualizer/blob/maste...
- ricklamers 7y ago
  The debugging experience in notebooks before this was through a fully text based PDB input field that would allow you to control code flow with 'c' or other characters. This new extension gives a graphical user interface that's similar to how debugging works in i.e. VS Code or PyCharm. I guess that's visual about it.
- throwqwerty 7y ago
  cool extension but literally every other use of the term visual debugger refers to a graphical interface for something like gdb (where you have to manually type in commands to set break points, step, continue, etc). so I don't think it makes sense to claim that this is not a visual debugger simply because you've decided to redefine the term to suit your needs (publicizing your extension).
  - Gehinnn 7y ago
    I guess, growing up with Delphi IDE, Visual Studio, Browser Dev Tools and finally Visual Studio Code, all calling their "visual debugger" just a "debugger", I got used to debuggers being "visual" by default. As Jupyter has very powerful visualizations, I initially thought they somehow integrated their visualizations into the debugger, just to see that it's an ordinary debugger every modern IDE has. I know a debugger and it's UI is a crazy complicated beast, but I would expect any modern programming language to have such a debugger. Just because there are text-based browser like lynx, modern browsers shouldn't start calling their products "visual browser", but I get your point. Sorry for publicizing my extension here. It's free and open source, works with Python and might help a lot of people. According to github insights, traffic decreases whenever I don't publicize it somewhere.
    - lonelappde 7y ago
      You're not wrong, but Jupyter is terrible for everything about programming except in-document HTML/SVG visualizations of expressions (no support for unit testing, laggy Ui response to user input, broken Undo/Redo that throws away data, buggy browser based text editing), yet that one thing is worth all the suffering, so we'll take any improvement we can get. I don't understand why VisualStudio doesn't have LightTable like expression playground yet, but until it does, Jupyter is what we have -- a fancy REPL and document publishing format being abused as an IDE.
  - ken 7y ago
    I see two camps here. Microsoft has been calling anything with a GUI "visual" for a few decades (Basic, Studio, C++, FoxPro, ...), so people from industry often use that definition. For people from academia, "visual" has a somewhat different meaning. Thus, people in one camp see this as the standard name for the feature, and people in the other camp view it as misleading, and a continued diluting of the word in a way which trivializes their research. After a decade of hearing "oh, visual programming, you mean like Visual C++?", I've learned to avoid the word entirely. It's a loaded term. Every common word that is used as a popular brand name has this problem. I'm getting flashbacks to the 80's/90's and trying to explain my home computer to IBM PC people. "Do you have Windows?" "Well, it's an Apple. There are windows on the screen but it's not Microsoft Windows." "If you've got windows on the screen you can drag around with a mouse, that's Microsoft Windows."
- tgv 7y ago
  Looks great. I'm going to give it a try.
kbumsik 7y ago
I haven't used Jupter since I switched to web dev, but it's greate to see Jupyter's debugging experience gets better. BTW, it looks pretty much like Spyder. AFAIR Spyder is a funded project by Jupyter. What happened between two projects?
- mattip 7y ago
  Spyder was funded by Anaconda for a while, and is now funded by NumFocus and Quansight https://github.com/spyder-ide/spyder/wiki/Current-Funding-and-Development-Status https://github.com/spyder-ide/spyder/wiki/Current-Funding-an...
alfla 7y ago
NICE!
mxcrossb 7y ago
I always thought a debugger was more useful for when you got a big buried in a long execution sequence and produced by unknown combinations of variables. Where as usually my notebook cells are only a few lines long, and with a very clear state going in. Can any of you speak to the practical need for a debugger in Jupter?
- Hendrikto 7y ago
  > Where as usually my notebook cells are only a few lines long, and with a very clear state going in. That is only true in one very specific case: When you are executing all code cells in order, from top to bottom, exactly once. If you start executing cells multiple times, or out of order, or even worse, execute only parts of cells (which is possibke in many Jupyter UIs), all bets are off. Anything can happen.
throwqwerty 7y ago
I'll take this opportunity to get jupyter advice: one thing I don't understand about jupyter vs jupyterlab is why notebooks in jupyter classic have functioning vim bindings (ie in cells) through the codemirror extension but jupyterlab notebooks do not. in jupyterlab you can have vim bindings in the text editing view but not in the notebook view. for the life of me I cannot understand this design decision. how hard can it be to just leave whatever thing that makes it work in jupyter classic alone so that it continues to work in jupyterlab.
- tomrod 7y ago
  They may welcome your contribution. Hop on gitter and run it by the dev crew directly!
- lp251 7y ago
  there is an extension to provide vim bindings within cells- jupyterlab-vim
  - throwqwerty 7y ago
    yes there is but somehow it is not as good as the codementor extension (iirc you it doesn't play well with cell navigation bindings)
    - lkirk 7y ago
      Can you elaborate a little here? I made the switch from classic notebooks to jupyterlab recently and find the bin experience very similar (there's a few subtle differences that I can't remember offhand, but I don't recall having any problems with cell navigation)
arnsholt 7y ago
This looks nice and all, but on the other hand it's kind of depressing how people are expending all this effort on Jupyter and surrounding stuff, just to make something that almost, but not quite as good as Smalltalk.
- throwqwerty 7y ago
  omg jesus christ the hn trolls are out in full force today. what a valuable hard-hitting insightful opinion. really turing award quality stuff here.
- mutant 7y ago
  Would you expand on this? I don't know smalltalk, and I'm interested in what you see as better?
  - xkriva11 7y ago
    It uses the fusion of the IDE and the program you develop. You have vast possibilities of objects visualizations (inside a debugger or in separate windows), you can select some part of the code inside the debugger and let run it in separate debugger etc. See https://pharo.org/features https://pharo.org/features
- eternauta3k 7y ago
  A technology needs more than technical superiority in order to thrive.
scoot_718 7y ago
Who cares? Once you're editing text in a browser you've failed.
- throwqwerty 7y ago
  are you blind? this is a comment on a post about jupyterlab - a browser based IDE.
  - dang 7y ago
    You've been posting a shocking number of nasty swipes at people. If you keep doing this we are going to have to ban you again. Would you please just follow the site guidelines instead? https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    - throwqwerty 7y ago
      >You've been posting a shocking number of nasty swipes at people. i don't take swipes at people who don't post vile things to begin with. i will never understand the whole "you have to tolerate the intolerant" ethos. you can peruse my comments (as you have) - i never incite but censure is a public service.
- lonelappde 7y ago
  Why?
Myrmornis 7y ago
I'm a programmer, mostly not a data scientist nowadays, mostly working with Python. I have tried Jupyter Lab/Notebook on and off over the last 10 years, and I believe I have now firmly settled on my conclusion: Everyone should aim to minimize the amount of work they do in Jupyter Lab / Notebook. It shocks me a bit to find myself saying that, as it is such a beautiful piece of work. Furthermore the people who wrote it are better software engineers than I'll ever be: the frontend, the zeromq-mediated communication with the kernel, the fact that the architecture has generalized so successfully to other language kernels, its huge popularity and reach. Nevertheless, I believe I'm serious. It really comes down to just two related issues, but they're extremely important: debugging and version control. If you're a software engineer, and not a data scientist, here's how you probably debug already, or if not then how you should debug: - You identify (a) commit(s) on which the behavior is correct, and (a) commit(s) where it is not correct. - You experiment with fixes. Perhaps you stash them, perhaps you create experimental commits. The critical point is that you use your version control system (probably Git) to navigate between alternative versions of the code. With a single command, you can switch the version of your code base, and the subsequent process you invoke to test your code is a fresh process, unpolluted by any state from the version of your code that you were on 30 seconds ago. In contrast, Jupyter notebook does not encourage this style of work at all. In practice, what you will do when trying to debug some code in Jupyter is comment out lines, temporarily delete code, add experimental lines, add experimental new cells, etc. All creating a working tree, and a collection of in-memory python objects, that is a baffling mixture of changes related to the original feature development, and changes related to experimental debugging. Debugging will wear you out, as the state of your notebook gradually approaches complete incomprehensibility. If you're a software engineer, you'll already know the benefits of being able to make precise adjustments to the state of your code with git commands. You want to learn statistics and data analysis skills from data scientists, but in doing so you should not regress to a worse style of development by starting to write much of your code in Jupyter notebooks. And if you're a data scientist, you will want to acquire the debugging skills of software engineers. If you are not using git, you want to start learning it now. Crudely, we can imagine a 2-dimensional diagram with one axis for engineering skills and another for data science skills. Everyone wants to be in the top-right quadrant. In that quadrant, version control is used, and the version control system is used for debugging. Debugging is rather important in developing all software, whether scientific/numerical or not. So both groups should be minimizing the amount of code written in the Jupyter notebook UI: instead, write code in a standard Python package, in a virtualenv, installed in editable mode with `pip install -e`. If you need to use a notebook for graphical display, or HTML display of Pandas dataframes, or display of an audio playing widget, or any of the other amazing things it does so well then fine: use importlib.reload in your notebook to load and reload the bulk of your code from your Python package. The notebook should just feature calls to plotting routines etc that you have implemented in standard code files using your text editor/IDE. You could even aim for your notebook to contain so few lines of code that in some projects you might not even bother committing it.
anonu 7y ago
When will jupyter have "highlight and execute" functionality? The cell concept is fine, but I'm constantly copy pasting snippets of code into new cells to get that "incremental" coding approach...
- kylebarron 7y ago
  I use Hydrogen for that: https://github.com/nteract/hydrogen https://github.com/nteract/hydrogen
- carreau 7y ago
  It has already been in JupyterLab for more than a year. You just need to attach a kernel console to your text file.
- westurner 7y ago
  So, I went looking for the answer to this because in the past I've installed the scratchpad extension by installing jupyter_contrib_nbextensions, but those don't work with JupyterLab because there's a new extension model for JupyterLab that requires node and npm. Turns out that with JupyterLab, all you have to to is right-click and select "New Console for Notebook" and it opens a console pane below the notebook already attached to the notebook kernel. You can also instead do File > New > Console and select a kernel listed under "Use Kernel From Other Session". The "New action runInConsole to allow line by line execution of cell content" "PR adds a notebook command `notebook:run-in-console`" but you have to add the associated keyboard shortcut to your config yourself; e.g. `Ctrl Shift Enter` or `Ctrl-G` that calls `notebook:run-in-console`. https://github.com/jupyterlab/jupyterlab/pull/4330 https://github.com/jupyterlab/jupyterlab/pull/4330 "In Jupyter Lab, execute editor code in Python console" describes how to add the associated keyboard shortcut to your config: https://stackoverflow.com/questions/38648286/in-jupyter-lab-execute-editor-code-in-python-console https://stackoverflow.com/questions/38648286/in-jupyter-lab-...
cfv 7y ago
Been wondering about this for a while... How does one "graduate" a notebook to an actual program on a server? Is there an actual standard procedure for that? I'm pretty sure that if your notebook is big enough to debug what's in it you'd be doing everyone a disservice keeping it contained like that.
- realbarack 7y ago
  The fast.ai folks wrote nbdev to help with that. https://github.com/fastai/nbdev https://github.com/fastai/nbdev
  - lllmarclll 7y ago
    yfcuyrfuyoguyvuytfvty
    - lllmarclll 7y ago
      ycfyitcyvy
- starpilot 7y ago
  Run https://jupyter.org/hub https://jupyter.org/hub if you mean sharing the notebook with central administration. Deploying is a whole nother deal.
- zomglings 7y ago
  I generally use the `jupyter nbconvert` command when I have to do something like this. This can convert a Jupyter notebook into a Python script but unless the notebook has been written with some care (with the intention of converting it into a script later), a lot of manual twiddling will be required. What I generally do is use Jupyter notebooks when I'm exploring a problem or dataset. When I'm done with the exploration phase, I immediately mark the notebook as deprecated (trying to keep it in sync with code is a nightmare) and link the the relevant source code elsewhere in the project that it was deprecated for. Databricks does a decent job productionizing their notebooks, and have seen people be quite productive on that platform.
- 7y ago
deleted 7y ago
[deleted]
cs702 7y ago
This animated GIF shows the new debugger in action, allowing point-and-click step-by-step execution at breakpoints and easy inspection of variables, call stack, and source code: https://miro.medium.com/max/1400/1*NP0bYBdrhwgpJpKDhPLWrQ.gif https://miro.medium.com/max/1400/1*NP0bYBdrhwgpJpKDhPLWrQ.gi... It looks to be a very welcome improvement for those of us who routinely use Jupyter notebooks for "REVL" (read-eval-visualize-loop) experimentation. Has anyone here had a chance to test how well this works with objects from Python's scientific stack such as numpy arrays, pandas dataframes, matplotlib objects, PyTorch and TensorFlow CPU/GPU tensors, and so on?

197 points by sandGorgon 7y ago

Source: jupyterlab/debugger https://github.com/jupyterlab/debugger https://github.com/jupyterlab/debugger

I don't see how this debugger is visual, but anyway it's cool to see that jupyter finally has a debugger. This is what I would call a visual debugging experience: https://github.com/hediet/vscode-debug-visualizer/blob/master/extension/README.md https://github.com/hediet/vscode-debug-visualizer/blob/maste... (Disclaimer: I'm the author of that extension)

idoby 7y ago
This is very cool and seems useful. I've been meaning to do something like this for years. Thanks!
Gehinnn 7y ago
Anyone here interested in helping me improving the Python experience of this VS Code extension? I would be very happy if you could reach out on Twitter (@hediet_dev) ;)
- serf 7y ago
  as someone who saw the demo on the github page, and wanted to use it for Python, this comment throws up some red flags. What needs to be improved for Python use?
  - Gehinnn 7y ago
    As of now, before you can visualize your data structure, you must convert it to a compatible json string (like demonstrated in [1]). This json must match the schema of the visualizer you want to use. For TypeScript/JavaScript however, my extension injects some helpers that convert some common data structures (like number arrays) automatically to appropriate JSON. It requires a deep understanding of the target programming language to see how such helpers could be injected. I don't have this unterstanding for Python and would be very happy if someone who really knows Python could help me out. [1] https://github.com/hediet/vscode-debug-visualizer/blob/master/demos/python/graph.py#L26 https://github.com/hediet/vscode-debug-visualizer/blob/maste...
ricklamers 7y ago
The debugging experience in notebooks before this was through a fully text based PDB input field that would allow you to control code flow with 'c' or other characters. This new extension gives a graphical user interface that's similar to how debugging works in i.e. VS Code or PyCharm. I guess that's visual about it.
throwqwerty 7y ago
cool extension but literally every other use of the term visual debugger refers to a graphical interface for something like gdb (where you have to manually type in commands to set break points, step, continue, etc). so I don't think it makes sense to claim that this is not a visual debugger simply because you've decided to redefine the term to suit your needs (publicizing your extension).
- Gehinnn 7y ago
  I guess, growing up with Delphi IDE, Visual Studio, Browser Dev Tools and finally Visual Studio Code, all calling their "visual debugger" just a "debugger", I got used to debuggers being "visual" by default. As Jupyter has very powerful visualizations, I initially thought they somehow integrated their visualizations into the debugger, just to see that it's an ordinary debugger every modern IDE has. I know a debugger and it's UI is a crazy complicated beast, but I would expect any modern programming language to have such a debugger. Just because there are text-based browser like lynx, modern browsers shouldn't start calling their products "visual browser", but I get your point. Sorry for publicizing my extension here. It's free and open source, works with Python and might help a lot of people. According to github insights, traffic decreases whenever I don't publicize it somewhere.
  - lonelappde 7y ago
    You're not wrong, but Jupyter is terrible for everything about programming except in-document HTML/SVG visualizations of expressions (no support for unit testing, laggy Ui response to user input, broken Undo/Redo that throws away data, buggy browser based text editing), yet that one thing is worth all the suffering, so we'll take any improvement we can get. I don't understand why VisualStudio doesn't have LightTable like expression playground yet, but until it does, Jupyter is what we have -- a fancy REPL and document publishing format being abused as an IDE.
- ken 7y ago
  I see two camps here. Microsoft has been calling anything with a GUI "visual" for a few decades (Basic, Studio, C++, FoxPro, ...), so people from industry often use that definition. For people from academia, "visual" has a somewhat different meaning. Thus, people in one camp see this as the standard name for the feature, and people in the other camp view it as misleading, and a continued diluting of the word in a way which trivializes their research. After a decade of hearing "oh, visual programming, you mean like Visual C++?", I've learned to avoid the word entirely. It's a loaded term. Every common word that is used as a popular brand name has this problem. I'm getting flashbacks to the 80's/90's and trying to explain my home computer to IBM PC people. "Do you have Windows?" "Well, it's an Apple. There are windows on the screen but it's not Microsoft Windows." "If you've got windows on the screen you can drag around with a mouse, that's Microsoft Windows."
tgv 7y ago
Looks great. I'm going to give it a try.

kbumsik 7y ago

I haven't used Jupter since I switched to web dev, but it's greate to see Jupyter's debugging experience gets better. BTW, it looks pretty much like Spyder. AFAIR Spyder is a funded project by Jupyter. What happened between two projects?

mattip 7y ago
Spyder was funded by Anaconda for a while, and is now funded by NumFocus and Quansight https://github.com/spyder-ide/spyder/wiki/Current-Funding-and-Development-Status https://github.com/spyder-ide/spyder/wiki/Current-Funding-an...

alfla 7y ago

NICE!

mxcrossb 7y ago

I always thought a debugger was more useful for when you got a big buried in a long execution sequence and produced by unknown combinations of variables. Where as usually my notebook cells are only a few lines long, and with a very clear state going in. Can any of you speak to the practical need for a debugger in Jupter?

Hendrikto 7y ago
> Where as usually my notebook cells are only a few lines long, and with a very clear state going in. That is only true in one very specific case: When you are executing all code cells in order, from top to bottom, exactly once. If you start executing cells multiple times, or out of order, or even worse, execute only parts of cells (which is possibke in many Jupyter UIs), all bets are off. Anything can happen.

throwqwerty 7y ago

I'll take this opportunity to get jupyter advice: one thing I don't understand about jupyter vs jupyterlab is why notebooks in jupyter classic have functioning vim bindings (ie in cells) through the codemirror extension but jupyterlab notebooks do not. in jupyterlab you can have vim bindings in the text editing view but not in the notebook view. for the life of me I cannot understand this design decision. how hard can it be to just leave whatever thing that makes it work in jupyter classic alone so that it continues to work in jupyterlab.

tomrod 7y ago
They may welcome your contribution. Hop on gitter and run it by the dev crew directly!
lp251 7y ago
there is an extension to provide vim bindings within cells- jupyterlab-vim
- throwqwerty 7y ago
  yes there is but somehow it is not as good as the codementor extension (iirc you it doesn't play well with cell navigation bindings)
  - lkirk 7y ago
    Can you elaborate a little here? I made the switch from classic notebooks to jupyterlab recently and find the bin experience very similar (there's a few subtle differences that I can't remember offhand, but I don't recall having any problems with cell navigation)

arnsholt 7y ago

This looks nice and all, but on the other hand it's kind of depressing how people are expending all this effort on Jupyter and surrounding stuff, just to make something that almost, but not quite as good as Smalltalk.

throwqwerty 7y ago
omg jesus christ the hn trolls are out in full force today. what a valuable hard-hitting insightful opinion. really turing award quality stuff here.
mutant 7y ago
Would you expand on this? I don't know smalltalk, and I'm interested in what you see as better?
- xkriva11 7y ago
  It uses the fusion of the IDE and the program you develop. You have vast possibilities of objects visualizations (inside a debugger or in separate windows), you can select some part of the code inside the debugger and let run it in separate debugger etc. See https://pharo.org/features https://pharo.org/features
eternauta3k 7y ago
A technology needs more than technical superiority in order to thrive.

scoot_718 7y ago

Who cares? Once you're editing text in a browser you've failed.

throwqwerty 7y ago
are you blind? this is a comment on a post about jupyterlab - a browser based IDE.
- dang 7y ago
  You've been posting a shocking number of nasty swipes at people. If you keep doing this we are going to have to ban you again. Would you please just follow the site guidelines instead? https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - throwqwerty 7y ago
    >You've been posting a shocking number of nasty swipes at people. i don't take swipes at people who don't post vile things to begin with. i will never understand the whole "you have to tolerate the intolerant" ethos. you can peruse my comments (as you have) - i never incite but censure is a public service.
lonelappde 7y ago
Why?

Myrmornis 7y ago

I'm a programmer, mostly not a data scientist nowadays, mostly working with Python. I have tried Jupyter Lab/Notebook on and off over the last 10 years, and I believe I have now firmly settled on my conclusion: Everyone should aim to minimize the amount of work they do in Jupyter Lab / Notebook. It shocks me a bit to find myself saying that, as it is such a beautiful piece of work. Furthermore the people who wrote it are better software engineers than I'll ever be: the frontend, the zeromq-mediated communication with the kernel, the fact that the architecture has generalized so successfully to other language kernels, its huge popularity and reach. Nevertheless, I believe I'm serious. It really comes down to just two related issues, but they're extremely important: debugging and version control. If you're a software engineer, and not a data scientist, here's how you probably debug already, or if not then how you should debug: - You identify (a) commit(s) on which the behavior is correct, and (a) commit(s) where it is not correct. - You experiment with fixes. Perhaps you stash them, perhaps you create experimental commits. The critical point is that you use your version control system (probably Git) to navigate between alternative versions of the code. With a single command, you can switch the version of your code base, and the subsequent process you invoke to test your code is a fresh process, unpolluted by any state from the version of your code that you were on 30 seconds ago. In contrast, Jupyter notebook does not encourage this style of work at all. In practice, what you will do when trying to debug some code in Jupyter is comment out lines, temporarily delete code, add experimental lines, add experimental new cells, etc. All creating a working tree, and a collection of in-memory python objects, that is a baffling mixture of changes related to the original feature development, and changes related to experimental debugging. Debugging will wear you out, as the state of your notebook gradually approaches complete incomprehensibility. If you're a software engineer, you'll already know the benefits of being able to make precise adjustments to the state of your code with git commands. You want to learn statistics and data analysis skills from data scientists, but in doing so you should not regress to a worse style of development by starting to write much of your code in Jupyter notebooks. And if you're a data scientist, you will want to acquire the debugging skills of software engineers. If you are not using git, you want to start learning it now. Crudely, we can imagine a 2-dimensional diagram with one axis for engineering skills and another for data science skills. Everyone wants to be in the top-right quadrant. In that quadrant, version control is used, and the version control system is used for debugging. Debugging is rather important in developing all software, whether scientific/numerical or not. So both groups should be minimizing the amount of code written in the Jupyter notebook UI: instead, write code in a standard Python package, in a virtualenv, installed in editable mode with `pip install -e`. If you need to use a notebook for graphical display, or HTML display of Pandas dataframes, or display of an audio playing widget, or any of the other amazing things it does so well then fine: use importlib.reload in your notebook to load and reload the bulk of your code from your Python package. The notebook should just feature calls to plotting routines etc that you have implemented in standard code files using your text editor/IDE. You could even aim for your notebook to contain so few lines of code that in some projects you might not even bother committing it.

anonu 7y ago

When will jupyter have "highlight and execute" functionality? The cell concept is fine, but I'm constantly copy pasting snippets of code into new cells to get that "incremental" coding approach...

kylebarron 7y ago
I use Hydrogen for that: https://github.com/nteract/hydrogen https://github.com/nteract/hydrogen
carreau 7y ago
It has already been in JupyterLab for more than a year. You just need to attach a kernel console to your text file.
westurner 7y ago
So, I went looking for the answer to this because in the past I've installed the scratchpad extension by installing jupyter_contrib_nbextensions, but those don't work with JupyterLab because there's a new extension model for JupyterLab that requires node and npm. Turns out that with JupyterLab, all you have to to is right-click and select "New Console for Notebook" and it opens a console pane below the notebook already attached to the notebook kernel. You can also instead do File > New > Console and select a kernel listed under "Use Kernel From Other Session". The "New action runInConsole to allow line by line execution of cell content" "PR adds a notebook command `notebook:run-in-console`" but you have to add the associated keyboard shortcut to your config yourself; e.g. `Ctrl Shift Enter` or `Ctrl-G` that calls `notebook:run-in-console`. https://github.com/jupyterlab/jupyterlab/pull/4330 https://github.com/jupyterlab/jupyterlab/pull/4330 "In Jupyter Lab, execute editor code in Python console" describes how to add the associated keyboard shortcut to your config: https://stackoverflow.com/questions/38648286/in-jupyter-lab-execute-editor-code-in-python-console https://stackoverflow.com/questions/38648286/in-jupyter-lab-...

cfv 7y ago

Been wondering about this for a while... How does one "graduate" a notebook to an actual program on a server? Is there an actual standard procedure for that? I'm pretty sure that if your notebook is big enough to debug what's in it you'd be doing everyone a disservice keeping it contained like that.

realbarack 7y ago
The fast.ai folks wrote nbdev to help with that. https://github.com/fastai/nbdev https://github.com/fastai/nbdev
- lllmarclll 7y ago
  yfcuyrfuyoguyvuytfvty
  - lllmarclll 7y ago
    ycfyitcyvy
starpilot 7y ago
Run https://jupyter.org/hub https://jupyter.org/hub if you mean sharing the notebook with central administration. Deploying is a whole nother deal.
zomglings 7y ago
I generally use the `jupyter nbconvert` command when I have to do something like this. This can convert a Jupyter notebook into a Python script but unless the notebook has been written with some care (with the intention of converting it into a script later), a lot of manual twiddling will be required. What I generally do is use Jupyter notebooks when I'm exploring a problem or dataset. When I'm done with the exploration phase, I immediately mark the notebook as deprecated (trying to keep it in sync with code is a nightmare) and link the the relevant source code elsewhere in the project that it was deprecated for. Databricks does a decent job productionizing their notebooks, and have seen people be quite productive on that platform.
7y ago

deleted 7y ago

[deleted]

cs702 7y ago

This animated GIF shows the new debugger in action, allowing point-and-click step-by-step execution at breakpoints and easy inspection of variables, call stack, and source code: https://miro.medium.com/max/1400/1*NP0bYBdrhwgpJpKDhPLWrQ.gif https://miro.medium.com/max/1400/1*NP0bYBdrhwgpJpKDhPLWrQ.gi... It looks to be a very welcome improvement for those of us who routinely use Jupyter notebooks for "REVL" (read-eval-visualize-loop) experimentation. Has anyone here had a chance to test how well this works with objects from Python's scientific stack such as numpy arrays, pandas dataframes, matplotlib objects, PyTorch and TensorFlow CPU/GPU tensors, and so on?