7 ms·
Jupyter notebooks in the IDE: VS Code vs. PyCharm
- JA7Cal 7y agoAlso available at https://github.com/mwouts/notebooks_in_vscode_and_pycharm_jan_2020 https://github.com/mwouts/notebooks_in_vscode_and_pycharm_ja...
- oefrha 7y agoAnyone else noticed that when editing a notebook in VS Code, one could lose unexecuted edits to a cell with accidental arrow key input (that defocuses the current cell and erases changes)? Too many times I have lost a paragraph worth of Markdown text midst typing, it’s kind of maddening.
- derriz 7y agoYep - it's very annoying but for me it only happens with markdown cells? So now I generally select all and copy now before leaving a markdown cell if I've edited it. The other issue is the language server seems to get into a funk regularly - the completion stops working as does the display of docstrings so I regularly use the "Developer: reload window" feature to reset everything. Also the plots view is poor - it's basically just an SVG you can magnify - there's no proper zoom or rotation for 3d plots. I was also irritated by the "smart" environment discovery stuff - just let me specify which python install or vent I want to use. Don't make be google to see what names/locations I have to use in order for an environment to magically appear in the list. Also the editor extension (I use vi mode) is ignored. And even saving the notebook doesn't work the way every other editor pane works in vscode. Hmm... writing this list makes me wonder why I haven't looked at alternatives.
- connor4312 7y agoFyi VS Code is working on[1] native first-class notebook support, which should make the experience of using a notebook much better. Notebooks right now stretch the bounds what extensions can do, particularly in that they can't use the VS Code's own text editing services, so things like vi mode aren't inherited. (Disclaimer: I work on VS Code, but am not working on notebooks) 1. https://github.com/microsoft/vscode/issues/88243 https://github.com/microsoft/vscode/issues/88243
- oefrha 7y agoCool. A while ago I wanted to report bugs wrt notebooks but the bug reporting flow asked me to turn off extensions, and after turning off extensions notebooks no longer work, so I didn’t even know where to report the bugs.
- connor4312 7y agoJupyter Notebooks today are handled by the Python extension, which is a separate codebase and team from VS Code. You should file issues there: https://github.com/Microsoft/vscode-python https://github.com/Microsoft/vscode-python
- greazer 7y agoDisclaimer: I am the engineering manager for the jupyter support supplied in the Python extension. I LOVE this feedback! Keep it coming (though entering issues https://github.com/Microsoft/vscode-python https://github.com/Microsoft/vscode-python would be ideal) Yes, as conner4312 VS Code is investigating how to better incorporate a juypter notebook experience within it, natively. As has been pointed out here, there are a number of limitations that the python extension notebook support has run into. We have been working closely with the VS Code team to find solutions to these limitations. The custom editor work (https://github.com/microsoft/vscode/issues/77131 https://github.com/microsoft/vscode/issues/77131) is one such mechanism. We plan on rolling out our use of that work soon after it is officially released (sometime in February). However, this will not address all of the problems mentioned in this thread. Those issues will be addressed as native jupyter support arrives in VS Code. Here's some additional info: - Find/replace. (Personally, this drove me NUTS! Just ask my team! :) ) That being said a simple find (ctrl-f) is avaiable in the current official release. Unfortunately, it does not include replace. While we recognize that this is a severe limitation for some, implementing a full find and replace solution would end up requiring the extension to duplicate most of the functionality provided by VS Code itself. Therefore, we currently planning to piggy back off of that work when it becomes available. If that winds up taking an inordinate amount of time, we will consider further stop gap work. - The markdown loss issue has been addressed and will be in the February release. - For the language server problem discussed above, we need more info. I would encourage you to enter a new issue at https://github.com/Microsoft/vscode-python https://github.com/Microsoft/vscode-python. In the meantime, there are known related issues that will be fixed the Feb release. One in particular, was caused by the backend jupyter server inexplicably taking a long time to return us a full completion list. So when you don't see completion pop up, it may not be that it's broken, it may just be taking a long time. We are working around this issue for the February release. Also, if you're having docstring issues, you may find that turning off the python jedi language service in settings will help. - Plots view. Yes, it is limited and honestly can cause rendering slowdowns. We are working through ways of addressing both problems. A good solution for supporting "proper" zoom will require adding ipywidget support overall, which is coming soon. - "Smart environment stuff". I'm not following completely. Is this a general python extension issue or does it have to do with the jupyter kernel selection support we have for notebooks? Again, feel free to enter additional details at: https://github.com/Microsoft/vscode-python https://github.com/Microsoft/vscode-python - Supporting global editor extensions like "VI mode", isn't something that we can support without help from VS Code itself. One of the main reason's VS Code is pursuing native notebook support is to be able to handle issues exactly like this. - Saving the notebook weirdness will be addressed in the February release. We were waiting for the custom editor support I referred to above to arrive. Hope this is useful information!
- codydh 7y agoThis misses a pretty stellar entrant in the field, Atom + Hydrogen. Works nicely with multiple kernels (I use R and Python), allowing notebook-like functionality across multiple languages.
- Jedd 7y agoWhat do you mean misses? The author mentions that exact combination in paragraph 4.
- OldGuyInTheClub 7y agoSpyder has a Jupyter notebook plugin. I was able to get a notebook going and link the kernel to the IDE. The variable editor worked and I could debug code in Spyder and then paste it into the notebook. The notebook piece of it was good for graphics, widgets, markdown, etc.
- CinchWrench 7y agoIs spyder still super slow? I gave it up back in 2018 because of the (perceived) performance problems.
- OldGuyInTheClub 7y agoHard to say as I am not a power-user/coder by any standard least of all those of this forum. I've tried Python on and off over the years and the Spyder 3.x/Jupyter combo was the closest I could get to the kind of UI that Matlab offers: Variable inspector, debugger, persistent/docked plots, documentation, recordkeeping, etc. It wasn't clear whether the Spyder/Jupyter plugin was going to be maintained in future Spyder releases and the combo, although close, was not as smooth as I would have liked. Ultimately, Matlab is a better fit for the things I want to do in technical computing. My employer offers it and Simulink with a wide variety of toolboxes and blocksets so I can get farther faster for the problems that interest me or that I am asked to solve. The new table datatype and the steadily improving Livescript make exploration and notetaking very easy. I've also been able to rely on Mathworks documentation to get up-to-speed in new areas.
- taeric 7y agoNeeds more mention of emacs and org mode... :) I am curious to see the general workflow expanded on. I hesitate to think there is a single correct way to keep a notebook. Seems flexibility is the key feature. And is why text is so well used.
- tomrod 7y agoI'm curious if there is an objective breakdown of emacs org mode and other methods. I'd suspect emacs penetration is not terribly high due to initial learning curve.
- joseph8th 7y agoI'm sure you're right about penetration, but I've found EIN [1] very usable, and particularly like the org support. [1https://github.com/millejoh/emacs-ipython-notebook/blob/master/README.rst https://github.com/millejoh/emacs-ipython-notebook/blob/mast...
- radarsat1 7y agoI haven't even gotten into org mode, I just edit Python using emacs and it works great. I can evaluate parts of the code and jump to the interpreter when needed. I keep a matplotlib window open with plt.ion() when running locally, or use plt.savefig() when running remotely and have a qiv window open that automatically detects new images. Works great. The only thing I would like is to be able to use the Python interpreter _within_ the context of a function sometimes, it's annoying when an error occurs inside a function and I can't access the local variables. I like jupyter notebooks too, but one of the things stopping me is that I can't use emacs features for editing ;)
- cjbillington 7y agoTo interactively inspect within a function, you could use the debugger with `breakpoint()`. But it's not a regular Python interpreter, which annoys me having to learn the different syntax. You can also embed IPython interactively with: import IPython; IPython.embed() though IPython kinda doesn't clean up after itself, so it's a bit messier (your prompt is messed up afterwards). Finally, you can: import code; code.interact() The downside of this is there's no history or autocomplete by default. I have a custom function `embed()` [1] that calls `code.interact()`, but with history and autocomplete configured. This is the closest I got it to looking like a regular Python interpreter at an arbitrary place in the code. [1] https://pastebin.com/WDuheBfV https://pastebin.com/WDuheBfV
- MayeulC 7y agoKDE's Cantor isn't too bad either, but I am not sure how well it interoperates with Jupyter notebooks.
- ptrott2017 7y agoOverall its pretty good but some known issues. See: https://sirgienkogsoc2019.blogspot.com/2019/08/cantor-and-support-for-jupyter.html https://sirgienkogsoc2019.blogspot.com/2019/08/cantor-and-su...
- swiley 7y agoWhy are you using a notebook and an IDE? I thought the notebook UI was an alternative to the IDE UI.
- ptd 7y agoI'd compare it loosely to taking notes by hand on a tablet. There is benefit in both, when they are combined it can be a better tool than either alone.
- bshipp 7y agoIn my case? pycharm for code completion, debugging, refactoring, etc and notebooks for recording reproducible data analysis steps.
- _coveredInBees 7y agoI feel like this is the only sane way to go about things. I really don't understand people using notebooks to actually develop code and train ML/deeplearning models. A real IDE like Pycharm is so much more powerful at debugging, refactoring and in-general, writing good code, structuring it, and not to mention, being able to version control it in a sane manner.
- pinouchon 7y agoThe IDE (I also use PyCharms) is great for editing code (search, refactoring, syntax highlight...) Jupyter notebooks (or JupyterLab in my case) is great for prototyping and data visualisation. In my work (data engineering and training deep neural nets) I also happen to use both. It's unwieldy, but I can't get rid of either one (pycharms or jupyter lab)
- skwb 7y agoFrankly I HATE web based notebooks. I do deep learning research, and I write a lot of complex functions to handle pre-processing. And with that there's a lot of debugging I need to do, particularly visual debugging of images. The interactivity of notebooks and the features of an IDE has made my programming experience much better.
- robenkleene 7y agoI'm trying to implement these same features in my app, Repla, but without requiring an editor plugin, and instead having a view in a separate app that automatically updates via file-system events when you save. There's some more information here https://repla.app/live-coding.html https://repla.app/live-coding.html It's an approach that I think has somes pros and cons relative to integrating these features into an IDE. The main argument for moving them to a separate app, is that IDEs are already really complicated.
- tvb12 7y agoI'm not on macOS, so I can't try it right now, but I've bookmarked this and will keep an eye on it. Basically the report/output view of Iodide/Jupyter without the "editor" view? I was really hoping something like this would come along.
- robenkleene 7y agoYes, that's exactly right. Great to hear other people are also interested in this approach! The technology I developed has ended up being pretty flexible, you can read about it here (https://blog.repla.app/2020/01/13/what-is-repla/ https://blog.repla.app/2020/01/13/what-is-repla/) and here (https://blog.repla.app/2020/01/13/repla-use-cases/ https://blog.repla.app/2020/01/13/repla-use-cases/) if you're curious. But yeah, that's exactly the use case I developed the technology for. I wanted a browser engine with a plugin system so that I could use to display output of systems like Jupyter and Observable HQ, while still editing the source code in my exiting text editor, the same way I edit other source code. I sort of discovered along the way that this system is also a nice way to run a local development web server, and that was a bit easier to implement, so that's what I've launched with first, the homepage has more info on that (https://repla.app/ https://repla.app/). I'd like to support Linux and Windows eventually too, but it's easier to start with supporting just one platform while I'm just testing out whether there's other people interested in the idea.
- mweatherill 7y agoMicrosoft have released another IDE with notebook support called Azure Data Studio. It uses the same underlying code base as VS Code but is targeted at data driven tasks such as connecting to databases. Strangely, it uses a different jupyter notebook implementation to VS Code. I found it avoids some of the annoying bugs mentioned in another comment. It also introduces a SQL kernel which I’ve found useful for organising my queries.
- wayneftw 7y ago> SQL kernel I've never heard this term before and a brief search for it did not bear any definitions. Can you expand on what exactly makes an SQL kernel?
- sachdevap 7y agoNot GP, but I think they are referring to https://github.com/jupyter/jupyter/wiki/Jupyter-kernels https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
- mweatherill 7y agoWhilst jupyter notebooks are often synonymous with python code, the languages are actually pluggable by using a different kernel. Microsoft have implemented an SQL kernel which means you can write SQL queries directly in a notebook.
- greazer 7y agoAzure Data Studio is a proper fork of VS Code all up. This is why it was able to work around some of the limitations the Python extension is currently dealing with.
- teddyuk 7y agoI can’t help thinking that notebooks would be less relevant if people embraced unit testing more
- prepend 7y agoI don’t think they are very related, aside from both being useful. I use primarily for doing stepwise development. I also use them for a decent way to see the output of my code, usually tables and charts. I have unit tests for the various modules and classes created.
- mstump 7y agoI'm surprised that nobody mentioned that find/replace doesn't work for notebooks in VSCode. It's by far my biggest annoyance. https://github.com/microsoft/vscode-python/issues/7903 https://github.com/microsoft/vscode-python/issues/7903
- greazer 7y agoSee my reply below.
- jonnycomputer 7y agoI see this, and I cannot help but think, hey, one step closer to achieving the plain-text goodness magic of the R Notebook. But maybe I'm being too sassy. Its just that every time I pick up Jupyter I think, what is this clumsy thing? I know people love it, but that's how I feel. I wish pycharm had the in-line capabilities of the R Notebook, because I really like pycharm.
- andykx 7y agoI don’t do data science and I really don’t understand what Jupyter notebooks are used for. Are they just a way to explore ideas with immediate feedback?
- fulafel 7y agoYou can publish them on github or mybinder as persistent self documenting interactive repl sessions. Then others can see what the code did, the intermediate results, and can easily play with it. (No "data science" here either, but into repl based instant feedback development & collaboratiove programming tools)
- andykx 7y agoOh, very cool. Sounds like something I’d love to use. I always loved the Clojure REPL.
- gimagon 7y agoNotebooks also inline visualizations more naturally than the repl, which makes them really nice for the data analysis workflow.
- deleted 7y ago[deleted]
- fulafel 7y agoFun fact: you can use Clojure with Jupyter: https://github.com/clojupyter/clojupyter https://github.com/clojupyter/clojupyter (Jupyter supports "kernels" and there are also more non-Python language kernels that you can use)