5 ms·

If you ever put notebooks in source control, you owe it to yourself to try the text-based notebooks supported in Visual Studio Code[1]. They're round-trippable

by kbd 7y ago

If you ever put notebooks in source control, you owe it to yourself to try the text-based notebooks supported in Visual Studio Code[1]. They're round-trippable with real (i.e. browser-based) notebooks, yet are much better for collaboration, diffing, and editing. [1] https://code.visualstudio.com/docs/python/jupyter-support https://code.visualstudio.com/docs/python/jupyter-support

claytonjy 7y ago
How does this compare to Jupytext? I prefer pipenv to Conda, and I don't like having Jupyter(Lab) installed in each venv separately, so instead I only add `Ipykernel` to each venv and then use my system-level JupyterLab to access per-project kernels; seems like that wouldn't work here?
- kbd 7y ago
  Thanks for mentioning that project. I'm wasn't familiar with it, but the difference seems to be that Jupytext is a tool to convert notebooks between ipynb and text formats, while VSCode supports the same format (#%% cell delimiters) but enables you to edit text-based notebooks similar to how you would in the browser, including keyboard shortcuts to execute individual cells, it'll render the markdown, and so on. I'm not sure if I understand your kernel question, but VSCode's Python extension has everything built-in. As soon as you add a #%% comment it considers what follows to be a notebook cell and automatically gives you a "run cell" button that uses your chosen Python interpreter.
- bawana 7y ago
  I've noticed that if I dont install jupyter to each venv, then the pythonpath defaults to the jupyter install. As a result, the jupyter notebook cannot find the packages in the venv(for example, caffe, module not found errors)
  - claytonjy 7y ago
    huh, I've never used Caffe but wrapping my venvs with pipenv has never caused me this kind of grief. I tell pipenv to store venv stuff in the project folder (`export PIPENV_VENV_IN_PROJECT=1`), and then do the following to start a new project: pipenv --python 3.7 pipenv install ipykernel pipenv run python -m ipykernel install --user --name=`basename $(pipenv run dirname '$VIRTUAL_ENV') jupyter lab (all the magic is in that third command) Then in the list of kernels, in addition to the usual suspects I'll have one named for the folder I ran the above in. If I started a notebook before all that, I'll have to change it's kernel. Doing more `pipenv install` at the prompt makes new packages immediately available in the running notebook.
- xvilka 7y ago
  Exactly. Conda is the worst of Python world - installing gigabytes of unnecessary garbage every time.
  - linalgmixer 7y ago
    But the purpose of Anaconda is to have a easily installable set of frequently used tool for a variety of data science tasks. It's not meant to be a minimalist package.
  - kalefranz 7y ago
    Conda can create environments with literally nothing in them. If you’re putting the full-blown anaconda metapackage in every environment, then it sounds like you’re using conda wrong for your use case. Just tell conda to install the packages you actually need.
kadal 7y ago
Oh man, that's basically org-babel [0] but with working jupyter server support! 0: https://orgmode.org/manual/Working-with-Source-Code.html#Working-with-Source-Code https://orgmode.org/manual/Working-with-Source-Code.html#Wor...
- fathead_glacier 7y ago
  This sounds so good! I just read that Latex support is also included. If there is also an extension to handle references I can move out of emacs / org-mode / orgref because the maintenance cost is too high. Cannot wait to give it a try.
- d0mine 7y ago
  you can use jupyter with org-babel too https://github.com/dzop/emacs-jupyter/blob/master/README.org#org-mode-source-blocks https://github.com/dzop/emacs-jupyter/blob/master/README.org...
nerdponx 7y ago
The R ecosystem also has a text-based notebook format that they call R Markdown, with a long lineage going back to the TeX days. It never caught on outside of the R community, but the format itself is language-agnostic. https://rmarkdown.rstudio.com/ https://rmarkdown.rstudio.com/
- meztez 7y ago
  I believe people underestimate what can be done in R/RStudio these days because they haven't been exposed to it. R Markdown and its derivative are such great tools. I would also suggest xaringan if you want to present your work on a big screen.
zachguo 7y ago
The jupyter support in VSCode is working very well. I submitted this a week ago, but too bad nobody cared. https://news.ycombinator.com/item?id=19794865 https://news.ycombinator.com/item?id=19794865
mlevental 7y ago
pycharm supports the same thing, with debug, but frustratingly doesn't allow running all cells

by kbd 7y ago

claytonjy 7y ago

How does this compare to Jupytext? I prefer pipenv to Conda, and I don't like having Jupyter(Lab) installed in each venv separately, so instead I only add `Ipykernel` to each venv and then use my system-level JupyterLab to access per-project kernels; seems like that wouldn't work here?

kbd 7y ago
Thanks for mentioning that project. I'm wasn't familiar with it, but the difference seems to be that Jupytext is a tool to convert notebooks between ipynb and text formats, while VSCode supports the same format (#%% cell delimiters) but enables you to edit text-based notebooks similar to how you would in the browser, including keyboard shortcuts to execute individual cells, it'll render the markdown, and so on. I'm not sure if I understand your kernel question, but VSCode's Python extension has everything built-in. As soon as you add a #%% comment it considers what follows to be a notebook cell and automatically gives you a "run cell" button that uses your chosen Python interpreter.
bawana 7y ago
I've noticed that if I dont install jupyter to each venv, then the pythonpath defaults to the jupyter install. As a result, the jupyter notebook cannot find the packages in the venv(for example, caffe, module not found errors)
- claytonjy 7y ago
  huh, I've never used Caffe but wrapping my venvs with pipenv has never caused me this kind of grief. I tell pipenv to store venv stuff in the project folder (`export PIPENV_VENV_IN_PROJECT=1`), and then do the following to start a new project: pipenv --python 3.7 pipenv install ipykernel pipenv run python -m ipykernel install --user --name=`basename $(pipenv run dirname '$VIRTUAL_ENV') jupyter lab (all the magic is in that third command) Then in the list of kernels, in addition to the usual suspects I'll have one named for the folder I ran the above in. If I started a notebook before all that, I'll have to change it's kernel. Doing more `pipenv install` at the prompt makes new packages immediately available in the running notebook.
xvilka 7y ago
Exactly. Conda is the worst of Python world - installing gigabytes of unnecessary garbage every time.
- linalgmixer 7y ago
  But the purpose of Anaconda is to have a easily installable set of frequently used tool for a variety of data science tasks. It's not meant to be a minimalist package.
- kalefranz 7y ago
  Conda can create environments with literally nothing in them. If you’re putting the full-blown anaconda metapackage in every environment, then it sounds like you’re using conda wrong for your use case. Just tell conda to install the packages you actually need.

kadal 7y ago

Oh man, that's basically org-babel [0] but with working jupyter server support! 0: https://orgmode.org/manual/Working-with-Source-Code.html#Working-with-Source-Code https://orgmode.org/manual/Working-with-Source-Code.html#Wor...

fathead_glacier 7y ago
This sounds so good! I just read that Latex support is also included. If there is also an extension to handle references I can move out of emacs / org-mode / orgref because the maintenance cost is too high. Cannot wait to give it a try.
d0mine 7y ago
you can use jupyter with org-babel too https://github.com/dzop/emacs-jupyter/blob/master/README.org#org-mode-source-blocks https://github.com/dzop/emacs-jupyter/blob/master/README.org...

nerdponx 7y ago

The R ecosystem also has a text-based notebook format that they call R Markdown, with a long lineage going back to the TeX days. It never caught on outside of the R community, but the format itself is language-agnostic. https://rmarkdown.rstudio.com/ https://rmarkdown.rstudio.com/

meztez 7y ago
I believe people underestimate what can be done in R/RStudio these days because they haven't been exposed to it. R Markdown and its derivative are such great tools. I would also suggest xaringan if you want to present your work on a big screen.

zachguo 7y ago

The jupyter support in VSCode is working very well. I submitted this a week ago, but too bad nobody cared. https://news.ycombinator.com/item?id=19794865 https://news.ycombinator.com/item?id=19794865

mlevental 7y ago

pycharm supports the same thing, with debug, but frustratingly doesn't allow running all cells