12 ms·

Jupyter Notebook 5.0

• FiReaNG3L 10y ago
  The only thing I want from Jupyter notebook is easy version control workflow - not sure what's the best option is right now but last time I checked it was 'delete all outputs before commit' which is great in many cases.
  • RobinL 10y ago
    There's some good advice here: http://stackoverflow.com/questions/18734739/using-ipython-notebooks-under-version-control http://stackoverflow.com/questions/18734739/using-ipython-no.... Obviously imperfect, but it helps until it's implemented properly
  • Y_Y 10y ago
    Use nbstripout[0] as a git filter. Then you have seamless git control of notebooks. There's talk of automatically saving code only separate versions in future releases. [0] https://github.com/kynan/nbstripout https://github.com/kynan/nbstripout
    • lancefisher 10y ago
      This looks pretty good for handling diffing/merging of notebooks. I'm just getting started with jupyter, but it seems like somewhat of a pain to have to ssh into a server to manage versioning with git while I work on the code in the browser. I'd prefer if I could do all of my jupyter work directly in a notebook, and commit from the notebook webpage. I'm not sure how others work with this stack, so there is likely tooling I don't know about. I'd love to hear suggestions.
      • carreau 10y ago
        Well you can commit from the UI: https://gab41.lab41.org/commit-and-push-to-github-from-jupyter-notebooks-579f5743a50b https://gab41.lab41.org/commit-and-push-to-github-from-jupyt... you just need to install the extension.
      • coconutrandom 10y ago
        prepending commands with a "!" runs it in the shell, assuming you are using python. ! git commit add ./my_file.ipnb http://ipython.readthedocs.io/en/stable/interactive/python-ipython-diff.html?highlight=exclamation#shell-assignment http://ipython.readthedocs.io/en/stable/interactive/python-i...
  • deleted 10y ago
    [deleted]
  • spott 10y ago
    Check out nbdime[0], it might be what you are looking for. [0]https://nbdime.readthedocs.io/en/latest/vcs.html https://nbdime.readthedocs.io/en/latest/vcs.html
    • spott 10y ago
      This probably needs more explanation: From the docs: "nbdime provides tools for diffing and merging Jupyter notebooks." It includes both graphical diff and merge tools, command line diff and merge tools, VCS integration for git (so git uses the nbdime diff and merge for notebooks), etc. Might be worth looking into if that is what you are looking for.
  • goerz 10y ago
    I would even settle for an undo/redo (of more than one action)
  • roystonvassey 10y ago
    Jupyter will become like the Google of software development then ;) To be fair, wasn't Jupyter designed to be an IDE - experiment with code, make tweaks and get the final version out in a text editor - rather than a repository of production-ready code?
    • carreau 10y ago
      Not really, you rarely have IDE designed to be around live-code. IDE are more meant to develop full-featured software. We are moving in this direction (as many users request it), but we still want to stay focus on Data Exploration.
      • goatlover 10y ago
        Smalltalk people have thought otherwise since the 70s. The live code part is a big reason to use Jupyter. It makes me think the ST (and Lisp) people were on to something.
    • Sean1708 10y ago
      IDE here meaning Interactive Development Environment, rather than Integrated Development Environment?
  • m_mueller 10y ago
    This very much. I'd like for Jupyter to have an option to keep the cell code in individual python files in th backend, so you can just check in those to VCS. It would furthermore allow outside python code to import code from notebooks if jupyter would just provide a simple __init__.py file together with the cell files.
    • carreau 10y ago
      So you are asking for https://github.com/takluyver/nbexplode https://github.com/takluyver/nbexplode and https://github.com/ipython/ipynb https://github.com/ipython/ipynb respectively. There just not maintained enough to be in the core :-)
    • mctx 10y ago
      You can create a post save hook to output .py files - http://jupyter-notebook.readthedocs.io/en/latest/extending/savehooks.html http://jupyter-notebook.readthedocs.io/en/latest/extending/s... (You used to be able to run ipython notebook --script)
  • TimSAstro 10y ago
    This is something I've been giving some thought to recently. I came to the conclusion that the best way to go currently is git-clean filters built using jq. It's still a bit hacky though. Gentle write-up here: http://timstaley.co.uk/posts/making-git-and-jupyter-notebooks-play-nice/ http://timstaley.co.uk/posts/making-git-and-jupyter-notebook...
• roystonvassey 10y ago
  Hands down the best and most used tool in my portfolio. Well done, guys! Does anyone know if there's a plan to introduce multi-kernel support in single notebooks like what Zeppelin does? Not that I have a strong preference for it but it appears to hold a lot of appeal in Spark-like environments where not all packages are available in Pyspark and you need to move between native Scala/spark and Pyspark.
  • nl 10y ago
    Spark-like environments where not all packages are available in Pyspark and you need to move between native Scala/spark and Pyspark. What exactly do you mean here? Are you referring to the parts of Spark which don't currently have a Python API? Because those are becoming smaller and smaller. There is also Jupyter magics to let you change languages within a notebook. See %Rpush and %RPull from [1]. Not sure if there is a way to have a Scala kernel running and sharing the same Spark context though. I think IBM is working on something in this area. [1] https://blog.dominodatalab.com/lesser-known-ways-of-using-notebooks/ https://blog.dominodatalab.com/lesser-known-ways-of-using-no...
    • roystonvassey 10y ago
      Yes, that's what I exactly meant. And you're right - that's a pretty small subset and that's what I tell others too. But, the ease with which you can load interpreters on Zeppelin (apart from the pre-loaded ones) is impressive. I imagine it comes at a cost of some instability because it hangs more often than Jupyter.
  • carreau 10y ago
    See project like metakernel [1], in the end a kernel can be "just" controlling process that manage multiple language. So why you won't get "multiple kernel" you can get "one kernel with multiple language". Its just asking for "multiple kernel" which not the best way to describe what you want to do. In the end it the same excpet you do the language dispatch in the backend instead of the frontend. It's also more efficient for data sharing. [2] show you haw Hydrogen (based on Jupyter as well), does it. And [3] (mine), show you how in the the same notebook to use Python, R, C, Rust, Fortran, Cython and Julia with data sharing and sending functions back and forth between languages. I was definitively lazy and did not include things like SQL, javascript and a few others. I haven't used spark in a while, and definitively never from scala directly, but I doubt it would be much harder to do as the C/Rust/Fortran/Cython took me an afternoon to write. [1]: https://github.com/Calysto/metakernel https://github.com/Calysto/metakernel [2]: https://www.google.com/url?hl=en&q=https://medium.com/nteract/hydrogen-introducing-rich-multi-language-documents-b5057ff34efc&source=gmail&ust=1491411006830000&usg=AFQjCNF-dWWzLXr2H6zJRuJyMpc2RqiWpQ https://www.google.com/url?hl=en&q=https://medium.com/nterac... [3]: http://carreau.github.io/posts/23-Cross-Language-Integration.html http://carreau.github.io/posts/23-Cross-Language-Integration...
• zeptomu 10y ago
  Did anybody succeed in deploying Jupyter into a multi-user environment? I know there is jupyterhub and some people deployed it for selected trustworthy users (e.g. in a university course context or a company's intranet), but AFAIK it hasn't been deployed in production to potential untrustworthy users. Current approaches seem to rely on the idea to sandbox you in a Docker environment, but mostly these are work-in-progress solutions and I am not sure how much more secure they are, beside the fact that also authentication and persistent storage issues have to be solved. The obvious problem is that accessing Jupyter is technically similar to allowing full shell-access and you have to deal with local privilege escalation, but I wonder if there has been any progress. I evaluated to use it as a UI for domain-specific applications that give users some kind of graphical shell, but in the end I decided against it, because of security concerns.
  • makmanalp 10y ago
    try.jupyter.org is something like this, and I think it uses this: https://github.com/jupyter/tmpnb https://github.com/jupyter/tmpnb
    • takluyver 10y ago
      You're right, it does use that. The security against untrusted users is whatever you get from docker - each user's code runs in an individual container. In the case of try.jupyter.org a breakout would not be catastrophic, though, because those servers shouldn't be holding any sensitive data.
  • carreau 10y ago
    http://mybinder.org/ http://mybinder.org/ does that, as well as https://notebooks.azure.com/ https://notebooks.azure.com/ – well this one is authenticated, but it's easy to get a microsoft live account, and it does not even needs to be validated so that count (for me) as non trustworthy users. Technically giving access to Jupyter does not give full shell access. Only if the installed kernel give full shell access (which the default Python one does). The terminal feature can be deactivated as well.
• minimaxir 10y ago
  For those who use R, I strongly recommend looking into R Notebooks (http://rmarkdown.rstudio.com/r_notebooks.html http://rmarkdown.rstudio.com/r_notebooks.html), as there is a lot more versatility involved, especially over the Jupyter/IRKernel approach. Although it's R only, unfortunately (you can run Python code in it but not the way you expect) I'd like to see some things ported into Jupyter from R Notebooks, like JavaScript data tables and the separation of code and output, making it easy to version control only the code. (Atleast this 5.0 release makes tables nonugly)
  • carreau 10y ago
    It would be great to sit with the Rstudio-notebooks developers and user and discuss how we could get the two platform to converge and be more interoperable. I guess both team have a lot on their plate, and we would need more manpower, but understanding and collaborating with other project for the good of user is always something the Jupyter team is happy to do.
    • imcoconut 10y ago
      The jupyter team has been working on jupyter lab which is a more r studio style environment. It incorperates notebooks, terminals, file editing, traditional ipython console, plus some other cool stuff in a web ide. You should be able to use an R jupyter kernel with it. It's in alpha right now but they're making crazy fast progress on it. I've been using it and it's awesome
      • carreau 10y ago
        Yes I know I'm one of the Jupyter dev :-) there is still a difference in notebook format and protocols. In that sens JupyterLab is not more compatible with Rstudio than the normal notebook.
        • imcoconut 10y ago
          Lol, sorry - well hopefully someone else reading this thread can find out about jupyter lab. it really is amazing. Thanks so much for all your hard work!
• jacquesm 10y ago
  Jupyter is super useful, together with anaconda it's a winning team. I love anaconda because it takes the sting out of all the dependencies and cruft that stops python packages from installing cleanly (besides the v3 / v2 mess and installing as a regular user rather than root).
  • ciupicri 10y ago
    Also Anaconda tends to come with newer versions than those available in most Linux distributions.
• superfx 10y ago
  Still no collapsable hierarchy of cells. This is the one feature I miss most from Mathematica notebooks.
  • wannesm 10y ago
    I can recommend http://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/collapsible_headings/readme.html http://jupyter-contrib-nbextensions.readthedocs.io/en/latest...
    • mFixman 10y ago
      Also Table of Contents 2 [1] is useful, since it can numerate headings and subheadings in a natural way. Jupyter extensions are incredibly powerful, and I don't know how I used to do data analysis before finding them. [1] http://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/toc2/README.html http://jupyter-contrib-nbextensions.readthedocs.io/en/latest...
• deleted 10y ago
  [deleted]
• deleted 10y ago
  [deleted]
• deleted 10y ago
  [deleted]
• sandGorgon 10y ago
  Does anyone know how to setup a multi user notebook? Because of the nature of the notebook and how it maintains state ... that a single instance is usable by a single person. We are building dashboards in Jupiter and really would love it to be multi user... Without getting into the hub and stuff (way too complex to set it up)
  • yuvipanda 10y ago
    Disclaimer: I work on the hub. Sorry to hear you found the hub too complex. We're working on making easier-to-use hub setups that fit different use cases. Can you tell us a little more about what your use case was and (optionally) which parts of the hub setup you found too complex? Thanks!
    • sandGorgon 10y ago
      Thanks for replying. So it's a bunch of different technologies - nodejs,etc. I'm kinda wondering if it can be built in Python itself. Make it part of a normal jupyter install, so just a "jupyter hub start " will work ? EDIT: adding to that, you have built a nodejs based http proxy - can you not build it within Python (for uniformity) or nginx (for performance as well as mind share) ? Do you even need to mandate a http proxy ? Second question is that can it run in a multiprocess - I don't want to run it in interactive mode, but just straight top to bottom. Perhaps there's huge memory savings there.
      • carreau 10y ago
        I think yuvi wrote the CHP on nginx (https://github.com/yuvipanda/jupyterhub-nginx-chp https://github.com/yuvipanda/jupyterhub-nginx-chp) when we first wrote CHP, node was the only viable solution to have a dynamic websocket proxy. Nowdays Go, or Python 3 with AsyncIO may be potential contenders. It may be possible to rewrite in Python but time is limitted. I'm unsure about your second question.. run notebook top to bottom ? `nbconvert --to notebook --execute --inplace yournotebook.ipynb` ?
        • 10y ago
• marcinkuzminski 10y ago
  Looking forward to next OpenSource RhodeCode release which will now support rendering jupyter notebooks. It'll be second to Github source code management with Jupyter support.
• agumonkey 10y ago
  The computational dom
• m_mueller 10y ago
  I absolutely love Jupyter. Thank you guys and gals for your great efforts. I've had this idea about how it could become a very simple and powerful rapid application development tool based upon the widgets you already have: What if you could preview and bundle it in a mode with hidden code to standalone executable files on all platforms? There's already many tools in python to do this, all it would take is some export feature to tie it all together.
  • jasongrout 10y ago
    We're going to be exploring building dashboarding solutions more with JupyterLab. See https://github.com/jupyterlab/jupyterlab/issues/1640 https://github.com/jupyterlab/jupyterlab/issues/1640 for some of our current discussion. That said, the deployment issue you've talked about has been experimented with in the IBM dashboarding features (i.e., one-click to deploy an application-like interface).
• JupiterMoon 10y ago
  Does this update break all the addons again? (Vim keybindings in edit mode, code folding etc.)
  • tmearnest 10y ago
    It seemed fine for me. All of my addons were working.
• wodenokoto 10y ago
  I didn't understand the purpose of cell-tags. Can anyone give an example of how they might be used?
  • krastanov 10y ago
    An example among others: I tag certain cells as "initialization cells" and they run automatically when a kernel is restarted (it requires a plugin).
• krick 10y ago
  Maybe it's just me, but the former table style seems much more readable and, well, compact, even if you claim otherwise. I hope, this behavior is customizable?
  • guidopallemans 10y ago
    You can inject custom css through a ~/.jupyter/custom/custom.css file
  • wassyape 10y ago
    I agree. It could be because our brains are used to the old style but there are some objective drawbacks. The new style right aligns everything which is good for numbers but bad for text. Also, it might just be the screenshot, but the contrast is poorer and the column sizes aren't as fitted. It's a pity, I would rather have a decent default than to customize every notebook.
• jfdi 10y ago
  If you want to install this as part of a more integrated sciences environment check out vnode https://github.com/thomaswilley/vnode https://github.com/thomaswilley/vnode
• amenod 10y ago
  One major issue for me is copying with mouse select + middle click, which is unfortunately not solved in this release. Ah well. :-/