7 ms·

Voilà turns Jupyter notebooks to standalone web applications

262 points by jpdus 7y ago

sandGorgon 7y ago
This is huge - This basically allows your analysts and data scientists to have a company wide impact by productionizing your code. Plotly dash tried to do this, but was a bit weird to use.
- polotics 7y ago
  I've been using the Jupyter AppMode plugin for a few years, it has been very useful for internal tools. Voilà looks similar, with more sheen, it's going to enable true rapid SPA development.
  - SylvainCorlay 7y ago
    Voila author here: a major difference with AppMode is that AppMode is merely hiding the notebook interface, but still allows for arbitrary code execution. Voila is meant for producing secure web apps.
    - sandGorgon 7y ago
      hi - its very interesting that you talk about security. what's the deployment story here ? in fact, now that i think of it - can you generate an output folder with a Dockerfile ? so a mere docker build will produce a running docker compatible "build" folder ? Also, can you have a mode where you hide the source code ? i just get the output data ? for world viewable dashboards.
      - SylvainCorlay 7y ago
        There are some details about this in the blog post. - no execution request is sent by the front-end. - code is stripped out unless explicitely stated otherwise. For deployment, we are working on several scenarios around JupyterHub.
        sandGorgon 7y ago
        your blog post still displays the code. Not sure if that can be turned off as well. Please consider the request for a Docker based output build in addition to whatever you do for jupyterhub
        hobofan 7y ago
        From the blog post: > with --strip_sources=False, input cells will be included in the resulting web application (as read-only pygment snippets). So the sources are presumably not there by default.
- discordance 7y ago
  I still feel like R/Shiny is miles ahead here, but Voilà looks like a step in the right direction
  - randomvectors 7y ago
    Shiny is okay as a way to share simple analytical apps but it's syntax and performance are atrocious.
blackrobe 7y ago
I wonder if this works with Jupyterhub-spawned notebooks, definitely want to try.
77ko 7y ago
This is awesome! Is there a way to host these somewhere easily? Like a git push of a notebook to a repo and you get voila powered renders (and I assume some kind of automagic lightweight kernel behind the scenes which executes the code if required). It reminds me of DeepNote[0] though that is geared towards the producers of notebooks rather than consumers. [0]: https://beta.deepnote.org/ https://beta.deepnote.org/
- isoprophlex 7y ago
  I'm going to try this out at work, using the following approach: - gitlab repo that hosts the notebook - gitlab cicd configured to build a docker container running Voila, on every push to dev or master branch - automated deployment of freshly built containers to kubernetes Don't know if there's easier solutions, this will require some setting up
- eoinmurray92 7y ago
  At Kyso [1] we let you do this (I'm one of the founders) - you can run Jupyterlab online and start servers inside Jupyterlab that you can share with the world (Jupyterlab will run on instance-id.cloud.kyso.io). You can install viola and then just run it on port 8000 from within the jupyterlab terminal and you can share the url with anyone (it will be instance-id-8000.cloud.kyso.io) We're actually thinking of supporting Viola natively in the next few weeks, what kind of app do you want to run? [1] https://kyso.io https://kyso.io
mlevental 7y ago
why just jupyter? why not pure python as well?
- mkl 7y ago
  What do you mean? Jupyter notebooks are usually pure Python apart from (sometimes) interactive widgets, which have Javascript behind them since it's in a browser. I don't think this is much use without widgets, so maybe you mean something else?
  - mlevental 7y ago
    I mean why can't I write python outside of jupyter and generate these dashboards? Or can I?
    - deleted 7y ago
      [deleted]
    - rhizome31 7y ago
      If you want to avoid Jupyter Notebooks you can look into Dash or Bokeh Server. Jupyter is convenient though :)
    - jbednar 7y ago
      You can if you use Panel (panel.pyviz.org); it does not depend on Jupyter in any way (but like Voila, works well in Jupyter).
enriquto 7y ago
so, it has come to that
eoinmurray92 7y ago
If it spawns a new kernel for each request it will be very slow no?
- eoinmurray92 7y ago
  Or at least each page load will be slow and memory intensive for the server
  - timkpaine 7y ago
    It is a bit slow, but there's some exploration of keeping a hot pool of prexecuted kernels available. At the end of the day it's still 1 user 1 kernel though, and this is important for security/isolation reasons.
bhl 7y ago
Is there any work on bringing Python into the browser like with JavaScript? At the beginning of the year, I was looking at the best way of making an interactive textbook either with Python (requiring a backend it seemed), or with JavaScript via D3. You can still use Jupyter Notebooks with JS; there's a library called BeakerX by Two Sigma that does this. I went the D3 route but it was hard to stay motivated when most of the time I was just learning how d3 and JavaScript worked instead of the interactability part itself. Closet project I found is Pyodide which was posted here [1]. Wondering if I should wait for someone to come up with a magicul implementation before restarting since this area seems to be experimental (?). [1] https://news.ycombinator.com/item?id=19677721 https://news.ycombinator.com/item?id=19677721
- meredydd 7y ago
  There is indeed! I'll start with the one I founded: Anvil, which is an end-to-end Web dev system with Python (plus a drag-and-drop UI builder, and lots more good stuff): https://anvil.works https://anvil.works Here's our version of how to turn a Jupyter notebook into a web app: https://anvil.works/blog/jupyter-notebook-to-web-app https://anvil.works/blog/jupyter-notebook-to-web-app (tl;dr there's a library you can `pip install`. It makes a websocket connection to Anvil, and then your code - wherever it's running, including a Jupyter notebook - can do everything you can do in our serverless environment, such as handling requests from browser code.) But of course, we are neither the first nor the last to think Python in the browser is a good idea. A colleague of mine did a survey and comparison of the major open source implementations (including PyIodide, which you mention, and Skulpt, which is what Anvil uses): https://pythontips.com/2019/05/22/running-python-in-the-browser/ https://pythontips.com/2019/05/22/running-python-in-the-brow...
  - celias 7y ago
    The latest podcast from podcast.__init__ is an interview of Meredydd about anvil https://www.pythonpodcast.com/anvil-web-application-development-episode-215/ https://www.pythonpodcast.com/anvil-web-application-developm...
- fleur-de-lotus 7y ago
Kiro 7y ago
For years I thought Jupyter Notebook was hardware, a specific laptop made for scientists. Am I alone thinking it's a confusing name?
- aapjaap 7y ago
  Yes
- purple-again 7y ago
  No you are not alone. I have no crossover with the Python world aside from hearing about it on HN and Reddit. I too assumed it was like a Bloomberg terminal for data scientists and was surprised to discover it’s just a virtual environment wrapper you install. Poor name for the project if you are marketing it outside of your domain which I doubt they care much about.
  - cheez 7y ago
    It's not a virtual environment wrapper... It's closer to an environment for coding in the browser.
- randomvectors 7y ago
  Yes, it's very rare to have the term notebook mixed up in this case. Unless you've only heard of the term "Jupyter Notebook" and have zero context. By the same logic Python can get confused with snakes.
rhizome31 7y ago
There's a somehow similar project called Panel: http://panel.pyviz.org/ http://panel.pyviz.org/ However it's limited to Bokeh widgets whereas Voila is framework and language agnostic. As I understand it Panel allows to start Bokeh servers from notebooks.
- snthpy 7y ago
  I'd also like to know how Voilà compares to Panel. The recent Panel announcement stated that it was widget/graphing library agnostic and mentioned plotly and Altair explicitly. I have not used it myself though.
  - rhizome31 7y ago
    Last time I checked you could indeed plug different graphing libraries but Panel comes with its own widgets API which a set of wrappers around Bokeh widgets. So yes for Plotly and Altair but no ipywigets.
  - jbednar 7y ago
    Panel and Voila attacked the same problem (moving easily between Jupyter and standalone server contexts) from completely opposite directions. This difference has some implications on their design and function. Voila is based on ipywidgets running in Jupyter notebooks, and to make a standalone dashboard they had to create a standalone server that can securely execute Jupyter cells and display the results without allowing arbitrary code execution. The server is thus a work in progress, while the Jupyter integration was already solid. Panel is based on Bokeh models, and because Bokeh models already had a full-blown standalone server, the task for Panel was to make Bokeh models (a) work seamlessly in Jupyter (previously they were awkward and limited in that context), (b) support other plotting libraries (by wrapping everything as a Bokeh model), and (c) have an API that's easier to use than native Bokeh for easy prototyping and design. Solid, secure server support came for free. Once both libraries support each others models (soon!) and get a bit more polished, then off the top of my head, the main differences will be: - Panel can use a Jupyter notebook, but it works equally well with a plain Python file; the notebook is just a source of Python code for it. Panel can be used fully even without Jupyter installed. Voila is closely tied to the Jupyter cell-based execution model, which is good or bad depending on your point of view. - Panel allows you to construct a "server view" of your notebook that can be completely independent of what is shown in the notebook, even though it is specified inside the notebook. I use that capability to have the same notebook go step by step analyzing a given dataset in detail, and then separately designate what should be shown in the server context, which is very handy; the boss sees one view, I work on another, and it all stays in sync. Voila works with notebook cell outputs only, and so I don't think it's possible to have fully different views of your data in the two contexts. - Panel supports building complex GUIs, with hierarchies of nested objects that each define their own editable parameters, without having to tie any of that code to Jupyter, Bokeh, or any other GUI or plotting system. This approach is really important for building large, complex codebases (e.g. simulators or data-analysis systems) that sometimes are used in dashboards, sometimes in notebooks, sometimes in batch runs, and sometimes on e.g. large remote computing systems. I'm sure there are lots more differences, but that's enough for now!
yboris 7y ago
side note: I hope everyone is aware of Jupyter Lab, the successor of Jupyter Notebooks -- it's been basically version 1.0 for over a year: https://blog.jupyter.org/jupyterlab-is-ready-for-users-5a6f039b8906 https://blog.jupyter.org/jupyterlab-is-ready-for-users-5a6f0...
- nnq 7y ago
  ...a lot of us are practically addicted to Jupyter Notebook Extensions [1] features and these aren't compatibly with Jupyter Lab and Jupyter Lab didn't add equivalent builtin features. Once you get used to little things like (a) auto-TOC/outline of current doc in a sidebar (with highlight for current running cells and other goodies), (b) auto scroll to end of output, (c) browser notifications for end of long running calculations (so you can be alerted of stuff being done while browsing in other tabs) etc. etc. you really can't live without them! It's this little things that matter 100x more than freely rearrangeable tab-grids and detachable outputs and terminals... these are cool but it's the other small details that make a bigger difference in productivity! (As someone who also has some frontend web experience I was planning to maybe try and code these and contribute, but I never find the time, and last time I looked I couldn't find a good entry point to start learning about jupyter lab's frontend code...) [1] https://github.com/ipython-contrib/jupyter_contrib_nbextensions https://github.com/ipython-contrib/jupyter_contrib_nbextensi...
  - frankc 7y ago
    I'm not sure what you mean. Jupyter-lab has extensions, such as the toc?
    - wodenokoto 7y ago
      Without knowing anything about what I'm saying, I think parents complaint is that Jupiter notebook extension don't work in labs, and that the labs extension system does not allow for as much customisation, leaving parent to stick to the notebook server.
- dhogan 7y ago
amelius 7y ago
Any real-life examples? The blog only seems to contain animated gifs.
- ovi256 7y ago
  http://voila-gallery.org http://voila-gallery.org is their gallery of examples. It's built using jupyter hub, which spawns a python kernel for every new user. UX is not great, you have to go to http://voila-gallery.org/hub/logout http://voila-gallery.org/hub/logout after you've started an example dashboard to be able to see another.
nknealk 7y ago
If this interests you, check out Papermill and NTeract: https://github.com/nteract/papermill https://github.com/nteract/papermill https://github.com/nteract https://github.com/nteract

262 points by jpdus 7y ago

This is huge - This basically allows your analysts and data scientists to have a company wide impact by productionizing your code. Plotly dash tried to do this, but was a bit weird to use.

polotics 7y ago
I've been using the Jupyter AppMode plugin for a few years, it has been very useful for internal tools. Voilà looks similar, with more sheen, it's going to enable true rapid SPA development.
- SylvainCorlay 7y ago
  Voila author here: a major difference with AppMode is that AppMode is merely hiding the notebook interface, but still allows for arbitrary code execution. Voila is meant for producing secure web apps.
  - sandGorgon 7y ago
    hi - its very interesting that you talk about security. what's the deployment story here ? in fact, now that i think of it - can you generate an output folder with a Dockerfile ? so a mere docker build will produce a running docker compatible "build" folder ? Also, can you have a mode where you hide the source code ? i just get the output data ? for world viewable dashboards.
    - SylvainCorlay 7y ago
      There are some details about this in the blog post. - no execution request is sent by the front-end. - code is stripped out unless explicitely stated otherwise. For deployment, we are working on several scenarios around JupyterHub.
      - sandGorgon 7y ago
        your blog post still displays the code. Not sure if that can be turned off as well. Please consider the request for a Docker based output build in addition to whatever you do for jupyterhub
        hobofan 7y ago
        From the blog post: > with --strip_sources=False, input cells will be included in the resulting web application (as read-only pygment snippets). So the sources are presumably not there by default.
discordance 7y ago
I still feel like R/Shiny is miles ahead here, but Voilà looks like a step in the right direction
- randomvectors 7y ago
  Shiny is okay as a way to share simple analytical apps but it's syntax and performance are atrocious.

blackrobe 7y ago

I wonder if this works with Jupyterhub-spawned notebooks, definitely want to try.

77ko 7y ago

This is awesome! Is there a way to host these somewhere easily? Like a git push of a notebook to a repo and you get voila powered renders (and I assume some kind of automagic lightweight kernel behind the scenes which executes the code if required). It reminds me of DeepNote[0] though that is geared towards the producers of notebooks rather than consumers. [0]: https://beta.deepnote.org/ https://beta.deepnote.org/

isoprophlex 7y ago
I'm going to try this out at work, using the following approach: - gitlab repo that hosts the notebook - gitlab cicd configured to build a docker container running Voila, on every push to dev or master branch - automated deployment of freshly built containers to kubernetes Don't know if there's easier solutions, this will require some setting up
eoinmurray92 7y ago
At Kyso [1] we let you do this (I'm one of the founders) - you can run Jupyterlab online and start servers inside Jupyterlab that you can share with the world (Jupyterlab will run on instance-id.cloud.kyso.io). You can install viola and then just run it on port 8000 from within the jupyterlab terminal and you can share the url with anyone (it will be instance-id-8000.cloud.kyso.io) We're actually thinking of supporting Viola natively in the next few weeks, what kind of app do you want to run? [1] https://kyso.io https://kyso.io

mlevental 7y ago

why just jupyter? why not pure python as well?

mkl 7y ago
What do you mean? Jupyter notebooks are usually pure Python apart from (sometimes) interactive widgets, which have Javascript behind them since it's in a browser. I don't think this is much use without widgets, so maybe you mean something else?
- mlevental 7y ago
  I mean why can't I write python outside of jupyter and generate these dashboards? Or can I?
  - deleted 7y ago
    [deleted]
  - rhizome31 7y ago
    If you want to avoid Jupyter Notebooks you can look into Dash or Bokeh Server. Jupyter is convenient though :)
  - jbednar 7y ago
    You can if you use Panel (panel.pyviz.org); it does not depend on Jupyter in any way (but like Voila, works well in Jupyter).

enriquto 7y ago

so, it has come to that

eoinmurray92 7y ago

If it spawns a new kernel for each request it will be very slow no?

eoinmurray92 7y ago
Or at least each page load will be slow and memory intensive for the server
- timkpaine 7y ago
  It is a bit slow, but there's some exploration of keeping a hot pool of prexecuted kernels available. At the end of the day it's still 1 user 1 kernel though, and this is important for security/isolation reasons.

bhl 7y ago

Is there any work on bringing Python into the browser like with JavaScript? At the beginning of the year, I was looking at the best way of making an interactive textbook either with Python (requiring a backend it seemed), or with JavaScript via D3. You can still use Jupyter Notebooks with JS; there's a library called BeakerX by Two Sigma that does this. I went the D3 route but it was hard to stay motivated when most of the time I was just learning how d3 and JavaScript worked instead of the interactability part itself. Closet project I found is Pyodide which was posted here [1]. Wondering if I should wait for someone to come up with a magicul implementation before restarting since this area seems to be experimental (?). [1] https://news.ycombinator.com/item?id=19677721 https://news.ycombinator.com/item?id=19677721

meredydd 7y ago
There is indeed! I'll start with the one I founded: Anvil, which is an end-to-end Web dev system with Python (plus a drag-and-drop UI builder, and lots more good stuff): https://anvil.works https://anvil.works Here's our version of how to turn a Jupyter notebook into a web app: https://anvil.works/blog/jupyter-notebook-to-web-app https://anvil.works/blog/jupyter-notebook-to-web-app (tl;dr there's a library you can `pip install`. It makes a websocket connection to Anvil, and then your code - wherever it's running, including a Jupyter notebook - can do everything you can do in our serverless environment, such as handling requests from browser code.) But of course, we are neither the first nor the last to think Python in the browser is a good idea. A colleague of mine did a survey and comparison of the major open source implementations (including PyIodide, which you mention, and Skulpt, which is what Anvil uses): https://pythontips.com/2019/05/22/running-python-in-the-browser/ https://pythontips.com/2019/05/22/running-python-in-the-brow...
- celias 7y ago
  The latest podcast from podcast.__init__ is an interview of Meredydd about anvil https://www.pythonpodcast.com/anvil-web-application-development-episode-215/ https://www.pythonpodcast.com/anvil-web-application-developm...
fleur-de-lotus 7y ago

Kiro 7y ago

For years I thought Jupyter Notebook was hardware, a specific laptop made for scientists. Am I alone thinking it's a confusing name?

aapjaap 7y ago
Yes
purple-again 7y ago
No you are not alone. I have no crossover with the Python world aside from hearing about it on HN and Reddit. I too assumed it was like a Bloomberg terminal for data scientists and was surprised to discover it’s just a virtual environment wrapper you install. Poor name for the project if you are marketing it outside of your domain which I doubt they care much about.
- cheez 7y ago
  It's not a virtual environment wrapper... It's closer to an environment for coding in the browser.
randomvectors 7y ago
Yes, it's very rare to have the term notebook mixed up in this case. Unless you've only heard of the term "Jupyter Notebook" and have zero context. By the same logic Python can get confused with snakes.

rhizome31 7y ago

There's a somehow similar project called Panel: http://panel.pyviz.org/ http://panel.pyviz.org/ However it's limited to Bokeh widgets whereas Voila is framework and language agnostic. As I understand it Panel allows to start Bokeh servers from notebooks.

snthpy 7y ago
I'd also like to know how Voilà compares to Panel. The recent Panel announcement stated that it was widget/graphing library agnostic and mentioned plotly and Altair explicitly. I have not used it myself though.
- rhizome31 7y ago
  Last time I checked you could indeed plug different graphing libraries but Panel comes with its own widgets API which a set of wrappers around Bokeh widgets. So yes for Plotly and Altair but no ipywigets.
- jbednar 7y ago
  Panel and Voila attacked the same problem (moving easily between Jupyter and standalone server contexts) from completely opposite directions. This difference has some implications on their design and function. Voila is based on ipywidgets running in Jupyter notebooks, and to make a standalone dashboard they had to create a standalone server that can securely execute Jupyter cells and display the results without allowing arbitrary code execution. The server is thus a work in progress, while the Jupyter integration was already solid. Panel is based on Bokeh models, and because Bokeh models already had a full-blown standalone server, the task for Panel was to make Bokeh models (a) work seamlessly in Jupyter (previously they were awkward and limited in that context), (b) support other plotting libraries (by wrapping everything as a Bokeh model), and (c) have an API that's easier to use than native Bokeh for easy prototyping and design. Solid, secure server support came for free. Once both libraries support each others models (soon!) and get a bit more polished, then off the top of my head, the main differences will be: - Panel can use a Jupyter notebook, but it works equally well with a plain Python file; the notebook is just a source of Python code for it. Panel can be used fully even without Jupyter installed. Voila is closely tied to the Jupyter cell-based execution model, which is good or bad depending on your point of view. - Panel allows you to construct a "server view" of your notebook that can be completely independent of what is shown in the notebook, even though it is specified inside the notebook. I use that capability to have the same notebook go step by step analyzing a given dataset in detail, and then separately designate what should be shown in the server context, which is very handy; the boss sees one view, I work on another, and it all stays in sync. Voila works with notebook cell outputs only, and so I don't think it's possible to have fully different views of your data in the two contexts. - Panel supports building complex GUIs, with hierarchies of nested objects that each define their own editable parameters, without having to tie any of that code to Jupyter, Bokeh, or any other GUI or plotting system. This approach is really important for building large, complex codebases (e.g. simulators or data-analysis systems) that sometimes are used in dashboards, sometimes in notebooks, sometimes in batch runs, and sometimes on e.g. large remote computing systems. I'm sure there are lots more differences, but that's enough for now!

yboris 7y ago

side note: I hope everyone is aware of Jupyter Lab, the successor of Jupyter Notebooks -- it's been basically version 1.0 for over a year: https://blog.jupyter.org/jupyterlab-is-ready-for-users-5a6f039b8906 https://blog.jupyter.org/jupyterlab-is-ready-for-users-5a6f0...

nnq 7y ago
...a lot of us are practically addicted to Jupyter Notebook Extensions [1] features and these aren't compatibly with Jupyter Lab and Jupyter Lab didn't add equivalent builtin features. Once you get used to little things like (a) auto-TOC/outline of current doc in a sidebar (with highlight for current running cells and other goodies), (b) auto scroll to end of output, (c) browser notifications for end of long running calculations (so you can be alerted of stuff being done while browsing in other tabs) etc. etc. you really can't live without them! It's this little things that matter 100x more than freely rearrangeable tab-grids and detachable outputs and terminals... these are cool but it's the other small details that make a bigger difference in productivity! (As someone who also has some frontend web experience I was planning to maybe try and code these and contribute, but I never find the time, and last time I looked I couldn't find a good entry point to start learning about jupyter lab's frontend code...) [1] https://github.com/ipython-contrib/jupyter_contrib_nbextensions https://github.com/ipython-contrib/jupyter_contrib_nbextensi...
- frankc 7y ago
  I'm not sure what you mean. Jupyter-lab has extensions, such as the toc?
  - wodenokoto 7y ago
    Without knowing anything about what I'm saying, I think parents complaint is that Jupiter notebook extension don't work in labs, and that the labs extension system does not allow for as much customisation, leaving parent to stick to the notebook server.
dhogan 7y ago

amelius 7y ago

Any real-life examples? The blog only seems to contain animated gifs.

ovi256 7y ago
http://voila-gallery.org http://voila-gallery.org is their gallery of examples. It's built using jupyter hub, which spawns a python kernel for every new user. UX is not great, you have to go to http://voila-gallery.org/hub/logout http://voila-gallery.org/hub/logout after you've started an example dashboard to be able to see another.

nknealk 7y ago

If this interests you, check out Papermill and NTeract: https://github.com/nteract/papermill https://github.com/nteract/papermill https://github.com/nteract https://github.com/nteract