8 ms·

JupyterLite – WASM-powered Jupyter running in the browser

237 points by ahurmazda 5y ago

ktpsns 5y ago
Despite Pyolite has a miserable performance (20MB of downloads), the overall project direction is correct. I said this already 10 years ago: We don't need more cloud computing but need to empower users end devices again. Jupyter is typically operated on powerful notebooks and not on mobile devices.
- atoav 5y ago
  The solution I use for local jupyter notebooks is nteract [0] which is like a standalone application that can edit/open .ipynb files. It has a few quirks, but works quite good for daily use. [0] https://nteract.io/ https://nteract.io/
  - ipsum2 5y ago
    What do you like about nteract over Jupyter's frontend? The website is devoid of details.
    - rerx 5y ago
      It's convenient to directly open a notebook from a browser download or a slack message without having to fire up jupyter in a terminal and navigate it to that temp file.
      - codefreakxff 5y ago
        I use vscode to open and run notebooks using the jupyter plugin. Works great. Don’t need to fire up the browser
        rdedev 5y ago
        Vscode is now my go-to for opening and running notebooks. The issue I had with the normal jupyter was it it's support for normal python files were bad. With vscode I can work on both normal python files and notebook files in the same window without much overhead
        Godel_unicode 5y ago
        Pycharm is also excellent at this. My biggest problem with the normal jupyter interface is years of reflex that Ctrl+w is for deleting words. Quite annoying in a web browser.
        rerx 5y ago
        Yeah, that's exactly the same reasoning. Because it's more convenient you run your notebooks in a dedicated tool. Nteract or Vscode or Pycharm..
    - atoav 5y ago
      I like that it is a standalone notepad.exe-like program that opens my notebook and just works. I usually use it more for looking at notebooks than writing them tho.
  - kzrdude 5y ago
    vscodium can also open ipynb files, to give an alternative. I try to work mostly with "py:percent" script files now, in the same jupyter notebook style but without saving outputs.
- aiNohY6g 5y ago
  Similar to jupyterlite: https://starboard.gg/jupystar https://starboard.gg/jupystar (and https://starboard.gg https://starboard.gg)
- croes 5y ago
  Isn't that not just Java all over again, but this time with JavaScript?
  - fmajid 5y ago
    No, WASM gets compiled to native code
    - youngtaff 5y ago
      Java bytecode got translated to native too Think we've just got better at runtimes over time (JIT, intermediate formats etc.) and WASM was designed to be good for this rather than needed to work with an already exising bytecode
      - fmajid 5y ago
        Reflection and other language features preclude direct translation of Java bytecode to machine code, whereas WASM is designed to be a portable assembly language, closer to the IL of GCC or LLVM.
        _old_dude_ 5y ago
        java -XX:+PrintCompilation prints all methods/loops generated to native code The reflection API has two issues, a security check each time you call a method and the arguments being transformed to objects. The code is still generated to assembly code but the assembly code is slower because of that overhead.
  - pjmlp 5y ago
    Java and Flash, but now it is Good (TM), because the powers that be decided so. "Everything Old is New Again: Binary Security of WebAssembly" https://www.usenix.org/conference/usenixsecurity20/presentation/lehmann https://www.usenix.org/conference/usenixsecurity20/presentat... So, enjoy the 2nd coming of applets/flash, https://platform.uno/ https://platform.uno/ https://dotnet.microsoft.com/apps/aspnet/web-apps/blazor https://dotnet.microsoft.com/apps/aspnet/web-apps/blazor https://tinygo.org/ https://tinygo.org/ .... favourite stack compiled into WASM.
- throwaway894345 5y ago
  > I said this already 10 years ago: We don't need more cloud computing but need to empower users end devices again. Jupyter is typically operated on powerful notebooks and not on mobile devices. If you’re working with data of any significant size at all it then it doesn’t matter how fast your user device is—it’s so much cheaper (time and network egress costs) to send the computations from a user device to the cloud than to pull tens-thousands of GB of data to your local machine. Moreover, I don’t know of many local machines with tens of CPU cores, hundreds or thousands of GB of RAM, or tens-hundreds of TB of SSD for handling that computation quickly. User devices are great for very small data, but I don’t see the point for larger datasets.
  - OldTimeCoffee 5y ago
    Most users don't have 'tens-thousands' of GB of data as part of their use case. You're describing a business case, not an end user consumer case.
    - throwaway894345 5y ago
      I don’t believe that the majority of Jupyterhub users are “consumers” rather than professionals, but more importantly, that doesn’t change the fact that the professional use case exist and isn’t amenable to the fat client approach. As an anecdote, I work on a Jupyterhub managed service offering with customers in both the private and public sectors and our data sizes are pretty much all in this range.
      - deleted 5y ago
        [deleted]
      - mrtesthah 5y ago
        > I work on a Jupyterhub managed service offering ... That sounds like selection bias.
        throwaway894345 5y ago
- jcims 5y ago
  Unfortunately we will need more cloud computing. If you're watching what's going on in the ransomware and cyber insurance space, small and many medium-sized companies that require E&O coverage for their contracts are not going to be able to afford to run on their own equipment.
  - Godel_unicode 5y ago
    Good. People who are bad at administering computers will stop doing it, and will focus on what they're good at. Then, we can use the on-demand nature of cloud services to reduce their power consumption. Simultaneously we can move that consumption into renewable-powered datacenters. This is literally better for everyone.
    - jcims 5y ago
      Meanwhile we create tremendous concentration risk and the world pays rent to Amazon, Google and Microsoft? I wouldn't call that 'good'.
      - deleted 5y ago
        [deleted]
      - Godel_unicode 5y ago
        I don't understand your argument, people should do things they're bad at to keep some other company from getting too big? And oh by the way they'll now probably be using more and less sustainable power, but at least they'll be less secure. That sounds like a very high price to pay for ideological purity...
airocker 5y ago
The kernels will still run on the cloud I believe. It would be great if Jupiter works better for larger programs and ui development.
- qbasic_forever 5y ago
  No, read more from the README. This uses pyodide, a WASM port of desktop Python that runs entirely in the browser (obviously stuff like file access is sandboxed). In addition it adds a web worker that runs a Javascript kernel powered by your browser's JS engine. All of this runs 100% in your browser, there is no server component at all.
qbasic_forever 5y ago
This is _really_ cool. I don't know why a lot of commenters here are going into the weeds to grouse about Java, flash, and general anger at computational notebooks. What we have here is a complete client-side browser environment for development. Not some half-assed language or hyper restricted toy--this is real Python, and your browser's full JS engine all available in JupyterLab's IDE (basically a simpler VS Code at this point, it uses the same editing component). We all freaked out a bit as Apple drove out IDEs from their app store, Google locked down Termux and similar developer tools from Android. Well, here's the answer to those situations. Something no app store owner can kill on a whim. I love stuff like this and hope it helps to enable and inspire the next generation of developers.
bhl 5y ago
How far are we away from having _collaborative_ Jupyter in the browser? Would love a Google Docs experience of sharing a no-sign up required link to help remotely teach basic python.
- mcintyre1994 5y ago
  Livebook does this! It only runs Elixir for now, but there is an issue to add other languages in future. It’s a really cool project IMO. https://github.com/elixir-nx/livebook https://github.com/elixir-nx/livebook
- qbasic_forever 5y ago
  Not that far, someone just needs to make a JupyterLab plugin that uses Automerge or a similar OT/CRDT structure for collaborative editing documents in a workspace (perhaps using WebRTC data channels for P2P sync between clients, or stick with the tried and true server model like Google docs). The trouble is turning that into something as polished and secure as Google Docs collaborative editing experience--there's a _lot_ of work to get there with tons of little corner cases, security issues (you're potentially giving strangers over the internet access to remotely run code in your browser--that should raise big alarm bells), etc. to think through. But the basic stuff is all out there for someone motivated to pick up and go wild with.
  - jtpx 5y ago
    There is a PR to add initial support for real time collaboration in JupyterLite: https://github.com/jtpio/jupyterlite/pull/109 https://github.com/jtpio/jupyterlite/pull/109 This reuses almost all the RTC work done upstream in JupyterLab itself. And since this is implemented as a regular JupyterLab plugin, folks will then be able to swap it for something else and implement their own if they want to, as a federated extension.
- dynamicwebpaige 5y ago
  JupyterLab does support real-time collaborative editing in the browser, as of a few weeks ago. https://github.com/jupyterlab/rtc https://github.com/jupyterlab/rtc
brumar 5y ago
I use frequently the basthon notebook https://notebook.basthon.fr/ https://notebook.basthon.fr/ which is also a wasm powered jupyter (based on pyodide) and quite like it. It's flying a bit under the radar as it's not translated in other langage than french. How does it compare to this project?
el_oni 5y ago
Just to throw in my 2c. I'm a big fan of jupyter notebooks for data visualisation and prototyping pipelines. Being able to run all of your data analysis code then run your visualisation code without having to repipe your data is great. For other projects such as bots, webapps or regular programming I'll use vscode or neovim. Which brings me to a neat feature in vscode that has me starting to do my data vis there. The `# %%` in python files lets me run code blocks in the same way as a notebook, but then i can run the file top to bottom aswell. It is starting to change the way i prototype. Not to mention it can open and edit .ipynb files

237 points by ahurmazda 5y ago

Despite Pyolite has a miserable performance (20MB of downloads), the overall project direction is correct. I said this already 10 years ago: We don't need more cloud computing but need to empower users end devices again. Jupyter is typically operated on powerful notebooks and not on mobile devices.

atoav 5y ago
The solution I use for local jupyter notebooks is nteract [0] which is like a standalone application that can edit/open .ipynb files. It has a few quirks, but works quite good for daily use. [0] https://nteract.io/ https://nteract.io/
- ipsum2 5y ago
  What do you like about nteract over Jupyter's frontend? The website is devoid of details.
  - rerx 5y ago
    It's convenient to directly open a notebook from a browser download or a slack message without having to fire up jupyter in a terminal and navigate it to that temp file.
    - codefreakxff 5y ago
      I use vscode to open and run notebooks using the jupyter plugin. Works great. Don’t need to fire up the browser
      - rdedev 5y ago
        Vscode is now my go-to for opening and running notebooks. The issue I had with the normal jupyter was it it's support for normal python files were bad. With vscode I can work on both normal python files and notebook files in the same window without much overhead
      - Godel_unicode 5y ago
        Pycharm is also excellent at this. My biggest problem with the normal jupyter interface is years of reflex that Ctrl+w is for deleting words. Quite annoying in a web browser.
      - rerx 5y ago
        Yeah, that's exactly the same reasoning. Because it's more convenient you run your notebooks in a dedicated tool. Nteract or Vscode or Pycharm..
  - atoav 5y ago
    I like that it is a standalone notepad.exe-like program that opens my notebook and just works. I usually use it more for looking at notebooks than writing them tho.
- kzrdude 5y ago
  vscodium can also open ipynb files, to give an alternative. I try to work mostly with "py:percent" script files now, in the same jupyter notebook style but without saving outputs.
aiNohY6g 5y ago
Similar to jupyterlite: https://starboard.gg/jupystar https://starboard.gg/jupystar (and https://starboard.gg https://starboard.gg)
croes 5y ago
Isn't that not just Java all over again, but this time with JavaScript?
- fmajid 5y ago
  No, WASM gets compiled to native code
  - youngtaff 5y ago
    Java bytecode got translated to native too Think we've just got better at runtimes over time (JIT, intermediate formats etc.) and WASM was designed to be good for this rather than needed to work with an already exising bytecode
    - fmajid 5y ago
      Reflection and other language features preclude direct translation of Java bytecode to machine code, whereas WASM is designed to be a portable assembly language, closer to the IL of GCC or LLVM.
      - _old_dude_ 5y ago
        java -XX:+PrintCompilation prints all methods/loops generated to native code The reflection API has two issues, a security check each time you call a method and the arguments being transformed to objects. The code is still generated to assembly code but the assembly code is slower because of that overhead.
- pjmlp 5y ago
  Java and Flash, but now it is Good (TM), because the powers that be decided so. "Everything Old is New Again: Binary Security of WebAssembly" https://www.usenix.org/conference/usenixsecurity20/presentation/lehmann https://www.usenix.org/conference/usenixsecurity20/presentat... So, enjoy the 2nd coming of applets/flash, https://platform.uno/ https://platform.uno/ https://dotnet.microsoft.com/apps/aspnet/web-apps/blazor https://dotnet.microsoft.com/apps/aspnet/web-apps/blazor https://tinygo.org/ https://tinygo.org/ .... favourite stack compiled into WASM.
throwaway894345 5y ago
> I said this already 10 years ago: We don't need more cloud computing but need to empower users end devices again. Jupyter is typically operated on powerful notebooks and not on mobile devices. If you’re working with data of any significant size at all it then it doesn’t matter how fast your user device is—it’s so much cheaper (time and network egress costs) to send the computations from a user device to the cloud than to pull tens-thousands of GB of data to your local machine. Moreover, I don’t know of many local machines with tens of CPU cores, hundreds or thousands of GB of RAM, or tens-hundreds of TB of SSD for handling that computation quickly. User devices are great for very small data, but I don’t see the point for larger datasets.
- OldTimeCoffee 5y ago
  Most users don't have 'tens-thousands' of GB of data as part of their use case. You're describing a business case, not an end user consumer case.
  - throwaway894345 5y ago
    I don’t believe that the majority of Jupyterhub users are “consumers” rather than professionals, but more importantly, that doesn’t change the fact that the professional use case exist and isn’t amenable to the fat client approach. As an anecdote, I work on a Jupyterhub managed service offering with customers in both the private and public sectors and our data sizes are pretty much all in this range.
    - deleted 5y ago
      [deleted]
    - mrtesthah 5y ago
      > I work on a Jupyterhub managed service offering ... That sounds like selection bias.
      - throwaway894345 5y ago
jcims 5y ago
Unfortunately we will need more cloud computing. If you're watching what's going on in the ransomware and cyber insurance space, small and many medium-sized companies that require E&O coverage for their contracts are not going to be able to afford to run on their own equipment.
- Godel_unicode 5y ago
  Good. People who are bad at administering computers will stop doing it, and will focus on what they're good at. Then, we can use the on-demand nature of cloud services to reduce their power consumption. Simultaneously we can move that consumption into renewable-powered datacenters. This is literally better for everyone.
  - jcims 5y ago
    Meanwhile we create tremendous concentration risk and the world pays rent to Amazon, Google and Microsoft? I wouldn't call that 'good'.
    - deleted 5y ago
      [deleted]
    - Godel_unicode 5y ago
      I don't understand your argument, people should do things they're bad at to keep some other company from getting too big? And oh by the way they'll now probably be using more and less sustainable power, but at least they'll be less secure. That sounds like a very high price to pay for ideological purity...

airocker 5y ago

The kernels will still run on the cloud I believe. It would be great if Jupiter works better for larger programs and ui development.

qbasic_forever 5y ago
No, read more from the README. This uses pyodide, a WASM port of desktop Python that runs entirely in the browser (obviously stuff like file access is sandboxed). In addition it adds a web worker that runs a Javascript kernel powered by your browser's JS engine. All of this runs 100% in your browser, there is no server component at all.

qbasic_forever 5y ago

This is _really_ cool. I don't know why a lot of commenters here are going into the weeds to grouse about Java, flash, and general anger at computational notebooks. What we have here is a complete client-side browser environment for development. Not some half-assed language or hyper restricted toy--this is real Python, and your browser's full JS engine all available in JupyterLab's IDE (basically a simpler VS Code at this point, it uses the same editing component). We all freaked out a bit as Apple drove out IDEs from their app store, Google locked down Termux and similar developer tools from Android. Well, here's the answer to those situations. Something no app store owner can kill on a whim. I love stuff like this and hope it helps to enable and inspire the next generation of developers.

bhl 5y ago

How far are we away from having _collaborative_ Jupyter in the browser? Would love a Google Docs experience of sharing a no-sign up required link to help remotely teach basic python.

mcintyre1994 5y ago
Livebook does this! It only runs Elixir for now, but there is an issue to add other languages in future. It’s a really cool project IMO. https://github.com/elixir-nx/livebook https://github.com/elixir-nx/livebook
qbasic_forever 5y ago
Not that far, someone just needs to make a JupyterLab plugin that uses Automerge or a similar OT/CRDT structure for collaborative editing documents in a workspace (perhaps using WebRTC data channels for P2P sync between clients, or stick with the tried and true server model like Google docs). The trouble is turning that into something as polished and secure as Google Docs collaborative editing experience--there's a _lot_ of work to get there with tons of little corner cases, security issues (you're potentially giving strangers over the internet access to remotely run code in your browser--that should raise big alarm bells), etc. to think through. But the basic stuff is all out there for someone motivated to pick up and go wild with.
- jtpx 5y ago
  There is a PR to add initial support for real time collaboration in JupyterLite: https://github.com/jtpio/jupyterlite/pull/109 https://github.com/jtpio/jupyterlite/pull/109 This reuses almost all the RTC work done upstream in JupyterLab itself. And since this is implemented as a regular JupyterLab plugin, folks will then be able to swap it for something else and implement their own if they want to, as a federated extension.
dynamicwebpaige 5y ago
JupyterLab does support real-time collaborative editing in the browser, as of a few weeks ago. https://github.com/jupyterlab/rtc https://github.com/jupyterlab/rtc

brumar 5y ago

I use frequently the basthon notebook https://notebook.basthon.fr/ https://notebook.basthon.fr/ which is also a wasm powered jupyter (based on pyodide) and quite like it. It's flying a bit under the radar as it's not translated in other langage than french. How does it compare to this project?

el_oni 5y ago

Just to throw in my 2c. I'm a big fan of jupyter notebooks for data visualisation and prototyping pipelines. Being able to run all of your data analysis code then run your visualisation code without having to repipe your data is great. For other projects such as bots, webapps or regular programming I'll use vscode or neovim. Which brings me to a neat feature in vscode that has me starting to do my data vis there. The `# %%` in python files lets me run code blocks in the same way as a notebook, but then i can run the file top to bottom aswell. It is starting to change the way i prototype. Not to mention it can open and edit .ipynb files