11 ms·

WebAssembly: Adding Python support to WASM language runtimes

216 points by assambar 4y ago

assambar 4y ago
Ready-to-use python.wasm, also in a Docker+Wasm container image.
- still_grokking 4y ago
  But please don't forget to wrap it in at least some VM! /s That's not even funny, as in real life people would run something like that actually in a VM. So we have now: HW memory protection -> HW virtualization -> VM -> OS -> Docker -> WASM -> language runtime -> some code snippet. Things become quite crazy these days, to be honest…
  - assambar 4y ago
    I totally agree with you on how ridiculous this is. But it's all about trade offs in the end. In the case of Docker+Wasm you get portability and delivery speed. You build one image for the wasm32/wasi platform and now it can run on any architecture that has Docker, just like that. And also you don't carry the whole OS baggage of a typical container. Only the language runtime. So download times are faster. --- Lastly, the Docker deployment is just the easiest way for a WASM application to reach people. They can try it out without the necessity to setup and support a WASI runtime.
simonw 4y ago
This looks very promising! The thing I most want to solve right now is this: I want to write a regular Python application that can safely execute untrusted Python code in a WASM sandbox as part of its execution. I want to do this so I can let end users customize my web applications in weird and interesting ways by pasting their own Python code into a textarea - think features like "run this Python code to transform my stored data" - without them being able to break my system. This feels like it should be pretty easy with WebAssembly! It's the classic code sandboxing problem - long a big challenge in Python world - finally solved in a robust way. I've been finding it surprisingly hard to get a proof-of-concept of this working though. Essentially I want to be able to do this, in my regular Python code: import some_webassembly_engine python = some_webassembly_engine.load( "python.wasm", max_cpu_time_in_seconds=3.0, max_allowed_memory_in_bytes=32000000 ) result = python.execute("3 + 5") I've not yet figured out the incantations I need to actually do this - in particular the limits on CPU and memory time. I posed this question on Mastodon recently and Jim Kring put together this demo, which gets most of the way there (albeit using an old Python 3.6 build): https://github.com/jimkring/python-sandbox-wasm https://github.com/jimkring/python-sandbox-wasm It doesn't feel like this should be as hard to figure out as it is!
- samsquire 4y ago
  This would be great. And with an exposeable API for safety a memory safe API that could be exposed to wasm applications. And rate limited.
- mike_hearn 4y ago
  FWIW although it's not WebAssembly based, you can do that with GraalVM. It has a concept of language contexts which can be sandboxed including those constraints. There are two caveats: 1. Sandboxing for CPU time and max allowed memory requires the enterprise edition, so you'd have to pay for it. 2. The Python engine isn't 100% compatible with regular Python, although that may not matter for your use case as the compatibility is pretty good and issues mostly show up around extension modules.
  - sebk 4y ago
    Unfortunately there are at least two more major caveats: 1. Capability control only works for JavaScript (https://www.graalvm.org/latest/reference-manual/embed-languages/#host-access https://www.graalvm.org/latest/reference-manual/embed-langua...) 2. The documentation says in no uncertain terms that running untrusted code is unsupported (https://www.graalvm.org/latest/security-guide/#security-model https://www.graalvm.org/latest/security-guide/#security-mode...)
- callahad 4y ago
  The startup I'm working at is basically trying to do exactly that as a service, but a one-off thing for a regular Python application shouldn't be as hard to figure out as it is. Can you link to the Mastodon thread (darn lack of search!) and we can continue there?
  - simonw 4y ago
    Here's the Mastodon conversation: https://fedi.simonwillison.net/@simon/109682777068881522 https://fedi.simonwillison.net/@simon/109682777068881522 (I'm so close to building my own search engine just against my own content there.)
c120 4y ago
So for someone who has python installed locally, what's the point? Is it just the sandbox or is there anything else I'm missing?
- chc 4y ago
  If you're just looking to run trusted scripts locally, there isn't much point. If you're running a system that uses wasm, this means you can now easily support Python.
- kasajian 4y ago
  It's not for someone who only runs Python locally.
- angelmm 4y ago
  You get an extra layer of isolation, even at your development environment level. I remember a NodeJs CVE that was caused by a poisoned dependency. It was affecting people when downloading it from npm. There’s still a gap here to cover, but the benefits may be worth :)
  - ElectricalUnion 4y ago
    I don't see how this would in any way prevent you from being affected by a equivalent poisoned pypi dependency; after all your secrets/credentials are inside the sandbox anyways or your code can't work.
    - angelmm 4y ago
      With Wasm + WASI, you need to explicitly mount files and environment variables. Inside the Wasm VM, the Python interpreter, source code and dependencies only have access to a very reduced surface. Although you're right that if you mount credentials inside, they will be accessible too. The incident I was talking about was the event-stream[1] vulnerability. The attacker introduced code that looked for the data of a crypto wallet. This data was stored in the user's home. By default, interpreters may get access to the same resources that the user running the process. In Wasm, the resources are granted manually. [1] https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident https://blog.npmjs.org/post/180565383195/details-about-the-e...
brrrrrm 4y ago
the issue right now with Python support in WASM (at least for machine learning, the main driver of the language) is that Python is largely a wrapper language and none the utilities that make it so powerful (numpy, PyTorch, JAX) work particularly well in wasm, since it's so limited performance-wise (no FMA, no GPU support). I'm excited for pairing wasm with WebGPU, which will likely unblock these projects from building support for the web/untrusted ecosystem. A useful project would be one that makes this integration really easy to build today and a flip of the switch to turn on in the future.
- m00dy 4y ago
  I have integrated pyodide + webgpu recently. (you can do matmul using webgpu's compute pipeline). The real problem is that browser tabs have 4gb max memory size. So, training neural networks on this stack is almost impossible. ( I don't even want to mention pyTorch's dependency hell).
  - brrrrrm 4y ago
    My claim is that it’s not easy, not impossible. There’s little incentive to hack in JavaScript or maintain a Pyodide compatible build. The 4gb limit isn’t a technical limitation, just a standards thing (it could change easily).
  - miohtama 4y ago
    WebAssembly Memory64 is coming https://webassembly.org/roadmap/ https://webassembly.org/roadmap/
- mjw1007 4y ago
  I've come across this notion that nowadays machine learning provides (in some sense) the biggest group of Python users a few times recently. What reason is there to suppose this is true? It seems surprising to me.
  - claytonjy 4y ago
    It's really hard to do much ML in anything _except_ python. Virtually everyone improving the ML ecosystems of other language got their start in Python and are knowingly competing with Python (e.g. R, Julia). If you want to get started in ML today, python is the obvious easiest path forward. So, most ML users are python users. I don't know how that group compares to non-ML python users, but I have a feeling there isn't a flood of eager new Django devs the way there is Pytorch users. Most non-ML things you could do with python can be done similarly well in Go/Rust/Typescript, but there's no other option for most ML stuff.
dom96 4y ago
There seems to be so many different variants of the same thing out there. What makes this unique? For example I know Pyodide exists and also runs CPython under WASM.
- ridruejo 4y ago
  This one is designed to run on the server side and interface with the OS via WASI, so it can read/write files etc
  - dom96 4y ago
    Pyodide can do so too, though through an emscripten interface. I managed to get it running on Cloudflare Workers as a POC[0] (though I should add a disclaimer that I work at Cloudflare) 0 - https://github.com/dom96/pycloud https://github.com/dom96/pycloud
robertlagrant 4y ago
The non-Docker version seems to require an external site-packages, unless I missed it. Is it possible to produce a single wasm binary with all dependencies compiled in?
- angelmm 4y ago
  Hey! Dev here :) For external libraries, it requires you to mount the libraries with WASI when running the python.wasm module. Another option we're exploring is to use wasi-vfs[1] to include some common modules in our pre-built binaries. For example, Ruby does require some extra libraries for common workloads (like JSON parsing). This is still on the exploration phase, but we may do something with it. [1] https://github.com/kateinoigakukun/wasi-vfs https://github.com/kateinoigakukun/wasi-vfs
  - robertlagrant 4y ago
    Very cool. We ship some Python as a Debian dependency and so this could become a really interesting way to package everything up.
- seddonm1 4y ago
  I have been following and playing with this repository: https://github.com/singlestore-labs/python-wasi/ https://github.com/singlestore-labs/python-wasi/ It builds a single Python WASM module with all dependencies included (they use VFS) and a Dockerfile to make the process easy (and actually worked first go). It does produce large files though: wasi-python3.11.wasm 110MB
  - ridruejo 4y ago
    Yes! Single store is a great team. We are currently using some of their work for this Python release, like libz
dayeye2006 4y ago
Can anyone give me a ELI5 version what is the relationship between this and pyodie?
- ridruejo 4y ago
  Pyodide is for the browser, this is intended for server side environments, so it can interact with files, sockets etc via WASI standard
AshleysBrain 4y ago
How does this handle garbage collection? AFAIK the WebAssembly GC proposal is still in development. Does it implement GC in WASM code?
- amelius 4y ago
  Perhaps it just uses Python's built-in garbage collector that just increases/decreases the data segment size as needed by calling sbrk()?
  - ridruejo 4y ago
    Correct, it is just CPython compiled to Wasm (similar to compiling to x86 or arm)
eddhead 4y ago
If Pygame can do wasm outputs to WebGL/WebGPU it'll move things along quite a bit for game development
pjmlp 4y ago
This looks like a solution looking for a problem, like many others regarding the use of WebAssembly outside of the browser.
- ridruejo 4y ago
  Our most immediate goal is to be able to run traditional web apps written in PHP, Ruby, Python mostly unchanged but with the additional layer of isolation and sandboxing. We see this (once fully matured) as a great way to improve security without changing how web developers work
  - pjmlp 4y ago
    Containers and similar OS security features already provide that.
    - awwaiid 4y ago
      Exploring and providing new and different approaches is worthwhile, and I doubt the tradeoffs of each technology is identical.
      - pjmlp 4y ago
        Except these aren't new approaches. https://www.ibm.com/docs/en/zos/2.4.0?topic=descriptions-language-environment https://www.ibm.com/docs/en/zos/2.4.0?topic=descriptions-lan...
    - ridruejo 4y ago
      Not with that level of granularity. Wasm is able to run in severely constrained environments (ie microcontrollers) and non-posix OS that may not be able to support containers
      - pjmlp 4y ago
        Like Western Digital having hard disk controllers running P-Code in 1979. https://en.wikipedia.org/wiki/Pascal_MicroEngine https://en.wikipedia.org/wiki/Pascal_MicroEngine
3Sophons 4y ago
Sidenote: WasmEdge 0.12 (to be released in Feb) will be fully compatible with wasmtime’s socket API. There will be no need for separate builds soon. In addition to wasmtime socket, WasmEdge supports async non-blocking socket for high-performance apps
- ridruejo 4y ago
  Looking forward to it!
jokoon 4y ago
I thought this was running python in the browser. I don't understand what's the point of this article. I'm already okay with brython, which is fast enough, but sometimes it generates JavaScript errors, which are difficult to understand.
- Havoc 4y ago
  I believe this is envisioned a bit more like docker-lite minus the os image layer?

216 points by assambar 4y ago

assambar 4y ago

Ready-to-use python.wasm, also in a Docker+Wasm container image.

still_grokking 4y ago
But please don't forget to wrap it in at least some VM! /s That's not even funny, as in real life people would run something like that actually in a VM. So we have now: HW memory protection -> HW virtualization -> VM -> OS -> Docker -> WASM -> language runtime -> some code snippet. Things become quite crazy these days, to be honest…
- assambar 4y ago
  I totally agree with you on how ridiculous this is. But it's all about trade offs in the end. In the case of Docker+Wasm you get portability and delivery speed. You build one image for the wasm32/wasi platform and now it can run on any architecture that has Docker, just like that. And also you don't carry the whole OS baggage of a typical container. Only the language runtime. So download times are faster. --- Lastly, the Docker deployment is just the easiest way for a WASM application to reach people. They can try it out without the necessity to setup and support a WASI runtime.

simonw 4y ago

This looks very promising! The thing I most want to solve right now is this: I want to write a regular Python application that can safely execute untrusted Python code in a WASM sandbox as part of its execution. I want to do this so I can let end users customize my web applications in weird and interesting ways by pasting their own Python code into a textarea - think features like "run this Python code to transform my stored data" - without them being able to break my system. This feels like it should be pretty easy with WebAssembly! It's the classic code sandboxing problem - long a big challenge in Python world - finally solved in a robust way. I've been finding it surprisingly hard to get a proof-of-concept of this working though. Essentially I want to be able to do this, in my regular Python code: import some_webassembly_engine python = some_webassembly_engine.load( "python.wasm", max_cpu_time_in_seconds=3.0, max_allowed_memory_in_bytes=32000000 ) result = python.execute("3 + 5") I've not yet figured out the incantations I need to actually do this - in particular the limits on CPU and memory time. I posed this question on Mastodon recently and Jim Kring put together this demo, which gets most of the way there (albeit using an old Python 3.6 build): https://github.com/jimkring/python-sandbox-wasm https://github.com/jimkring/python-sandbox-wasm It doesn't feel like this should be as hard to figure out as it is!

samsquire 4y ago
This would be great. And with an exposeable API for safety a memory safe API that could be exposed to wasm applications. And rate limited.
mike_hearn 4y ago
FWIW although it's not WebAssembly based, you can do that with GraalVM. It has a concept of language contexts which can be sandboxed including those constraints. There are two caveats: 1. Sandboxing for CPU time and max allowed memory requires the enterprise edition, so you'd have to pay for it. 2. The Python engine isn't 100% compatible with regular Python, although that may not matter for your use case as the compatibility is pretty good and issues mostly show up around extension modules.
- sebk 4y ago
  Unfortunately there are at least two more major caveats: 1. Capability control only works for JavaScript (https://www.graalvm.org/latest/reference-manual/embed-languages/#host-access https://www.graalvm.org/latest/reference-manual/embed-langua...) 2. The documentation says in no uncertain terms that running untrusted code is unsupported (https://www.graalvm.org/latest/security-guide/#security-model https://www.graalvm.org/latest/security-guide/#security-mode...)
callahad 4y ago
The startup I'm working at is basically trying to do exactly that as a service, but a one-off thing for a regular Python application shouldn't be as hard to figure out as it is. Can you link to the Mastodon thread (darn lack of search!) and we can continue there?
- simonw 4y ago
  Here's the Mastodon conversation: https://fedi.simonwillison.net/@simon/109682777068881522 https://fedi.simonwillison.net/@simon/109682777068881522 (I'm so close to building my own search engine just against my own content there.)

c120 4y ago

So for someone who has python installed locally, what's the point? Is it just the sandbox or is there anything else I'm missing?

chc 4y ago
If you're just looking to run trusted scripts locally, there isn't much point. If you're running a system that uses wasm, this means you can now easily support Python.
kasajian 4y ago
It's not for someone who only runs Python locally.
angelmm 4y ago
You get an extra layer of isolation, even at your development environment level. I remember a NodeJs CVE that was caused by a poisoned dependency. It was affecting people when downloading it from npm. There’s still a gap here to cover, but the benefits may be worth :)
- ElectricalUnion 4y ago
  I don't see how this would in any way prevent you from being affected by a equivalent poisoned pypi dependency; after all your secrets/credentials are inside the sandbox anyways or your code can't work.
  - angelmm 4y ago
    With Wasm + WASI, you need to explicitly mount files and environment variables. Inside the Wasm VM, the Python interpreter, source code and dependencies only have access to a very reduced surface. Although you're right that if you mount credentials inside, they will be accessible too. The incident I was talking about was the event-stream[1] vulnerability. The attacker introduced code that looked for the data of a crypto wallet. This data was stored in the user's home. By default, interpreters may get access to the same resources that the user running the process. In Wasm, the resources are granted manually. [1] https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident https://blog.npmjs.org/post/180565383195/details-about-the-e...

brrrrrm 4y ago

the issue right now with Python support in WASM (at least for machine learning, the main driver of the language) is that Python is largely a wrapper language and none the utilities that make it so powerful (numpy, PyTorch, JAX) work particularly well in wasm, since it's so limited performance-wise (no FMA, no GPU support). I'm excited for pairing wasm with WebGPU, which will likely unblock these projects from building support for the web/untrusted ecosystem. A useful project would be one that makes this integration really easy to build today and a flip of the switch to turn on in the future.

m00dy 4y ago
I have integrated pyodide + webgpu recently. (you can do matmul using webgpu's compute pipeline). The real problem is that browser tabs have 4gb max memory size. So, training neural networks on this stack is almost impossible. ( I don't even want to mention pyTorch's dependency hell).
- brrrrrm 4y ago
  My claim is that it’s not easy, not impossible. There’s little incentive to hack in JavaScript or maintain a Pyodide compatible build. The 4gb limit isn’t a technical limitation, just a standards thing (it could change easily).
- miohtama 4y ago
  WebAssembly Memory64 is coming https://webassembly.org/roadmap/ https://webassembly.org/roadmap/
mjw1007 4y ago
I've come across this notion that nowadays machine learning provides (in some sense) the biggest group of Python users a few times recently. What reason is there to suppose this is true? It seems surprising to me.
- claytonjy 4y ago
  It's really hard to do much ML in anything _except_ python. Virtually everyone improving the ML ecosystems of other language got their start in Python and are knowingly competing with Python (e.g. R, Julia). If you want to get started in ML today, python is the obvious easiest path forward. So, most ML users are python users. I don't know how that group compares to non-ML python users, but I have a feeling there isn't a flood of eager new Django devs the way there is Pytorch users. Most non-ML things you could do with python can be done similarly well in Go/Rust/Typescript, but there's no other option for most ML stuff.

dom96 4y ago

There seems to be so many different variants of the same thing out there. What makes this unique? For example I know Pyodide exists and also runs CPython under WASM.

ridruejo 4y ago
This one is designed to run on the server side and interface with the OS via WASI, so it can read/write files etc
- dom96 4y ago
  Pyodide can do so too, though through an emscripten interface. I managed to get it running on Cloudflare Workers as a POC[0] (though I should add a disclaimer that I work at Cloudflare) 0 - https://github.com/dom96/pycloud https://github.com/dom96/pycloud

robertlagrant 4y ago

The non-Docker version seems to require an external site-packages, unless I missed it. Is it possible to produce a single wasm binary with all dependencies compiled in?

angelmm 4y ago
Hey! Dev here :) For external libraries, it requires you to mount the libraries with WASI when running the python.wasm module. Another option we're exploring is to use wasi-vfs[1] to include some common modules in our pre-built binaries. For example, Ruby does require some extra libraries for common workloads (like JSON parsing). This is still on the exploration phase, but we may do something with it. [1] https://github.com/kateinoigakukun/wasi-vfs https://github.com/kateinoigakukun/wasi-vfs
- robertlagrant 4y ago
  Very cool. We ship some Python as a Debian dependency and so this could become a really interesting way to package everything up.
seddonm1 4y ago
I have been following and playing with this repository: https://github.com/singlestore-labs/python-wasi/ https://github.com/singlestore-labs/python-wasi/ It builds a single Python WASM module with all dependencies included (they use VFS) and a Dockerfile to make the process easy (and actually worked first go). It does produce large files though: wasi-python3.11.wasm 110MB
- ridruejo 4y ago
  Yes! Single store is a great team. We are currently using some of their work for this Python release, like libz

dayeye2006 4y ago

Can anyone give me a ELI5 version what is the relationship between this and pyodie?

ridruejo 4y ago
Pyodide is for the browser, this is intended for server side environments, so it can interact with files, sockets etc via WASI standard

AshleysBrain 4y ago

How does this handle garbage collection? AFAIK the WebAssembly GC proposal is still in development. Does it implement GC in WASM code?

amelius 4y ago
Perhaps it just uses Python's built-in garbage collector that just increases/decreases the data segment size as needed by calling sbrk()?
- ridruejo 4y ago
  Correct, it is just CPython compiled to Wasm (similar to compiling to x86 or arm)

eddhead 4y ago

If Pygame can do wasm outputs to WebGL/WebGPU it'll move things along quite a bit for game development

pjmlp 4y ago

This looks like a solution looking for a problem, like many others regarding the use of WebAssembly outside of the browser.

ridruejo 4y ago
Our most immediate goal is to be able to run traditional web apps written in PHP, Ruby, Python mostly unchanged but with the additional layer of isolation and sandboxing. We see this (once fully matured) as a great way to improve security without changing how web developers work
- pjmlp 4y ago
  Containers and similar OS security features already provide that.
  - awwaiid 4y ago
    Exploring and providing new and different approaches is worthwhile, and I doubt the tradeoffs of each technology is identical.
    - pjmlp 4y ago
      Except these aren't new approaches. https://www.ibm.com/docs/en/zos/2.4.0?topic=descriptions-language-environment https://www.ibm.com/docs/en/zos/2.4.0?topic=descriptions-lan...
  - ridruejo 4y ago
    Not with that level of granularity. Wasm is able to run in severely constrained environments (ie microcontrollers) and non-posix OS that may not be able to support containers
    - pjmlp 4y ago
      Like Western Digital having hard disk controllers running P-Code in 1979. https://en.wikipedia.org/wiki/Pascal_MicroEngine https://en.wikipedia.org/wiki/Pascal_MicroEngine

3Sophons 4y ago

Sidenote: WasmEdge 0.12 (to be released in Feb) will be fully compatible with wasmtime’s socket API. There will be no need for separate builds soon. In addition to wasmtime socket, WasmEdge supports async non-blocking socket for high-performance apps

ridruejo 4y ago
Looking forward to it!

jokoon 4y ago

I thought this was running python in the browser. I don't understand what's the point of this article. I'm already okay with brython, which is fast enough, but sometimes it generates JavaScript errors, which are difficult to understand.

Havoc 4y ago
I believe this is envisioned a bit more like docker-lite minus the os image layer?