3 ms·

Right - I trust workerd in the context of Cloudflare because I know it has a team of people who's job it is to keep it secure who are on-call 24/7. The problem

by simonw 10mo ago

Right - I trust workerd in the context of Cloudflare because I know it has a team of people who's job it is to keep it secure who are on-call 24/7. The problem I have is that I'm just one person and I don't want to be on call 24/7 ready to react to sandbox escapes, so I'm hoping I can find a solution that someone else built where they are willing to say "this is safe: you can feed in a string of untrusted JavaScript and we are confident it won't break out again". I think I might be able to get there via WebAssembly (e.g. with QuickJS or MicroQuickJS compiled to WASM) because the whole point of WebAssembly is to solve this one problem. > But if you're not literally accepting code directly from anonymous internet users then the risk may be a lot lower That's the problem: this is exactly what I want to be able to do! I want to build extension systems for my own apps such that users can run their own code or paste in code written by other people and have it execute safely. Similar to Shopify Functions: https://shopify.dev/docs/apps/build/functions https://shopify.dev/docs/apps/build/functions I think the value unlocked by this kind of extension mechanism is ready to skyrocket, because users can use LLMs to help write that code for them.

kentonv 10mo ago
> the whole point of WebAssembly is to solve this one problem. For Wasm to be a secure sandbox, you have to assume a bug-free compiler/interpreter, which, alas, none of them really are. It's a somewhat easier problem than building a bug-free JavaScript runtime, but not by as much as you might expect, sadly. > I want to build extension systems for my own apps such that users can run their own code or paste in code written by other people and have it execute safely. Similar to Shopify Functions: https://shopify.dev/docs/apps/build/functions https://shopify.dev/docs/apps/build/functions Ah, this is exactly the Workers for Platforms use case: https://developers.cloudflare.com/cloudflare-for-platforms/workers-for-platforms/ https://developers.cloudflare.com/cloudflare-for-platforms/w... And indeed, Shopify uses it: https://shopify.engineering/how-we-built-oxygen https://shopify.engineering/how-we-built-oxygen (There's also the upcoming Dynamic Worker Loader API: https://developers.cloudflare.com/workers/runtime-apis/bindings/worker-loader/ https://developers.cloudflare.com/workers/runtime-apis/bindi...) But it sounds like you really do want to self-host? I don't blame you, but that does make it tough. I'm not sure there's any such thing as a secure sandbox that doesn't require some level of monitoring and daily maintenance, sadly. (But admittedly I may be biased.)
- simonw 10mo ago
  Yeah my ideal is to have something that cleanly "pip installs" as a dependency such that users of my open source Python projects can self host tools that let them extend using arbitrary code, including code written by LLMs. I've been picking at this problem for a few years now! On the one hand I get why it's so hard. But it really feels like it should be possible to solve this in 2026 - executing arbitrary code in a way that constrains its memory and CPU time usage is a problem our industry solves in browsers and hosting platforms and databases and all sorts of other places, and has done for decades. The whole LLM-assisted end-user programming thing makes solving this with the right developer affordances so valuable!
  - rattray 10mo ago
    Ah, in that context, why not just give the people workerd? People using & running OSS libraries are used to the fact that there might be vulns in libraries they're using, right?
    - kentonv 10mo ago
      Or put another way: If Simon's users choose to self-host the open source version of his service, they are probably using it to run their own code, and so the sandbox security matters less, and workerd may be fine. The sandbox only matters when Simon himself offers his software as a service, which he could do using Workers for Platforms. (But this is a self-serving argument coming from me.)
  - hiccuphippo 10mo ago
    If the code is gonna be written by LLMs anyway skip the js, directly use wasm and allow any language that compiles to it with something like https://extism.org/ https://extism.org/
neildhar 10mo ago
I worked on a sandbox of Hermes that compiles the engine to wasm, and then converts the wasm to C (like Mozilla's RLBox). It exposes the same C++ API as Hermes so it is pretty convenient to drop in, and should be fairly secure. It hasn't been updated in some time, but it should still be working, and can probably be brought up to date with some small effort: https://github.com/facebook/hermes/tree/static_h/API/hermes_sandbox https://github.com/facebook/hermes/tree/static_h/API/hermes_... EDIT: Reading some of your other comments, I should point out that this is more like a component of a possible solution. It does not attempt to prevent resource exhaustion or crashes due to corrupted internal state.
aardvark179 10mo ago
You need more than a simple sandbox for what you describe. You also need to avoid infinite loops or other constructs from causing a DoS or similar, and if you are doing this with the intention of interacting with some other parts of a system then you need to think about how that interaction works and whether there is a way to execute something outside of that sandbox. Even if you go with something backed by a full time team there is still going to be a chance you have to deal with a security issue in a hurry, maybe in the run up to Christmas. That is just going to come with the territory and if you don’t want to deal with that then you probably need to think about whether you really need a sandbox that can execute untrusted code.
laurencerowe 10mo ago
There's also JCO for creating JS based WebAssembly components. It's based on StarlingMonkey which I believe is slightly faster than QuickJS under web assembly. https://github.com/bytecodealliance/jco https://github.com/bytecodealliance/jco Benchmark numbers for request isolated JS hello world / React page rendering: JCO/wasmtime: 314µs / 13ms Bun process forking: 1.7ms / 8.2ms v8 isolate from snapshot: 0.7ms / 22ms TinyKVM: 52µs / 708µs Native with reuse 14µs / 640µs Numbers taken from our upcoming TinyKVM paper. Benchmark setup code for JCO/wasmtime is here: https://github.com/libriscv/kvmserver/tree/main/examples/wasmtime https://github.com/libriscv/kvmserver/tree/main/examples/was... (I suspect even if we are able to get TinyKVM into a state you'd feel comfortable with in the future it would still be an awkward fit for Datasette since nested virtualisation is not exposed on AWS EC2.)
rattray 10mo ago
Wait, why not just actually use the Cloudflare Sandboxes product then? Is it too costly or something? Or you need to be able to run without a connection to their cloud? https://developers.cloudflare.com/sandbox/ https://developers.cloudflare.com/sandbox/
- simonw 10mo ago
  I'm building software I want other people to be able to run themselves, I don't want to have to tell them to create a Cloudflare account as part of running that software.

by simonw 10mo ago

kentonv 10mo ago

> the whole point of WebAssembly is to solve this one problem. For Wasm to be a secure sandbox, you have to assume a bug-free compiler/interpreter, which, alas, none of them really are. It's a somewhat easier problem than building a bug-free JavaScript runtime, but not by as much as you might expect, sadly. > I want to build extension systems for my own apps such that users can run their own code or paste in code written by other people and have it execute safely. Similar to Shopify Functions: https://shopify.dev/docs/apps/build/functions https://shopify.dev/docs/apps/build/functions Ah, this is exactly the Workers for Platforms use case: https://developers.cloudflare.com/cloudflare-for-platforms/workers-for-platforms/ https://developers.cloudflare.com/cloudflare-for-platforms/w... And indeed, Shopify uses it: https://shopify.engineering/how-we-built-oxygen https://shopify.engineering/how-we-built-oxygen (There's also the upcoming Dynamic Worker Loader API: https://developers.cloudflare.com/workers/runtime-apis/bindings/worker-loader/ https://developers.cloudflare.com/workers/runtime-apis/bindi...) But it sounds like you really do want to self-host? I don't blame you, but that does make it tough. I'm not sure there's any such thing as a secure sandbox that doesn't require some level of monitoring and daily maintenance, sadly. (But admittedly I may be biased.)

simonw 10mo ago
Yeah my ideal is to have something that cleanly "pip installs" as a dependency such that users of my open source Python projects can self host tools that let them extend using arbitrary code, including code written by LLMs. I've been picking at this problem for a few years now! On the one hand I get why it's so hard. But it really feels like it should be possible to solve this in 2026 - executing arbitrary code in a way that constrains its memory and CPU time usage is a problem our industry solves in browsers and hosting platforms and databases and all sorts of other places, and has done for decades. The whole LLM-assisted end-user programming thing makes solving this with the right developer affordances so valuable!
- rattray 10mo ago
  Ah, in that context, why not just give the people workerd? People using & running OSS libraries are used to the fact that there might be vulns in libraries they're using, right?
  - kentonv 10mo ago
    Or put another way: If Simon's users choose to self-host the open source version of his service, they are probably using it to run their own code, and so the sandbox security matters less, and workerd may be fine. The sandbox only matters when Simon himself offers his software as a service, which he could do using Workers for Platforms. (But this is a self-serving argument coming from me.)
- hiccuphippo 10mo ago
  If the code is gonna be written by LLMs anyway skip the js, directly use wasm and allow any language that compiles to it with something like https://extism.org/ https://extism.org/

neildhar 10mo ago

I worked on a sandbox of Hermes that compiles the engine to wasm, and then converts the wasm to C (like Mozilla's RLBox). It exposes the same C++ API as Hermes so it is pretty convenient to drop in, and should be fairly secure. It hasn't been updated in some time, but it should still be working, and can probably be brought up to date with some small effort: https://github.com/facebook/hermes/tree/static_h/API/hermes_sandbox https://github.com/facebook/hermes/tree/static_h/API/hermes_... EDIT: Reading some of your other comments, I should point out that this is more like a component of a possible solution. It does not attempt to prevent resource exhaustion or crashes due to corrupted internal state.

aardvark179 10mo ago

You need more than a simple sandbox for what you describe. You also need to avoid infinite loops or other constructs from causing a DoS or similar, and if you are doing this with the intention of interacting with some other parts of a system then you need to think about how that interaction works and whether there is a way to execute something outside of that sandbox. Even if you go with something backed by a full time team there is still going to be a chance you have to deal with a security issue in a hurry, maybe in the run up to Christmas. That is just going to come with the territory and if you don’t want to deal with that then you probably need to think about whether you really need a sandbox that can execute untrusted code.

laurencerowe 10mo ago

There's also JCO for creating JS based WebAssembly components. It's based on StarlingMonkey which I believe is slightly faster than QuickJS under web assembly. https://github.com/bytecodealliance/jco https://github.com/bytecodealliance/jco Benchmark numbers for request isolated JS hello world / React page rendering: JCO/wasmtime: 314µs / 13ms Bun process forking: 1.7ms / 8.2ms v8 isolate from snapshot: 0.7ms / 22ms TinyKVM: 52µs / 708µs Native with reuse 14µs / 640µs Numbers taken from our upcoming TinyKVM paper. Benchmark setup code for JCO/wasmtime is here: https://github.com/libriscv/kvmserver/tree/main/examples/wasmtime https://github.com/libriscv/kvmserver/tree/main/examples/was... (I suspect even if we are able to get TinyKVM into a state you'd feel comfortable with in the future it would still be an awkward fit for Datasette since nested virtualisation is not exposed on AWS EC2.)

rattray 10mo ago

Wait, why not just actually use the Cloudflare Sandboxes product then? Is it too costly or something? Or you need to be able to run without a connection to their cloud? https://developers.cloudflare.com/sandbox/ https://developers.cloudflare.com/sandbox/

simonw 10mo ago
I'm building software I want other people to be able to run themselves, I don't want to have to tell them to create a Cloudflare account as part of running that software.