4 ms·

All these JavaScript engines and it's still remarkably hard to find a robust solution for executing JavaScript from untrusted sources inside my own server-side

by simonw 10mo ago

All these JavaScript engines and it's still remarkably hard to find a robust solution for executing JavaScript from untrusted sources inside my own server-side applications. Every time I look I find repos that look promising at first but are either unmaintained or have a team or just one or two maintainers running them as a side project. I want my sandbox to be backed by a large, well funded security team working for a product with real money on the line if there are any holes! (Playing with Cloudflare workerd this morning, which seems like it should cover my organizational requirements at least.) Update: Classic, even Cloudflare workerd has "WARNING: workerd is not a hardened sandbox" in the README! https://github.com/cloudflare/workerd?tab=readme-ov-file#warning-workerd-is-not-a-hardened-sandbox https://github.com/cloudflare/workerd?tab=readme-ov-file#war...

pastage 10mo ago
One process per sandbox will get you far, if all you want is to execute something. I would go as far as say it is pretty easy.
- simonw 10mo ago
  I want to execute untrusted code. This makes it very difficult indeed.
  - mike_hearn 10mo ago
    What's wrong with V8? You could also look at GraalJS. It's shipped as part of the Oracle Database, there's a security team, patching process etc. It's used in production by Amazon amongst others. It's got flexible sandbox features too. https://www.graalvm.org/latest/reference-manual/embed-languages/#access-privilege-configuration https://www.graalvm.org/latest/reference-manual/embed-langua... The way it's written is good for security as well: https://medium.com/graalvm/writing-truly-memory-safe-jit-compilers-f79ad44558dd https://medium.com/graalvm/writing-truly-memory-safe-jit-com... Disclosure: I sit next to the GraalVM team.
    - simonw 10mo ago
      The challenge with V8 is finding a wrapper for it that doesn't come with a big warning NOT to use it as a sandbox for untrusted code - here's the workerd one https://github.com/cloudflare/workerd?tab=readme-ov-file#warning-workerd-is-not-a-hardened-sandbox https://github.com/cloudflare/workerd?tab=readme-ov-file#war... and here's the PyMiniRacer section: https://bpcreech.com/PyMiniRacer/architecture/#security-goals https://bpcreech.com/PyMiniRacer/architecture/#security-goal... I looked at GraalVM but was put off by the licensing situation: https://www.graalvm.org/22.3/reference-manual/embed-languages/sandbox-resource-limits/ https://www.graalvm.org/22.3/reference-manual/embed-language... > GraalVM Enterprise provides the experimental Sandbox Resource Limits feature that allows for the limiting of resources used by guest applications. These resource limits are not available in the Community Edition of GraalVM. Part of my requirements for a sandbox are strong guarantees against memory or CPU exhaustion from poorly written or malicious code.
      - mike_hearn 10mo ago
        Licensing has changed since that release. You can use the EE for free, both for personal and commercial use cases: https://www.graalvm.org/latest/introduction/#licensing-and-support https://www.graalvm.org/latest/introduction/#licensing-and-s... > Oracle GraalVM is licensed under GraalVM Free Terms and Conditions (GFTC) including License for Early Adopter Versions. Subject to the conditions in the license, including the License for Early Adopter Versions, the GFTC is intended to permit use by any user including commercial and production use. It has all the sandboxing features you might want. I don't know if the disclaimers on the other engines changes much, open source software always disclaims all liability. Nobody will stand behind something security sensitive unless it's commercial because otherwise there's no way to pay for the security team it requires.
HatchedLake721 10mo ago
Not a silver bullet, but did you look into isolated-vm? https://github.com/laverdet/isolated-vm https://github.com/laverdet/isolated-vm But generally, I think best bet is to offload such things to e.g. Lambda per tenant.
- simonw 10mo ago
  The README says it's in maintenance mode and the single author doesn't have time to dedicate to the project.
15155 10mo ago
https://github.com/bellard/mquickjs https://github.com/bellard/mquickjs Featured recently on HN.
- simonw 10mo ago
  Yeah that (and full QuickJS) running inside WebAssembly do appear to be my best options. Here's my experiment running that one in WASM: https://github.com/simonw/research/tree/main/mquickjs-sandbox https://github.com/simonw/research/tree/main/mquickjs-sandbo...
  - 15155 10mo ago
    mquickjs shouldn't be escapable, in theory, given the memory model. I would do everything possible to avoid WASM, otherwise I'd just run V8.
    - simonw 10mo ago
      Why avoid WASM?
      - 15155 10mo ago
        Because what's the point? You're running JS (an 'interpreted', managed language) - it's already intentionally designed to be executed in a sandbox. Unless you provide hooks out to the host system, it can't do anything bad. With mquickjs, the untrusted code can't even overflow your heap or take too much execution time. If you were running untrusted C or something, it would make more sense to add the WASM layer.
        simonw 10mo ago
        Mainly defence in depth. I have enormous respect for Fabrice but mquickjs is only a few weeks old and I'm no way near skilled enough to audit his C code! Running it in WASM feels a lot safer to me.
ivankra 10mo ago
Put them in a container or VM? Security benefits from layering: engine/runtime is one layer, container/VM is another - an attacker would need two independent high-value exploits to breach both of them. High budget is no guarantee for absence of critical bugs in an engine, maybe even somewhat opposite - on a big team the incentives are aligned with shipping more features (since nobody gets promoted for maintenance, especially at Google) -> increasing complexity -> increasing bug surface. If speed is less important and you can live without JIT, that expands your options dramatically and eliminates a large class of bugs. You could take a lightweight engine and compile it to a memory-safe runtime, that'd give you yet another security layer for peace of mind. Several projects did such ports to Wasm/JS/Go - for example your browser likely runs QuickJS to interpret JavaScript inside .pdf (https://github.com/mozilla/pdf.js.quickjs https://github.com/mozilla/pdf.js.quickjs)
kentonv 10mo ago
To add some color to "workerd is not a hardened sandbox": workerd does not include any sandboxing layers other than V8 itself. If someone has a V8 zero-day exploit, they can break out of the sandbox. But putting aside zero-day exploits for a moment, workerd is designed to be a sandbox. That is, applications by default have access to nothing except what you give them. There is only one default-on type of access: public internet access (covering public IPs only). You can disable this by overriding `globalOutbound` in the config (with which you can either intercept internet requests, or just block them). This is pretty different from e.g. Node, which starts from the assumption that apps should have permission to run arbitrary native code, limited only by the permissions of the user account under which Node is running. Some other runtimes advertise various forms of permissions, but workerd is the only one I know of where this is the core intended use case, and where all permissions (other than optionally public internet access, as mentioned) must be granted via capability-based security. Unfortunately, JavaScript engines are complicated, which means they tend to have bugs, and these bugs are often exploitable to escape the sandbox. This is not just true of V8, it's true of all of them; any that claims otherwise is naive. Cloudflare in production has a multi-layer security model to mitigate this, but our model involves a lot of, shall we say, active management which can't easily be packaged up into an open source product. With all that said, not all threat models require you to worry about such zero-day exploits, and you need to think about risk/benefit tradeoffs. We obviously have to worry about zero-days at Cloudflare since anyone can just upload code to us and run it. But if you're not literally accepting code directly from anonymous internet users then the risk may be a lot lower, and the overall security benefit of fine-grained sandboxing may be worth the increased exposure to zero-days.
- simonw 10mo ago
  Right - I trust workerd in the context of Cloudflare because I know it has a team of people who's job it is to keep it secure who are on-call 24/7. The problem I have is that I'm just one person and I don't want to be on call 24/7 ready to react to sandbox escapes, so I'm hoping I can find a solution that someone else built where they are willing to say "this is safe: you can feed in a string of untrusted JavaScript and we are confident it won't break out again". I think I might be able to get there via WebAssembly (e.g. with QuickJS or MicroQuickJS compiled to WASM) because the whole point of WebAssembly is to solve this one problem. > But if you're not literally accepting code directly from anonymous internet users then the risk may be a lot lower That's the problem: this is exactly what I want to be able to do! I want to build extension systems for my own apps such that users can run their own code or paste in code written by other people and have it execute safely. Similar to Shopify Functions: https://shopify.dev/docs/apps/build/functions https://shopify.dev/docs/apps/build/functions I think the value unlocked by this kind of extension mechanism is ready to skyrocket, because users can use LLMs to help write that code for them.
smarx007 10mo ago
> I want my sandbox to be backed by a large, well funded security team How much are you ready to pay for a license?
- simonw 10mo ago
  Nothing, because I want this for open source projects that I give away for free. That doesn't work if I have to tell my users to go pay a license fee for one of the components.

by simonw 10mo ago

pastage 10mo ago

One process per sandbox will get you far, if all you want is to execute something. I would go as far as say it is pretty easy.

simonw 10mo ago
I want to execute untrusted code. This makes it very difficult indeed.
- mike_hearn 10mo ago
  What's wrong with V8? You could also look at GraalJS. It's shipped as part of the Oracle Database, there's a security team, patching process etc. It's used in production by Amazon amongst others. It's got flexible sandbox features too. https://www.graalvm.org/latest/reference-manual/embed-languages/#access-privilege-configuration https://www.graalvm.org/latest/reference-manual/embed-langua... The way it's written is good for security as well: https://medium.com/graalvm/writing-truly-memory-safe-jit-compilers-f79ad44558dd https://medium.com/graalvm/writing-truly-memory-safe-jit-com... Disclosure: I sit next to the GraalVM team.
  - simonw 10mo ago
    The challenge with V8 is finding a wrapper for it that doesn't come with a big warning NOT to use it as a sandbox for untrusted code - here's the workerd one https://github.com/cloudflare/workerd?tab=readme-ov-file#warning-workerd-is-not-a-hardened-sandbox https://github.com/cloudflare/workerd?tab=readme-ov-file#war... and here's the PyMiniRacer section: https://bpcreech.com/PyMiniRacer/architecture/#security-goals https://bpcreech.com/PyMiniRacer/architecture/#security-goal... I looked at GraalVM but was put off by the licensing situation: https://www.graalvm.org/22.3/reference-manual/embed-languages/sandbox-resource-limits/ https://www.graalvm.org/22.3/reference-manual/embed-language... > GraalVM Enterprise provides the experimental Sandbox Resource Limits feature that allows for the limiting of resources used by guest applications. These resource limits are not available in the Community Edition of GraalVM. Part of my requirements for a sandbox are strong guarantees against memory or CPU exhaustion from poorly written or malicious code.
    - mike_hearn 10mo ago
      Licensing has changed since that release. You can use the EE for free, both for personal and commercial use cases: https://www.graalvm.org/latest/introduction/#licensing-and-support https://www.graalvm.org/latest/introduction/#licensing-and-s... > Oracle GraalVM is licensed under GraalVM Free Terms and Conditions (GFTC) including License for Early Adopter Versions. Subject to the conditions in the license, including the License for Early Adopter Versions, the GFTC is intended to permit use by any user including commercial and production use. It has all the sandboxing features you might want. I don't know if the disclaimers on the other engines changes much, open source software always disclaims all liability. Nobody will stand behind something security sensitive unless it's commercial because otherwise there's no way to pay for the security team it requires.

HatchedLake721 10mo ago

Not a silver bullet, but did you look into isolated-vm? https://github.com/laverdet/isolated-vm https://github.com/laverdet/isolated-vm But generally, I think best bet is to offload such things to e.g. Lambda per tenant.

simonw 10mo ago
The README says it's in maintenance mode and the single author doesn't have time to dedicate to the project.

15155 10mo ago

https://github.com/bellard/mquickjs https://github.com/bellard/mquickjs Featured recently on HN.

simonw 10mo ago
Yeah that (and full QuickJS) running inside WebAssembly do appear to be my best options. Here's my experiment running that one in WASM: https://github.com/simonw/research/tree/main/mquickjs-sandbox https://github.com/simonw/research/tree/main/mquickjs-sandbo...
- 15155 10mo ago
  mquickjs shouldn't be escapable, in theory, given the memory model. I would do everything possible to avoid WASM, otherwise I'd just run V8.
  - simonw 10mo ago
    Why avoid WASM?
    - 15155 10mo ago
      Because what's the point? You're running JS (an 'interpreted', managed language) - it's already intentionally designed to be executed in a sandbox. Unless you provide hooks out to the host system, it can't do anything bad. With mquickjs, the untrusted code can't even overflow your heap or take too much execution time. If you were running untrusted C or something, it would make more sense to add the WASM layer.
      - simonw 10mo ago
        Mainly defence in depth. I have enormous respect for Fabrice but mquickjs is only a few weeks old and I'm no way near skilled enough to audit his C code! Running it in WASM feels a lot safer to me.

ivankra 10mo ago

Put them in a container or VM? Security benefits from layering: engine/runtime is one layer, container/VM is another - an attacker would need two independent high-value exploits to breach both of them. High budget is no guarantee for absence of critical bugs in an engine, maybe even somewhat opposite - on a big team the incentives are aligned with shipping more features (since nobody gets promoted for maintenance, especially at Google) -> increasing complexity -> increasing bug surface. If speed is less important and you can live without JIT, that expands your options dramatically and eliminates a large class of bugs. You could take a lightweight engine and compile it to a memory-safe runtime, that'd give you yet another security layer for peace of mind. Several projects did such ports to Wasm/JS/Go - for example your browser likely runs QuickJS to interpret JavaScript inside .pdf (https://github.com/mozilla/pdf.js.quickjs https://github.com/mozilla/pdf.js.quickjs)

kentonv 10mo ago

To add some color to "workerd is not a hardened sandbox": workerd does not include any sandboxing layers other than V8 itself. If someone has a V8 zero-day exploit, they can break out of the sandbox. But putting aside zero-day exploits for a moment, workerd is designed to be a sandbox. That is, applications by default have access to nothing except what you give them. There is only one default-on type of access: public internet access (covering public IPs only). You can disable this by overriding `globalOutbound` in the config (with which you can either intercept internet requests, or just block them). This is pretty different from e.g. Node, which starts from the assumption that apps should have permission to run arbitrary native code, limited only by the permissions of the user account under which Node is running. Some other runtimes advertise various forms of permissions, but workerd is the only one I know of where this is the core intended use case, and where all permissions (other than optionally public internet access, as mentioned) must be granted via capability-based security. Unfortunately, JavaScript engines are complicated, which means they tend to have bugs, and these bugs are often exploitable to escape the sandbox. This is not just true of V8, it's true of all of them; any that claims otherwise is naive. Cloudflare in production has a multi-layer security model to mitigate this, but our model involves a lot of, shall we say, active management which can't easily be packaged up into an open source product. With all that said, not all threat models require you to worry about such zero-day exploits, and you need to think about risk/benefit tradeoffs. We obviously have to worry about zero-days at Cloudflare since anyone can just upload code to us and run it. But if you're not literally accepting code directly from anonymous internet users then the risk may be a lot lower, and the overall security benefit of fine-grained sandboxing may be worth the increased exposure to zero-days.

simonw 10mo ago
Right - I trust workerd in the context of Cloudflare because I know it has a team of people who's job it is to keep it secure who are on-call 24/7. The problem I have is that I'm just one person and I don't want to be on call 24/7 ready to react to sandbox escapes, so I'm hoping I can find a solution that someone else built where they are willing to say "this is safe: you can feed in a string of untrusted JavaScript and we are confident it won't break out again". I think I might be able to get there via WebAssembly (e.g. with QuickJS or MicroQuickJS compiled to WASM) because the whole point of WebAssembly is to solve this one problem. > But if you're not literally accepting code directly from anonymous internet users then the risk may be a lot lower That's the problem: this is exactly what I want to be able to do! I want to build extension systems for my own apps such that users can run their own code or paste in code written by other people and have it execute safely. Similar to Shopify Functions: https://shopify.dev/docs/apps/build/functions https://shopify.dev/docs/apps/build/functions I think the value unlocked by this kind of extension mechanism is ready to skyrocket, because users can use LLMs to help write that code for them.

smarx007 10mo ago

> I want my sandbox to be backed by a large, well funded security team How much are you ready to pay for a license?

simonw 10mo ago
Nothing, because I want this for open source projects that I give away for free. That doesn't work if I have to tell my users to go pay a license fee for one of the components.