3 ms·

Neither side is "right" or "wrong", because this is not actually a binary thing. The two sides are judging the level of risk involved and whether or not that le

Neither side is "right" or "wrong", because this is not actually a binary thing. The two sides are judging the level of risk involved and whether or not that level of risk is acceptable, and they are coming to different conclusions. V8 is undeniably intended to be a secure sandbox. Chrome relied on V8 as the only thing isolating web sites from each other for about a decade. A couple years ago, Chrome implemented "strict site isolation", which adds defense in depth by also forcing every "site" into a separate OS process. However, this does not mean that the Chrome team no longer cares about V8 itself being secure. Strict site isolation is a defense-in-depth measure. Google will pay a bug bounty if you break either layer. However, V8, like any piece of software, has bugs. The possibility of security bugs implies some level of risk. The question is, how much risk is there, and it is acceptable? V8 is complex, and as a result it tends to have more bugs than, say, a virtual machine hypervisor. Hence, it is argued that V8 is more risky. Some believe the level of risk is unacceptable for a server environment. On the other hand, V8 receives more security research and better fuzzing than any other sandboxing technology. Most bugs in V8 are actually found by Google's own fuzzers. And process isolation is only one kind of defense-in-depth possible here; Cloudflare Workers implements many other types of defense-in-depth that align better with our particular requirements.[0] My take -- as the tech lead of Cloudflare Workers -- is that people are broadly overestimating the risk because the approach is different. In fact, I personally believe that typical cloud environments which allow you to run arbitrary native-code binaries are much riskier. The reason is, sandboxing native code doesn't just require secure virtual machine software, it also requires bug-free hardware. CPUs are extremely complex, certainly much more complex than V8. Do we really believe they don't have any bugs which allow VM breakouts? What happens if someone finds such a bug, and it's not possible to mitigate without new silicon? Say it turns out some obscure instruction accidentally allows unchecked access to all physical memory -- what then? It would be quite a disaster for the industry. In contrast, if your platform only accepts non-native code formats like JavaScript and WebAssembly, it's much easier to respond to such bugs by e.g. controlling access to the buggy instruction. So frankly, my take is the entire industry has accepted a much higher level of risk already, but people don't talk about it much because it's already broadly accepted as the status quo. Cloudflare Workers gets more questions because we've made a different judgment. With that said, it's absolutely possible for smart people to disagree on all these points, and probably neither side will ever be proven definitively right or wrong. [0] https://blog.cloudflare.com/mitigating-spectre-and-other-security-threats-the-cloudflare-workers-security-model/ https://blog.cloudflare.com/mitigating-spectre-and-other-sec...

pciexpgpu 4y ago
Thoughtful and insightful answer. Although I am on the other side of your argument. I like your rationale around other cloud providers’ virtualization solutions. I think the main comparison that’s lacking is one of v8 isolates vs strict site isolation which runs on separate processes. Why would the Chrome team build such a complex thing if v8 isolates passed rigorous security standards/bar etc? This would be good to get your insights on. What’s the actual cost of separate processes vs v8 isolate? In terms of cpu/memory/latency mainly. I would guess either would win over other virtualization techniques but between the two is it really that stark of a difference? I am more and more convinced you may have seen the light shine through and people such as me are missing the big picture: so providing answers for this line of attack would be more helpful than comparing with other virtualization mechanisms.
- kentonv 4y ago
  > Why would the Chrome team build such a complex thing if v8 isolates passed rigorous security standards/bar etc? This question assumes that there exists some threshold of "rigorous security standards" which clearly separates "secure" from "insecure". There's no such threshold. We know that risks always exist. No one can precisely quantify those risks. We can only sort of abstractly debate about which risks might be large, and take precautions by addressing those risks with defense-in-depth measures. Each such measure must, of course, be weighed against the costs -- both development costs and runtime costs. Chrome made the decision that strict site isolation was a worthwhile trade-off to reduce risks. I generally agree with this decision. In Chrome's case, the overhead of extra processes turned out to be acceptable, and I think the risk reduction was meaningful. In Cloudflare Workers' case, the cost of isolating every Worker in its own process would be much higher. We actually have the ability to do such isolation, and we selectively apply it in cases where we have other signals to suggest that risk may be elevated (such as when the Worker's performance characteristics suggest it may be engaging in a Spectre attack). But because Workers are designed to handle fine-grained events and often run for less than a millisecond at a time, the overhead of this isolation is much higher than in Chrome's case. The exact overhead depends on what the Worker is doing, but 5x-10x is typical (both for CPU and memory). Given this, implementing strict process isolation would not be a good trade-off for us. Some people respond to this with "You can't sacrifice security for performance!!!", but this is naive. Again, all of the big cloud providers are trading off a whole lot of security risk for performance when they choose to run untrusted native code. In the real world we always make trade-offs. Meanwhile, though, there are lots of other types of defense in depth that we can implement in Workers that isn't so easy to do in browsers. Dynamic process isolation is one example. Another is that we can automatically roll out V8 patches to production much faster than Chrome can. See the blog post I linked for some more examples.

pciexpgpu 4y ago

Thoughtful and insightful answer. Although I am on the other side of your argument. I like your rationale around other cloud providers’ virtualization solutions. I think the main comparison that’s lacking is one of v8 isolates vs strict site isolation which runs on separate processes. Why would the Chrome team build such a complex thing if v8 isolates passed rigorous security standards/bar etc? This would be good to get your insights on. What’s the actual cost of separate processes vs v8 isolate? In terms of cpu/memory/latency mainly. I would guess either would win over other virtualization techniques but between the two is it really that stark of a difference? I am more and more convinced you may have seen the light shine through and people such as me are missing the big picture: so providing answers for this line of attack would be more helpful than comparing with other virtualization mechanisms.

kentonv 4y ago
> Why would the Chrome team build such a complex thing if v8 isolates passed rigorous security standards/bar etc? This question assumes that there exists some threshold of "rigorous security standards" which clearly separates "secure" from "insecure". There's no such threshold. We know that risks always exist. No one can precisely quantify those risks. We can only sort of abstractly debate about which risks might be large, and take precautions by addressing those risks with defense-in-depth measures. Each such measure must, of course, be weighed against the costs -- both development costs and runtime costs. Chrome made the decision that strict site isolation was a worthwhile trade-off to reduce risks. I generally agree with this decision. In Chrome's case, the overhead of extra processes turned out to be acceptable, and I think the risk reduction was meaningful. In Cloudflare Workers' case, the cost of isolating every Worker in its own process would be much higher. We actually have the ability to do such isolation, and we selectively apply it in cases where we have other signals to suggest that risk may be elevated (such as when the Worker's performance characteristics suggest it may be engaging in a Spectre attack). But because Workers are designed to handle fine-grained events and often run for less than a millisecond at a time, the overhead of this isolation is much higher than in Chrome's case. The exact overhead depends on what the Worker is doing, but 5x-10x is typical (both for CPU and memory). Given this, implementing strict process isolation would not be a good trade-off for us. Some people respond to this with "You can't sacrifice security for performance!!!", but this is naive. Again, all of the big cloud providers are trading off a whole lot of security risk for performance when they choose to run untrusted native code. In the real world we always make trade-offs. Meanwhile, though, there are lots of other types of defense in depth that we can implement in Workers that isn't so easy to do in browsers. Dynamic process isolation is one example. Another is that we can automatically roll out V8 patches to production much faster than Chrome can. See the blog post I linked for some more examples.