9 ms·

Spectre Mitigations Part 2

71 points by cejetvole 8y ago

twtw 8y ago
I've asked this before, but I'm going to ask it again (sorry for the repetition). What makes this sandbox different from previous sandboxes (JVM, browsers, etc) that makes proponents sufficiently confident to put it in the kernel? All previous sandboxed designed to be secure have been broken on a regular basis. The rationale seems to be performance, as if it is an innovation to realize that having multiple isolated processes and privilege levels has a cost, and things could be faster without context switches or virtual memory. But everyone knows this already - the real innovation is daring to trust your sandbox so much that you think you can do without process isolation. Sorry to keep asking, but it strikes me as irresponsible to bash on context switching as having poor performance and not noting that it is really one of the foundations of computer security. First time I asked: https://news.ycombinator.com/item?id=18406623 https://news.ycombinator.com/item?id=18406623
- monocasa 8y ago
  I'm working towards a similar goal as WASMJIT on one of my side projects, and I fundamentally agree with what you're saying. I think that starting with a large env like WASM and trying to plug the gaps is a sisyphean task. That's why I'm starting with BPF which in a lot of ways was designed for super simple verification and code generation, and slowly letting more code graphs in rather than trying to start with arbitrary code graphs and giant single linear regions and plugging holes.
  - naasking 8y ago
    > I think that starting with a large env like WASM and trying to plug the gaps is a sisyphean task What makes WASM a large env? It was specifically designed to be more tractable to analyse than machine code, so it's already more restricted.
    - monocasa 8y ago
      WASM's not huge, but it's giant compared to eBPF. * eBPF explicitly doesn't do register allocation, which simplifies code generation. * Similarly, eBPF is a 2 address, but load/store architecture, so for the vast majority of cases for both CISC and RISC architectures, there's a one to one correlation between eBPF instructions and native processor instructions, also greatly simplifying code generation. * The only actually used eBPF implementation heavily reduces the options of the code flow graph to being a not quite Turing complete subset. The code flow graph can only be a DAG, so you can rely on being able to generate programs that you can easily, statically verify will terminate. * The memory usage is a lot more explicit. The stack can be checked for worst case usage. Heap (called "maps") memory is typed. So types can be easily checked from registers, through the stack, and through to the heap. And that includes when heaps are shared with user space or other eBPF programs running in parallel. So eBPF sort of starts from a place of remarkable verification, with the idea of loosening the constraints as new ways of guaranteeing code is safe are found. WASM tooling isn't as amenable to working under such circumstances. It also simplifies the JIT to an almost trivial translation, greatly simplifying verfication of correctness. That is a huge help in the constrained environment of kernel space.
      - vlovich123 8y ago
        My friend who's a security engineer has been bashing on e/BPF that there's actually some serious vulnerabilities that have been discovered/continue to be discovered. What's your take on that criticism?
        monocasa 8y ago
        It's hard to address such a vague statement, can you expand? FWIW, part of what I've been working on is formally verifying an eBPF runtime, to cut down the number of standard code bugs. If he's talking about Spectre stuff, there's going to have to be hardware mitigations as you can hit the same architecture flaws even without code generation. It was just easier for Project Zero to use BPF rather than track down the code sequences they were looking for in the existing kernel.
- smaddox 8y ago
  Well, considering the hardware sandboxing is also imperfect (Spectre, meltdown, rowhammer), you don't really have a choice but to use at least some software sandboxing. If you can use only software sandboxing, then there are potentially big wins. Seems like worth trying.
  - twtw 8y ago
    It is definitely worth trying as an experiment, but the discussion around wasmjit doesn't seem like it's a research project. Its worth noting that this has in many senses already been extensively research. This was a significant part of the design of Singularity OS, a micro kernel from Microsoft Research with software isolation between components (with much more formal sandboxing than wasmjit, verified with static analysis). Also there have been several operating systems with a JVM in the kernel over the last 2 decades with many of the same benefits.
- saagarjha 8y ago
  Your point seems valid to me as well; the responses seem to be making the argument that a singular sandbox is good enough, without keeping mind the fact that most software runs inside of multiple sandboxes as a sort of “defense in depth” to provide protection even if one of the layers fails.
- ec109685 8y ago
  What attack scenarios you are envisioning here? If the host is only running this software (e.g. a single purpose server, firewall, etc.), the security benefits of process isolation are reduced. Clearly if you are trying to run hostile software from multiple vendors on the same box (e.g. a browser), you want more sandboxes.
  - the8472 8y ago
    If you're running a single purpose server do you still need a sandbox though? Wouldn't a unikernel do the job and avoid the JIT compilation overhead?
    - cejetvole 8y ago
      It might make more sense for you to run a unikernel but it may also be important to you to use your existing tooling / monitoring infrastructure to administer your server. For example, being able to ssh into a server when things go wrong can make things a lot easier.
  - twtw 8y ago
    I see a future where the npm-style ecosystem develops with the JIT running inside the kernel, so you end up running a "trusted" application ends up running a whole lot of untrusted code via dependencies. Usually process isolation gives you some limit on how much damage something like that can do. Also, if you are 100% sure that the code in it is trusted, then there really is no reason to sandbox it, right? If the intent is to only run trusted code, why is this article about spectre mitigations?
    - kiriakasis 8y ago
      > if you are 100% sure that the code in it is trusted, then there really is no reason to sandbox it, right? If the intent is to only run trusted code, Trusted code can have bugs; sandboxing, in a sense, is always useful (not always beneficial)
      - deleted 8y ago
- cejetvole 8y ago
  There isn't anything inherently different about a WebAssembly-based software sandbox from previous software sandboxes like the ones you mentioned. It maintains a reasonable level of protection for additional benefits. If those benefits aren't worth it for your use case then you don't need to take the additional potential risk. We don't live in a computing monoculture, what's risky for you may not be risky for me, and what's valuable to you may not be valuable to me. It's worth noting that the code that will be run under Wasmjit will in general be more trusted than code you run in a browser while browsing the Web. For those who only run trusted code on their servers, Wasmjit incurs no additional risk and provides potentially significant benefit.
  - kiriakasis 8y ago
    https://news.ycombinator.com/item?id=18527535 https://news.ycombinator.com/item?id=18527535 Another story pointed out that wasm today still lacks a few features and so, mostly regarding run-time inspection, has drawbacks server-side. In the article there is no mention that wasm tomorrow could be quite different from wasm today; overall I believe the optimism is warranted.
- naasking 8y ago
  > What makes this sandbox different from previous sandboxes (JVM, browsers, etc) that makes proponents sufficiently confident to put it in the kernel Formalization was part of the process of developing WASM. That's already a large difference, and it led to a simpler instruction set that's more amenable to analysis.
  - twtw 8y ago
    Do you have any pointers to resources regarding the formal model for web assembly?
    - naasking 8y ago
      https://news.ycombinator.com/item?id=17575111 https://news.ycombinator.com/item?id=17575111
- saurik 8y ago
  This VM is super simple: it doesn't have garbage collection and objects and class-level dynamic linking and a type system... it just has integers and floating point numbers in a static memory array. It gives you a stack: that's about it; it is more similar to BPF or something, which notably is in the kernel. (I still agree with you.)
titzer 8y ago
Do not run untrusted code in the kernel. Period. Double period. Triple period. Speculative vulnerabilities and side channels are a lot more than just variant 1 and 2. WasmJIT is a nice project. But let me repeat. Do not put it in the kernel and run untrusted code through it, period. Software mitigations are not sufficient. This is not WASM's fault, this is not WasmJIT's fault; it's just a fundamental reality of modern hardware.
- reitzensteinm 8y ago
  What if the sandbox completely denies any method of timing to the untrusted code? Including thread to thread communication. Timing related side channels seem unavoidable, but that doesn't mean that answer in -> answer out computation is necessarily intrinsically able to perform it. Also, if you can trust code to not be malicious, but not to be correct, loading code via WASM and executing it in the same address space isn't necessarily crazy.
  - titzer 8y ago
    > What if the sandbox completely denies any method of timing to the untrusted code? Including thread to thread communication. You can construct timers from shared mutable memory (think: counter thread), but even in shared-nothing systems, one can construct timers using message passing (think: a crude timer that counts messages in one process and a sender hammering it with messages). > Also, if you can trust code to not be malicious, but not to be correct, loading code via WASM and executing it in the same address space isn't necessarily crazy. Sure, agreed. This is why my comment mentioned untrusted code. WebAssembly sandboxing makes sense to protect the kernel from OOB writes, but it cannot guarantee no OOB reads from speculative side-channels.

71 points by cejetvole 8y ago

twtw 8y ago

I've asked this before, but I'm going to ask it again (sorry for the repetition). What makes this sandbox different from previous sandboxes (JVM, browsers, etc) that makes proponents sufficiently confident to put it in the kernel? All previous sandboxed designed to be secure have been broken on a regular basis. The rationale seems to be performance, as if it is an innovation to realize that having multiple isolated processes and privilege levels has a cost, and things could be faster without context switches or virtual memory. But everyone knows this already - the real innovation is daring to trust your sandbox so much that you think you can do without process isolation. Sorry to keep asking, but it strikes me as irresponsible to bash on context switching as having poor performance and not noting that it is really one of the foundations of computer security. First time I asked: https://news.ycombinator.com/item?id=18406623 https://news.ycombinator.com/item?id=18406623

monocasa 8y ago
I'm working towards a similar goal as WASMJIT on one of my side projects, and I fundamentally agree with what you're saying. I think that starting with a large env like WASM and trying to plug the gaps is a sisyphean task. That's why I'm starting with BPF which in a lot of ways was designed for super simple verification and code generation, and slowly letting more code graphs in rather than trying to start with arbitrary code graphs and giant single linear regions and plugging holes.
- naasking 8y ago
  > I think that starting with a large env like WASM and trying to plug the gaps is a sisyphean task What makes WASM a large env? It was specifically designed to be more tractable to analyse than machine code, so it's already more restricted.
  - monocasa 8y ago
    WASM's not huge, but it's giant compared to eBPF. * eBPF explicitly doesn't do register allocation, which simplifies code generation. * Similarly, eBPF is a 2 address, but load/store architecture, so for the vast majority of cases for both CISC and RISC architectures, there's a one to one correlation between eBPF instructions and native processor instructions, also greatly simplifying code generation. * The only actually used eBPF implementation heavily reduces the options of the code flow graph to being a not quite Turing complete subset. The code flow graph can only be a DAG, so you can rely on being able to generate programs that you can easily, statically verify will terminate. * The memory usage is a lot more explicit. The stack can be checked for worst case usage. Heap (called "maps") memory is typed. So types can be easily checked from registers, through the stack, and through to the heap. And that includes when heaps are shared with user space or other eBPF programs running in parallel. So eBPF sort of starts from a place of remarkable verification, with the idea of loosening the constraints as new ways of guaranteeing code is safe are found. WASM tooling isn't as amenable to working under such circumstances. It also simplifies the JIT to an almost trivial translation, greatly simplifying verfication of correctness. That is a huge help in the constrained environment of kernel space.
    - vlovich123 8y ago
      My friend who's a security engineer has been bashing on e/BPF that there's actually some serious vulnerabilities that have been discovered/continue to be discovered. What's your take on that criticism?
      - monocasa 8y ago
        It's hard to address such a vague statement, can you expand? FWIW, part of what I've been working on is formally verifying an eBPF runtime, to cut down the number of standard code bugs. If he's talking about Spectre stuff, there's going to have to be hardware mitigations as you can hit the same architecture flaws even without code generation. It was just easier for Project Zero to use BPF rather than track down the code sequences they were looking for in the existing kernel.
smaddox 8y ago
Well, considering the hardware sandboxing is also imperfect (Spectre, meltdown, rowhammer), you don't really have a choice but to use at least some software sandboxing. If you can use only software sandboxing, then there are potentially big wins. Seems like worth trying.
- twtw 8y ago
  It is definitely worth trying as an experiment, but the discussion around wasmjit doesn't seem like it's a research project. Its worth noting that this has in many senses already been extensively research. This was a significant part of the design of Singularity OS, a micro kernel from Microsoft Research with software isolation between components (with much more formal sandboxing than wasmjit, verified with static analysis). Also there have been several operating systems with a JVM in the kernel over the last 2 decades with many of the same benefits.
saagarjha 8y ago
Your point seems valid to me as well; the responses seem to be making the argument that a singular sandbox is good enough, without keeping mind the fact that most software runs inside of multiple sandboxes as a sort of “defense in depth” to provide protection even if one of the layers fails.
ec109685 8y ago
What attack scenarios you are envisioning here? If the host is only running this software (e.g. a single purpose server, firewall, etc.), the security benefits of process isolation are reduced. Clearly if you are trying to run hostile software from multiple vendors on the same box (e.g. a browser), you want more sandboxes.
- the8472 8y ago
  If you're running a single purpose server do you still need a sandbox though? Wouldn't a unikernel do the job and avoid the JIT compilation overhead?
  - cejetvole 8y ago
    It might make more sense for you to run a unikernel but it may also be important to you to use your existing tooling / monitoring infrastructure to administer your server. For example, being able to ssh into a server when things go wrong can make things a lot easier.
- twtw 8y ago
  I see a future where the npm-style ecosystem develops with the JIT running inside the kernel, so you end up running a "trusted" application ends up running a whole lot of untrusted code via dependencies. Usually process isolation gives you some limit on how much damage something like that can do. Also, if you are 100% sure that the code in it is trusted, then there really is no reason to sandbox it, right? If the intent is to only run trusted code, why is this article about spectre mitigations?
  - kiriakasis 8y ago
    > if you are 100% sure that the code in it is trusted, then there really is no reason to sandbox it, right? If the intent is to only run trusted code, Trusted code can have bugs; sandboxing, in a sense, is always useful (not always beneficial)
    - deleted 8y ago
cejetvole 8y ago
There isn't anything inherently different about a WebAssembly-based software sandbox from previous software sandboxes like the ones you mentioned. It maintains a reasonable level of protection for additional benefits. If those benefits aren't worth it for your use case then you don't need to take the additional potential risk. We don't live in a computing monoculture, what's risky for you may not be risky for me, and what's valuable to you may not be valuable to me. It's worth noting that the code that will be run under Wasmjit will in general be more trusted than code you run in a browser while browsing the Web. For those who only run trusted code on their servers, Wasmjit incurs no additional risk and provides potentially significant benefit.
- kiriakasis 8y ago
  https://news.ycombinator.com/item?id=18527535 https://news.ycombinator.com/item?id=18527535 Another story pointed out that wasm today still lacks a few features and so, mostly regarding run-time inspection, has drawbacks server-side. In the article there is no mention that wasm tomorrow could be quite different from wasm today; overall I believe the optimism is warranted.
naasking 8y ago
> What makes this sandbox different from previous sandboxes (JVM, browsers, etc) that makes proponents sufficiently confident to put it in the kernel Formalization was part of the process of developing WASM. That's already a large difference, and it led to a simpler instruction set that's more amenable to analysis.
- twtw 8y ago
  Do you have any pointers to resources regarding the formal model for web assembly?
  - naasking 8y ago
    https://news.ycombinator.com/item?id=17575111 https://news.ycombinator.com/item?id=17575111
saurik 8y ago
This VM is super simple: it doesn't have garbage collection and objects and class-level dynamic linking and a type system... it just has integers and floating point numbers in a static memory array. It gives you a stack: that's about it; it is more similar to BPF or something, which notably is in the kernel. (I still agree with you.)

titzer 8y ago

Do not run untrusted code in the kernel. Period. Double period. Triple period. Speculative vulnerabilities and side channels are a lot more than just variant 1 and 2. WasmJIT is a nice project. But let me repeat. Do not put it in the kernel and run untrusted code through it, period. Software mitigations are not sufficient. This is not WASM's fault, this is not WasmJIT's fault; it's just a fundamental reality of modern hardware.

reitzensteinm 8y ago
What if the sandbox completely denies any method of timing to the untrusted code? Including thread to thread communication. Timing related side channels seem unavoidable, but that doesn't mean that answer in -> answer out computation is necessarily intrinsically able to perform it. Also, if you can trust code to not be malicious, but not to be correct, loading code via WASM and executing it in the same address space isn't necessarily crazy.
- titzer 8y ago
  > What if the sandbox completely denies any method of timing to the untrusted code? Including thread to thread communication. You can construct timers from shared mutable memory (think: counter thread), but even in shared-nothing systems, one can construct timers using message passing (think: a crude timer that counts messages in one process and a sender hammering it with messages). > Also, if you can trust code to not be malicious, but not to be correct, loading code via WASM and executing it in the same address space isn't necessarily crazy. Sure, agreed. This is why my comment mentioned untrusted code. WebAssembly sandboxing makes sense to protect the kernel from OOB writes, but it cannot guarantee no OOB reads from speculative side-channels.