3 ms·

IPC is a tad slower than not IPC. However, stuff like WebAssembly and other sandboxing methods can be used to leverage two processes into the same address spac

by zaarn 7y ago

IPC is a tad slower than not IPC. However, stuff like WebAssembly and other sandboxing methods can be used to leverage two processes into the same address space. Your filesystem driver then simply lives as a module in address space and it's a normal process. The IPC turns into a simple jump using a pointer value provided by the kernel (which depending on the trust level can provide parameter validation or can be a plain pointer to the correct function).

hvidgaard 7y ago
How would that work? The entire premise of Microkernels is that everything runs in a dedicated processes such that if one crashes, the system can continue all other processes and only need to recover that single process. The cost of switching processes are not a problem of address space, the MMU makes sure that is not a problem as shared memory is a reasonable tradeoff for certain domains where performance is necessary. Normally you'd rather want use messages but that is a different topic. In either case, the cost of a process switch is handling the registers and cache - that will not go away no matter how you do it, which is why multicore implementation with messages can actually turn out being faster. Less switching and more locality.
- zaarn 7y ago
  Well, with webassembly the process can run under ring0 but still be isolated as if it was running in ring3. The crash mechanism isn't different; if the driver crashes then the kernel can kill it and deallocate it's resources, a device manager process can then restart the driver. The advantage is that you can include small, audited and verified binaries that can run code as ring 0 to remove abstraction from accessing the raw hardware. The cost of switching processes is significantly reduced when you don't need to switch privilege levels and not having to invalidate the TLB as well as not having to change address space at all will make a context switch not significantly more expensive than a function call. You get compile-time isolation and can still take advantage of the MMU when needed. And using such an implementation does not prevent you from implementing your driver such that it can run on each core and take advantage of it or even passes messages. An ethernet driver could still, for example, pass a message when the "send data to tcp socket" function is called while allowing another program to use the same function without any message passing, depending on what is better for your use case.
  - hvidgaard 7y ago
    I'm not sure I believe it's a good idea to give up on the hardware protection, as that leaves it to the software implementation to ensure it's secure. If you compromise an application in user mode, it will not get you far without another exploit in something that runs in supervisor mode. The hardware makes that certain, and it's well verified. We've seen time and time again that simple buffer overflows exploit things, and the more the runs in supervisor mode, the larger the attack surface is. If a driver runs in user mode, an exploit needs to exploit the hardware as well - and that is for all intents and purposes something that we see very rarely. If the same driver runs in "software user mode", but executing as supervisor (basically inside an VM environment), we need constant security checks in software, and an exploit now have the VM code to further exploit, if successful that will automatically grant it supervisor access. In both cases it's assumed that neither implementation has access to more interfaces than necessary for it to do it's work. For instance, a driver for a mouse does not need access to the disks.
    - zaarn 7y ago
      The thing is, you don't have to give up hardware protection. If you don't trust code, you can still run it in ring 3 with all the associated overhead. The point is being able to choose how close an application runs to "no overhead" until you're at a level where the driver is a function call away. From my experience, a lot of hardware is terribly insecure against exploits. Not necessarily the CPU but stuff like your GPU or HBAs, ethernet cards, etc. With software containment, the advantage is that you can set it up that drivers need to declare their interfaces and privileges beforehand. In an ELF or WASM you have to declare imported and linked functions, it should not be difficult to leverage that to determine what a driver can effectively do. With WASM you get the added benefit that doing anything but using declared interfaces results in a compile-time error. A driver can be written so that a minimal, audited interface exists to talk to the hardware almost directly with some security checks and then the WASM part that handles the larger logic parts and provides the actual functionality. WASM isn't a supervisor, so exploits on VM code aren't that relevant. Exploiting the WASM compiler/interpreter/JIT is more interesting but those are exposed to the daily internet shitstorm exploits, so I think they are fairly safe.
- geofft 7y ago
  It works by defining "process" in a way that doesn't imply "registers and cache." If the concern is the ability for one process to survive despite arbitrary misbehavior from another process (and not, say, Spectre-style attacks), software-based fault isolation like NaCl or PittSFIeld or using a restricted language runtime like wasm or Lua demonstrably works fine for that, no hardware context switch required.
  - hvidgaard 7y ago
    That sounds like a lot of overhead for a kernel. There is a reason it's normally written in a mix of C and ASM.
    - zaarn 7y ago
      It wouldn't be that much overhead in reality, you can still write the kernel in a mix of C and ASM (or more modern; Rust and ASM). The kernel doesn't even need to know about these isolation mechanisms, since you run stuff in ring 0, you can hook the appropriate APIs and interfaces in a process. The kernel itself would be insanely small and thusly more easy to defend.
- eadan 7y ago
  The experimental Singularity OS [0] from Microsoft implemented processes in the same address space using so-called "Software Isolated Processes" or SIPs [1]. Singularity was a microkernel design with the filesystem, networking, drivers implemented outside the kernel in a variant of C# called Sing#. It seems Sing# had ownership semantics similar to Rust which allowed SIPs to share memory by transferring ownership through lightweight message passing. [0] https://en.wikipedia.org/wiki/Singularity_%28operating_system%29 https://en.wikipedia.org/wiki/Singularity_%28operating_syste... [1] https://courses.cs.washington.edu/courses/cse551/15sp/papers/singularity-osr07.pdf https://courses.cs.washington.edu/courses/cse551/15sp/papers...
- snvzz 7y ago
  >cost of context switch Page 135. http://sigops.org/s/conferences/sosp/2013/papers/p133-elphinstone.pdf http://sigops.org/s/conferences/sosp/2013/papers/p133-elphin...
targonca 7y ago
So, you'd like to expose the whole kernel to Spectre-type attacks. Wonderful!
- zaarn 7y ago
  Well, for trusted code that doesn't expose any mechanism to run foreign code (ie browsers), spectre is largely a non-issue. So the trusted core part of the OS can run without any spectre prevention, though you can still enable the various hardware protections available in the chicken bits. And if it's necessary to protect against spectre attacks, you can use shim layers or even isolation into ring3 to take preventative measures. This allows leveraging performance were important and security where necessary. If it's in webassembly, you can even run two versions of a driver; one with spectre-mitigations compiled in and one without, sharing one memory space and the kernel can choose to invoke either one depending on the call chain.
  - zzzcpan 7y ago
    Trusted code has to be free from vulnerabilities to be immune, so it's still an issue even for trusted code. And I'm pretty sure neither webassembly nor other sandboxing methods can fully mitigate speculative attacks on out-of-order CPUs within the same address space, you'd need a programming language with a compiler designed from scratch for it.
    - zaarn 7y ago
      Well, it doesn't have to be free from vulnerabilities, not any more than any other OS code. The sandboxed code that is running trusted (ie without trampolines and spectre-defenses) would still hold the guarantees given by the sandbox (WASM), which are pretty much on par with what a modern browser can do for JS and WASM. And keep in mind that both WASM and JS now have spectre-defenses, so there is no need for a PL from scratch for this.
      - zzzcpan 7y ago
        > And keep in mind that both WASM and JS now have spectre-defenses, so there is no need for a PL from scratch for this. As far as I remember they weren't able to defend from side channel attacks within the same process completely and decided to rely on process isolation instead, estimating it would be too much work to address all known spectre class vulnerabilities on their existing compilers and too hard to ensure for defenses not to be broken later by compiler developers.
eadan 7y ago
Nebulet [0] was an interesting take on this -- a Rust microkernel compiled to WebAssembly. Unfortunately, the project seems to have run out of steam. [0] https://github.com/nebulet/nebulet https://github.com/nebulet/nebulet

by zaarn 7y ago

hvidgaard 7y ago

How would that work? The entire premise of Microkernels is that everything runs in a dedicated processes such that if one crashes, the system can continue all other processes and only need to recover that single process. The cost of switching processes are not a problem of address space, the MMU makes sure that is not a problem as shared memory is a reasonable tradeoff for certain domains where performance is necessary. Normally you'd rather want use messages but that is a different topic. In either case, the cost of a process switch is handling the registers and cache - that will not go away no matter how you do it, which is why multicore implementation with messages can actually turn out being faster. Less switching and more locality.

zaarn 7y ago
Well, with webassembly the process can run under ring0 but still be isolated as if it was running in ring3. The crash mechanism isn't different; if the driver crashes then the kernel can kill it and deallocate it's resources, a device manager process can then restart the driver. The advantage is that you can include small, audited and verified binaries that can run code as ring 0 to remove abstraction from accessing the raw hardware. The cost of switching processes is significantly reduced when you don't need to switch privilege levels and not having to invalidate the TLB as well as not having to change address space at all will make a context switch not significantly more expensive than a function call. You get compile-time isolation and can still take advantage of the MMU when needed. And using such an implementation does not prevent you from implementing your driver such that it can run on each core and take advantage of it or even passes messages. An ethernet driver could still, for example, pass a message when the "send data to tcp socket" function is called while allowing another program to use the same function without any message passing, depending on what is better for your use case.
- hvidgaard 7y ago
  I'm not sure I believe it's a good idea to give up on the hardware protection, as that leaves it to the software implementation to ensure it's secure. If you compromise an application in user mode, it will not get you far without another exploit in something that runs in supervisor mode. The hardware makes that certain, and it's well verified. We've seen time and time again that simple buffer overflows exploit things, and the more the runs in supervisor mode, the larger the attack surface is. If a driver runs in user mode, an exploit needs to exploit the hardware as well - and that is for all intents and purposes something that we see very rarely. If the same driver runs in "software user mode", but executing as supervisor (basically inside an VM environment), we need constant security checks in software, and an exploit now have the VM code to further exploit, if successful that will automatically grant it supervisor access. In both cases it's assumed that neither implementation has access to more interfaces than necessary for it to do it's work. For instance, a driver for a mouse does not need access to the disks.
  - zaarn 7y ago
    The thing is, you don't have to give up hardware protection. If you don't trust code, you can still run it in ring 3 with all the associated overhead. The point is being able to choose how close an application runs to "no overhead" until you're at a level where the driver is a function call away. From my experience, a lot of hardware is terribly insecure against exploits. Not necessarily the CPU but stuff like your GPU or HBAs, ethernet cards, etc. With software containment, the advantage is that you can set it up that drivers need to declare their interfaces and privileges beforehand. In an ELF or WASM you have to declare imported and linked functions, it should not be difficult to leverage that to determine what a driver can effectively do. With WASM you get the added benefit that doing anything but using declared interfaces results in a compile-time error. A driver can be written so that a minimal, audited interface exists to talk to the hardware almost directly with some security checks and then the WASM part that handles the larger logic parts and provides the actual functionality. WASM isn't a supervisor, so exploits on VM code aren't that relevant. Exploiting the WASM compiler/interpreter/JIT is more interesting but those are exposed to the daily internet shitstorm exploits, so I think they are fairly safe.
geofft 7y ago
It works by defining "process" in a way that doesn't imply "registers and cache." If the concern is the ability for one process to survive despite arbitrary misbehavior from another process (and not, say, Spectre-style attacks), software-based fault isolation like NaCl or PittSFIeld or using a restricted language runtime like wasm or Lua demonstrably works fine for that, no hardware context switch required.
- hvidgaard 7y ago
  That sounds like a lot of overhead for a kernel. There is a reason it's normally written in a mix of C and ASM.
  - zaarn 7y ago
    It wouldn't be that much overhead in reality, you can still write the kernel in a mix of C and ASM (or more modern; Rust and ASM). The kernel doesn't even need to know about these isolation mechanisms, since you run stuff in ring 0, you can hook the appropriate APIs and interfaces in a process. The kernel itself would be insanely small and thusly more easy to defend.
eadan 7y ago
The experimental Singularity OS [0] from Microsoft implemented processes in the same address space using so-called "Software Isolated Processes" or SIPs [1]. Singularity was a microkernel design with the filesystem, networking, drivers implemented outside the kernel in a variant of C# called Sing#. It seems Sing# had ownership semantics similar to Rust which allowed SIPs to share memory by transferring ownership through lightweight message passing. [0] https://en.wikipedia.org/wiki/Singularity_%28operating_system%29 https://en.wikipedia.org/wiki/Singularity_%28operating_syste... [1] https://courses.cs.washington.edu/courses/cse551/15sp/papers/singularity-osr07.pdf https://courses.cs.washington.edu/courses/cse551/15sp/papers...
snvzz 7y ago
>cost of context switch Page 135. http://sigops.org/s/conferences/sosp/2013/papers/p133-elphinstone.pdf http://sigops.org/s/conferences/sosp/2013/papers/p133-elphin...

targonca 7y ago

So, you'd like to expose the whole kernel to Spectre-type attacks. Wonderful!

zaarn 7y ago
Well, for trusted code that doesn't expose any mechanism to run foreign code (ie browsers), spectre is largely a non-issue. So the trusted core part of the OS can run without any spectre prevention, though you can still enable the various hardware protections available in the chicken bits. And if it's necessary to protect against spectre attacks, you can use shim layers or even isolation into ring3 to take preventative measures. This allows leveraging performance were important and security where necessary. If it's in webassembly, you can even run two versions of a driver; one with spectre-mitigations compiled in and one without, sharing one memory space and the kernel can choose to invoke either one depending on the call chain.
- zzzcpan 7y ago
  Trusted code has to be free from vulnerabilities to be immune, so it's still an issue even for trusted code. And I'm pretty sure neither webassembly nor other sandboxing methods can fully mitigate speculative attacks on out-of-order CPUs within the same address space, you'd need a programming language with a compiler designed from scratch for it.
  - zaarn 7y ago
    Well, it doesn't have to be free from vulnerabilities, not any more than any other OS code. The sandboxed code that is running trusted (ie without trampolines and spectre-defenses) would still hold the guarantees given by the sandbox (WASM), which are pretty much on par with what a modern browser can do for JS and WASM. And keep in mind that both WASM and JS now have spectre-defenses, so there is no need for a PL from scratch for this.
    - zzzcpan 7y ago
      > And keep in mind that both WASM and JS now have spectre-defenses, so there is no need for a PL from scratch for this. As far as I remember they weren't able to defend from side channel attacks within the same process completely and decided to rely on process isolation instead, estimating it would be too much work to address all known spectre class vulnerabilities on their existing compilers and too hard to ensure for defenses not to be broken later by compiler developers.

eadan 7y ago

Nebulet [0] was an interesting take on this -- a Rust microkernel compiled to WebAssembly. Unfortunately, the project seems to have run out of steam. [0] https://github.com/nebulet/nebulet https://github.com/nebulet/nebulet