5 ms·

I hate to rain on this interesting project’s parade, but if you need full sandbox isolation then you should probably look to full VM isolation (ala Kata contain

I hate to rain on this interesting project’s parade, but if you need full sandbox isolation then you should probably look to full VM isolation (ala Kata containers, formerly Clear containers). User namespace-ing, SecComp, and Selinux/apparmor buy you about as much of a sandbox as you’ll need, with the one caveat that a kernel exploit could still take you down (all other exoits are rendered sandboxed). If you need that final kernel sandboxing, then a full VM is your only guarantee. UML still sits on top of an exploitable kernel, and presumably this project itself can be hacked. While it certainly is better than nothing the only thing it seems to be buying you above Kata containers is a faster spin up time and the ability to dynamically resize the container. Maybe that trade off is worth it to someone, but the added performance overhead and the demi-kernel isolation seem like a high price to for those features.

amscanne 8y ago
Hi! I work on this project. Nothing wrong with a full VM, but I don't think it's a panacea (and you probably shouldn't guarantee security). You may have taken the UML comparison to heart: did you look at the KVM platform? I'm not clear on the distinctions you're making in that case -- a kernel escape would look a lot like a user-space VMM code execution vulnerability (which also sits on top an exploitable kernel).
- hacknat 8y ago
  Well, I wasn’t arguing for a VM being a panacea, but in the context of not being satisfied with the Linux primitives for sandboxing, I think it is the next logical step up in security. From my perspective this project seems like an intermediate jump from Linux containerization primitives and a full blown VM, and I was wondering out loud who fits that use case? Finally, I didn’t mention KVM, but my understanding of KVM is that it’s isolation primitive was the hardware virtualization instructions (or at least could be, I’m not sure if it has a PV mode or not). I guess my question for you would be: In what context would I want to use this over something like Kata containers?
  - amscanne 8y ago
    KVM is the kernel interface for virtualization features, but the model created (i.e. the emulated hardware or lack thereof) is up to the user space component (normally QEMU). I think your understanding of KVM is tied with a specific implementation. FWIW, I don't disagree that it's an intermediate step in some regards. The use cases follow (also from trade-offs discussed in the README). I can't speculate on a stranger's needs, it's great if Kata works for yours. I also think that approach is valuable (as an aside, I authored an experimental project with a similar approach years ago [1]). [1] https://github.com/google/novm https://github.com/google/novm
    - tejasmanohar 8y ago
      Is there a comparison of Kata and gVisor based on how they act functionally rather than how they are implemented under-the-hood? Like the OP, I'm curious when you'd use this over Kata.
      - houseofzeus 8y ago
        Not a direct comparison of these projects specifically but here is the write-up that was presented in the context of the Kubernetes SIG Node discussions about this topic: https://docs.google.com/document/d/1QQ5u1RBDLXWvC8K3pscTtTRThsOeBSts_imYEoRyw8A/edit# https://docs.google.com/document/d/1QQ5u1RBDLXWvC8K3pscTtTRT...
acdha 8y ago
Your comment seems to be a murkier restatement of the discussion in the post, which also mentioned Kata. It’d be more useful to give details about why you don’t think the flexibility and lower resource usage are worthwhile rather than acting like they didn’t explicitly discuss the trade offs.
- hacknat 8y ago
  The question is, why choose this over Kata...I guess. The post didn’t really do a good job of telling me why.
deleted 8y ago
[deleted]
nickpsecurity 8y ago
" then a full VM is your only guarantee" VM's on commercial and FOSS platforms have been unable to ensure security as far back as first pentest of VM/370. They were too complicated depending on a lot of privileged code. The founders of INFOSEC took aim at it with KVM/370 and VAX VMM Security Kernel. One example (see esp layering and assurance sections): http://lukemuehlhauser.com/wp-content/uploads/Karger-et-al-A-retrospective-on-the-VAX-VMM-security-kernel.pdf http://lukemuehlhauser.com/wp-content/uploads/Karger-et-al-A... Even those systems had too much complexity by our standards. They aimed for nearly-perfect implementation of the core kernels mediating everything. That led to separation kernels like INTEGRITY-178B, LynxSecure, seL4, and Muen. They're usually just 4-12Kloc. Whereas, projects trying to achieve similar assurance activities on KVM and Xen mostly gave up due to complexity. If aiming for correctness more than security, projects like Nova microhypervisor and GenodeOS show similar partitioning can still help. Although separation kernels succeeded, their requirements assumed hardware/firmware that was sane and would work correctly. What people are using for virtualized workloads has been overly complex with shoddy implementations. Bypasses keep happening via everything from CPU's to RAM to peripheral firmware. Although methods exist to assure them, the companies providing them are not using them. So, if you want anything close to a guarantee, you have to use physical separation with strongly-mediated, communications systems over optical links. The boards need to be electrically isolated from each other in their own TEMPEST boxes or safes. Alternatively, use old technique from separation kernel era of putting all the complex, enemy-facing stuff on their own boxes that interpret stuff down to requests in simple protocols interfacing with the trusted boxes on better hardware. Reduces hardware cost but what you get is nothing like a Google or Amazon cloud. It's more like a partitioned version of SoftLayer.
- 0xdeadbeefbabe 8y ago
  This is a tangent: I'm baffled by the subtext that the attackers have such an easy time attacking, and defenders have such a hard time defending that security experts are our only hope. It's really self serving, even if partly true. I don't know where the truth begins and the self serving ends though.
  - peterwwillis 8y ago

amscanne 8y ago

Hi! I work on this project. Nothing wrong with a full VM, but I don't think it's a panacea (and you probably shouldn't guarantee security). You may have taken the UML comparison to heart: did you look at the KVM platform? I'm not clear on the distinctions you're making in that case -- a kernel escape would look a lot like a user-space VMM code execution vulnerability (which also sits on top an exploitable kernel).

hacknat 8y ago
Well, I wasn’t arguing for a VM being a panacea, but in the context of not being satisfied with the Linux primitives for sandboxing, I think it is the next logical step up in security. From my perspective this project seems like an intermediate jump from Linux containerization primitives and a full blown VM, and I was wondering out loud who fits that use case? Finally, I didn’t mention KVM, but my understanding of KVM is that it’s isolation primitive was the hardware virtualization instructions (or at least could be, I’m not sure if it has a PV mode or not). I guess my question for you would be: In what context would I want to use this over something like Kata containers?
- amscanne 8y ago
  KVM is the kernel interface for virtualization features, but the model created (i.e. the emulated hardware or lack thereof) is up to the user space component (normally QEMU). I think your understanding of KVM is tied with a specific implementation. FWIW, I don't disagree that it's an intermediate step in some regards. The use cases follow (also from trade-offs discussed in the README). I can't speculate on a stranger's needs, it's great if Kata works for yours. I also think that approach is valuable (as an aside, I authored an experimental project with a similar approach years ago [1]). [1] https://github.com/google/novm https://github.com/google/novm
  - tejasmanohar 8y ago
    Is there a comparison of Kata and gVisor based on how they act functionally rather than how they are implemented under-the-hood? Like the OP, I'm curious when you'd use this over Kata.
    - houseofzeus 8y ago
      Not a direct comparison of these projects specifically but here is the write-up that was presented in the context of the Kubernetes SIG Node discussions about this topic: https://docs.google.com/document/d/1QQ5u1RBDLXWvC8K3pscTtTRThsOeBSts_imYEoRyw8A/edit# https://docs.google.com/document/d/1QQ5u1RBDLXWvC8K3pscTtTRT...

acdha 8y ago

Your comment seems to be a murkier restatement of the discussion in the post, which also mentioned Kata. It’d be more useful to give details about why you don’t think the flexibility and lower resource usage are worthwhile rather than acting like they didn’t explicitly discuss the trade offs.

hacknat 8y ago
The question is, why choose this over Kata...I guess. The post didn’t really do a good job of telling me why.

deleted 8y ago

[deleted]

nickpsecurity 8y ago

" then a full VM is your only guarantee" VM's on commercial and FOSS platforms have been unable to ensure security as far back as first pentest of VM/370. They were too complicated depending on a lot of privileged code. The founders of INFOSEC took aim at it with KVM/370 and VAX VMM Security Kernel. One example (see esp layering and assurance sections): http://lukemuehlhauser.com/wp-content/uploads/Karger-et-al-A-retrospective-on-the-VAX-VMM-security-kernel.pdf http://lukemuehlhauser.com/wp-content/uploads/Karger-et-al-A... Even those systems had too much complexity by our standards. They aimed for nearly-perfect implementation of the core kernels mediating everything. That led to separation kernels like INTEGRITY-178B, LynxSecure, seL4, and Muen. They're usually just 4-12Kloc. Whereas, projects trying to achieve similar assurance activities on KVM and Xen mostly gave up due to complexity. If aiming for correctness more than security, projects like Nova microhypervisor and GenodeOS show similar partitioning can still help. Although separation kernels succeeded, their requirements assumed hardware/firmware that was sane and would work correctly. What people are using for virtualized workloads has been overly complex with shoddy implementations. Bypasses keep happening via everything from CPU's to RAM to peripheral firmware. Although methods exist to assure them, the companies providing them are not using them. So, if you want anything close to a guarantee, you have to use physical separation with strongly-mediated, communications systems over optical links. The boards need to be electrically isolated from each other in their own TEMPEST boxes or safes. Alternatively, use old technique from separation kernel era of putting all the complex, enemy-facing stuff on their own boxes that interpret stuff down to requests in simple protocols interfacing with the trusted boxes on better hardware. Reduces hardware cost but what you get is nothing like a Google or Amazon cloud. It's more like a partitioned version of SoftLayer.

0xdeadbeefbabe 8y ago
This is a tangent: I'm baffled by the subtext that the attackers have such an easy time attacking, and defenders have such a hard time defending that security experts are our only hope. It's really self serving, even if partly true. I don't know where the truth begins and the self serving ends though.
- peterwwillis 8y ago