3 ms·

(I am co-author of the post) System calls are important, but only one factor. The linked doc is an attempt to clarify and delineate various costs. There are nu

(I am co-author of the post) System calls are important, but only one factor. The linked doc is an attempt to clarify and delineate various costs. There are number of platform options (the platform is what does syscall interception), and I don't believe any of them are 100x so to say "at least" is a bit disingenuous. You may have confused the "runsc-kvm" number with "using a VM". "runsc-kvm" is the system call performance of gVisor using the kvm platform, which is not a full VM [1]. In general the syscall cost in a VM depends entirely on the guest OS, since there is no VMEXIT for this operation. VMs are a valid choice depending on your workload, and this is providing an additional tool that provides an easy control for containerized infrastructure. You can use what works for you. Native containers certainly work as well, but you'll probably want to consider additional security controls of some form if you're really running untrusted stuff in there. [1] https://github.com/google/gvisor/tree/master/pkg/sentry/platform/kvm https://github.com/google/gvisor/tree/master/pkg/sentry/plat...

muricula 7y ago
You're right, that's not the chart I wanted to see. I'm just dubious that reimplementing lots of the Linux kernel in Go while paying the cost of the ptrace interception is worthwhile. It seems like you're just adding a lot of attack surface (admittedly managed code > native code) with a large perf impact. Do you have any docs on how the kvm-runsc platform works? Skimming the files, I don't see some of the bits necessary for a bluepill style hypervisor, so I'm not sure why parts are named bluepill in there. I also don't see a lot of the linux kernel paravirt vdev code I would expect, and you seem to imply that you're not telling KVM to enable syscall trapping for the guest.
- amscanne 7y ago
  I'm not sure what you mean by KVM syscall trapping for the guest. The bluepill refers to the fact that the Sentry runs transparently in VMX non-root ring 0 and regular host ring 3. I'm not sure what to provide re: docs -- the code is all there, reasonably documented and there are discussions on the public groups of how the KVM platform works. I feel a bit like you're coming in with a specific set of ideas and skimming files (e.g. the performance guide and the code itself) in order to confirm an existing understanding, but it's just not working. I'd love more precise criticisms re: adding to the attack surface, but otherwise I'm not sure how I can help.
  - muricula 7y ago
    I'm very skeptical about the platform and don't have the time to devote to reading the codebase or having conversations as I would like. The TL;DR is that the syscall interception technique seems expensive and I wonder if you will write all sorts of logic bugs in the sentry broker. It seems like you folks care about security, and have some good ideas, but if you really care about hostile multi-tenant containers, why not stick the container in a VM and call it a day?
    - yoshiat 7y ago
      I replied in other comments but our talk at Next'19 [1] includes a story by one of our customers, which may help understand the use cases. In a nutshell, GKE Sandbox should allow sharing the resources of GKE Nodes (VMs) among multiple tenants. [1] https://www.youtube.com/watch?v=TQfc8OlB2sg https://www.youtube.com/watch?v=TQfc8OlB2sg

muricula 7y ago

You're right, that's not the chart I wanted to see. I'm just dubious that reimplementing lots of the Linux kernel in Go while paying the cost of the ptrace interception is worthwhile. It seems like you're just adding a lot of attack surface (admittedly managed code > native code) with a large perf impact. Do you have any docs on how the kvm-runsc platform works? Skimming the files, I don't see some of the bits necessary for a bluepill style hypervisor, so I'm not sure why parts are named bluepill in there. I also don't see a lot of the linux kernel paravirt vdev code I would expect, and you seem to imply that you're not telling KVM to enable syscall trapping for the guest.

amscanne 7y ago
I'm not sure what you mean by KVM syscall trapping for the guest. The bluepill refers to the fact that the Sentry runs transparently in VMX non-root ring 0 and regular host ring 3. I'm not sure what to provide re: docs -- the code is all there, reasonably documented and there are discussions on the public groups of how the KVM platform works. I feel a bit like you're coming in with a specific set of ideas and skimming files (e.g. the performance guide and the code itself) in order to confirm an existing understanding, but it's just not working. I'd love more precise criticisms re: adding to the attack surface, but otherwise I'm not sure how I can help.
- muricula 7y ago
  I'm very skeptical about the platform and don't have the time to devote to reading the codebase or having conversations as I would like. The TL;DR is that the syscall interception technique seems expensive and I wonder if you will write all sorts of logic bugs in the sentry broker. It seems like you folks care about security, and have some good ideas, but if you really care about hostile multi-tenant containers, why not stick the container in a VM and call it a day?
  - yoshiat 7y ago
    I replied in other comments but our talk at Next'19 [1] includes a story by one of our customers, which may help understand the use cases. In a nutshell, GKE Sandbox should allow sharing the resources of GKE Nodes (VMs) among multiple tenants. [1] https://www.youtube.com/watch?v=TQfc8OlB2sg https://www.youtube.com/watch?v=TQfc8OlB2sg