3 ms·

Google doesn't run VMs. Or, rather, even VMs that make up GCE are actually running in containers. From the Borg paper: https://pdos.csail.mit.edu/6.824/papers/

by puzzle 9y ago

Google doesn't run VMs. Or, rather, even VMs that make up GCE are actually running in containers. From the Borg paper: https://pdos.csail.mit.edu/6.824/papers/borg.pdf https://pdos.csail.mit.edu/6.824/papers/borg.pdf VMs and security sandboxing techniques are used to run external software by Google’s AppEngine (GAE) [38] and Google Compute Engine (GCE). We run each hosted VM in a KVM process [54] that runs as a Borg task. There might be a few machines here and there running actual VMs on bare metal, but those are going to be very special cases. Using Infrastore, it's easy to find which teams are running which containers whose binaries were built before the new compiler defaults. https://github.com/kubernetes/kubernetes/issues/44095 https://github.com/kubernetes/kubernetes/issues/44095

cthalupa 9y ago
I don't understand how you think that answers his question, as being in a container wouldn't magically add any more protections against these exploits than being in just a KVM VM. I'm also not sure that "Borg task" means it's in a container, vs. just being something scheduled by Borg, but it isn't particularly important either way. If the question is 'How do they guarantee their customers aren't using bad binaries', the answer is they don't, and has nothing to do with VMs vs VMs in containers, etc.
- puzzle 9y ago
  I should have been clearer: their external customers can and will use bad binaries inside their VMs, but everything else, especially anything at the boundaries between VMs, will not: it will have the appropriate mitigations in place. Running a VM in a Borg task (container) does not add extra security in this case, but it does help with auditing and catching the snowflakes and stragglers that invariably end up taking a disproportionate amount of time with changes like this. That kind of auditing can be easily done organization-wide, instead of having every team reinvent the wheel their own way.

by puzzle 9y ago

cthalupa 9y ago

I don't understand how you think that answers his question, as being in a container wouldn't magically add any more protections against these exploits than being in just a KVM VM. I'm also not sure that "Borg task" means it's in a container, vs. just being something scheduled by Borg, but it isn't particularly important either way. If the question is 'How do they guarantee their customers aren't using bad binaries', the answer is they don't, and has nothing to do with VMs vs VMs in containers, etc.

puzzle 9y ago
I should have been clearer: their external customers can and will use bad binaries inside their VMs, but everything else, especially anything at the boundaries between VMs, will not: it will have the appropriate mitigations in place. Running a VM in a Borg task (container) does not add extra security in this case, but it does help with auditing and catching the snowflakes and stragglers that invariably end up taking a disproportionate amount of time with changes like this. That kind of auditing can be easily done organization-wide, instead of having every team reinvent the wheel their own way.