5 ms·

For AWS: https://aws.amazon.com/security/security-bulletins/AWS-2018-019/ https://aws.amazon.com/security/security-bulletins/AWS-2018-... (Disclaimer: I work a

For AWS: https://aws.amazon.com/security/security-bulletins/AWS-2018-019/ https://aws.amazon.com/security/security-bulletins/AWS-2018-... (Disclaimer: I work at AWS, but I am not linking this in any sort of official capacity. I don't know any more details beyond what is listed in that bulletin, and can't answer any questions related to this, unfortunately.)

rodgerd 8y ago
Contrast: "Meanwhile, we suggest using the stronger security and isolation properties of EC2 instances to separate any untrusted workloads." with: "Google Compute Engine employs host isolation features which ensure that an individual core is never concurrently shared between distinct virtual machines. This isolation also ensures that, in the case that different virtual machines are scheduled sequentially, the L1 data cache is completely flushed to ensure that no vulnerable state remains." The former does not inspire confidence. Given the hypervisor on EC2 is opaque to me, I'm not sure how I'm supposed to avoid co-tenanting in a risky fashion.
- h4b4n3r0 8y ago
  How is the latter possible in case of 1 vCPU VM instances if 1 vCPU == 1 hyperthread? Do they give you a full core in this case?
  - rodgerd 8y ago
    Or map pairs of hyperthreads to each requested vCPU, or disable hyperthreading. Which kills co-tenancy overcommit and makes the infra more expensive for a cloud provider.
  - _msw_ 8y ago
    1 vCPU instances do not simultaneously share cores with other customer instances via SMT (Intel Hyper-Threading Technology). A core can be sequentially time sliced between customer instances when you use a fractional (m3.medium) or burst (T2 instances) CPU instance type. https://aws.amazon.com/ec2/instance-types/ https://aws.amazon.com/ec2/instance-types/ includes a note that provides some additional information about when vCPUs utilize Intel HT Technology: Each vCPU is a hyperthread of an Intel Xeon core except for T2 and m3.medium. (emphasis added) https://aws.amazon.com/ec2/virtualcores/ https://aws.amazon.com/ec2/virtualcores/ provides a table of the number of cores allocated per instance. Notice that an instance like m5.large that has 2 vCPUs and 1 core. An instance like t2.small has 1 vCPU and 1 core. Disclosure: I work for AWS
    - h4b4n3r0 8y ago
      Um, no. If I schedule a micro instance in a particular zone and nothing else, for the duration of my timeslice the VM will have to monopolize the core. Optimal use of vCPUs would demand that two VM threads get scheduled to the core to take advantage of HT, which Google says it won’t do. Timeslicing doesn’t solve this problem. At least that’s my reading of what they said.
      - _msw_ 8y ago
        Re-read what I wrote above. It covered both simultaneous and sequential sharing. Here is the part you are concerned about: instances do not simultaneously share cores with other customer instances via SMT.
        h4b4n3r0 8y ago
        Re-read what I wrote. Imagine a machine with 8 cores and 16 hyperthreads. Now imagine you need to optimally use the execution ports in each core. To do that you would need to have 16 VM threads all feeding the ports simultaneously, 2 per core. If all your workloads on that machine are from different customers, you can’t do more than 8. If you timeslice more than 8, then the question is, what is it that you’re selling as a vCPU? If it’s a timesliced core, then it’s not what they say they’re selling (a hyperthread).
        cthalupa 8y ago
        You: > If it’s a timesliced core, then it’s not what they say they’re selling (a hyperthread). vs. _msw_: > A core can be sequentially time sliced between customer instances when you use a fractional (m3.medium) or burst (T2 instances) CPU instance type. AWS instance types page ( https://aws.amazon.com/ec2/instance-types/ https://aws.amazon.com/ec2/instance-types/ ): > Each vCPU is a hyperthread of an Intel Xeon core except for T2 and m3.medium.
        h4b4n3r0 8y ago
- garethl 8y ago
  It does say, "All EC2 host infrastructure has been updated with these new protections, and no customer action is required at the infrastructure level." "Meanwhile, we suggest using the stronger security and isolation properties of EC2 instances to separate any untrusted workloads." As I read it, it is talking about running code within your instance - if you have untrusted workloads, rather run them in a separate instance, so as not to encounter issues like this cross-process.
- markdoubleyou 8y ago
  Amazon offers "dedicated hosts" and "dedicated instances", which you can specify in the "tenancy" field when launching an EC2 instance. I bet that's the stronger isolation that they're referring to. Costs more, though. https://aws.amazon.com/ec2/dedicated-hosts/ https://aws.amazon.com/ec2/dedicated-hosts/
  - _msw_ 8y ago
    This guidance is primarily for customers that are running untrusted code within their EC2 instances. The "strong isolation of EC2 instances" refers to the properties of isolation provided by EC2's virtualization compared to processes within an operating system. It is challenging to safely and securely run untrusted code within sandboxes and processes using general purpose software. However, the hypervisor and hardware based virtualization of EC2 instances is engineered to provide isolation between mutually untrusted instances. There are several reasons that customers may want to use dedicated instances or dedicated hosts, so we provide those tenancy options as well. The most common reason customers use dedicated hosts so that they can bring their own software licenses, which are often tied to a physical host. Disclosure: I work for AWS.
- deathanatos 8y ago
  > Given the hypervisor on EC2 is opaque to me, I'm not sure how I'm supposed to avoid co-tenanting in a risky fashion. In Linux, at least, the Xen hypervisor on EC2 exposes some information about itself at /sys/hypervisor. In particular, I think /sys/hypervisor/uuid would allow you to detect co-tenanting (between two VMs of your own). Not saying that I think you should do that, or that'd I'd want to — it'd be a PITA to coordinate amongst VMs, and I'm not sure it would matter (what if you're co-tentant w/ a malicious VM? even if you detect it, how do you get out of it?). That is, inside the VM seems like wholly the wrong place to attempt to deal with this. But I don't think many people realize /sys/hypervisor exists.

rodgerd 8y ago

Contrast: "Meanwhile, we suggest using the stronger security and isolation properties of EC2 instances to separate any untrusted workloads." with: "Google Compute Engine employs host isolation features which ensure that an individual core is never concurrently shared between distinct virtual machines. This isolation also ensures that, in the case that different virtual machines are scheduled sequentially, the L1 data cache is completely flushed to ensure that no vulnerable state remains." The former does not inspire confidence. Given the hypervisor on EC2 is opaque to me, I'm not sure how I'm supposed to avoid co-tenanting in a risky fashion.

h4b4n3r0 8y ago
How is the latter possible in case of 1 vCPU VM instances if 1 vCPU == 1 hyperthread? Do they give you a full core in this case?
- rodgerd 8y ago
  Or map pairs of hyperthreads to each requested vCPU, or disable hyperthreading. Which kills co-tenancy overcommit and makes the infra more expensive for a cloud provider.
- _msw_ 8y ago
  1 vCPU instances do not simultaneously share cores with other customer instances via SMT (Intel Hyper-Threading Technology). A core can be sequentially time sliced between customer instances when you use a fractional (m3.medium) or burst (T2 instances) CPU instance type. https://aws.amazon.com/ec2/instance-types/ https://aws.amazon.com/ec2/instance-types/ includes a note that provides some additional information about when vCPUs utilize Intel HT Technology: Each vCPU is a hyperthread of an Intel Xeon core except for T2 and m3.medium. (emphasis added) https://aws.amazon.com/ec2/virtualcores/ https://aws.amazon.com/ec2/virtualcores/ provides a table of the number of cores allocated per instance. Notice that an instance like m5.large that has 2 vCPUs and 1 core. An instance like t2.small has 1 vCPU and 1 core. Disclosure: I work for AWS
  - h4b4n3r0 8y ago
    Um, no. If I schedule a micro instance in a particular zone and nothing else, for the duration of my timeslice the VM will have to monopolize the core. Optimal use of vCPUs would demand that two VM threads get scheduled to the core to take advantage of HT, which Google says it won’t do. Timeslicing doesn’t solve this problem. At least that’s my reading of what they said.
    - _msw_ 8y ago
      Re-read what I wrote above. It covered both simultaneous and sequential sharing. Here is the part you are concerned about: instances do not simultaneously share cores with other customer instances via SMT.
      - h4b4n3r0 8y ago
        Re-read what I wrote. Imagine a machine with 8 cores and 16 hyperthreads. Now imagine you need to optimally use the execution ports in each core. To do that you would need to have 16 VM threads all feeding the ports simultaneously, 2 per core. If all your workloads on that machine are from different customers, you can’t do more than 8. If you timeslice more than 8, then the question is, what is it that you’re selling as a vCPU? If it’s a timesliced core, then it’s not what they say they’re selling (a hyperthread).
        cthalupa 8y ago
        You: > If it’s a timesliced core, then it’s not what they say they’re selling (a hyperthread). vs. _msw_: > A core can be sequentially time sliced between customer instances when you use a fractional (m3.medium) or burst (T2 instances) CPU instance type. AWS instance types page ( https://aws.amazon.com/ec2/instance-types/ https://aws.amazon.com/ec2/instance-types/ ): > Each vCPU is a hyperthread of an Intel Xeon core except for T2 and m3.medium.
        h4b4n3r0 8y ago
garethl 8y ago
It does say, "All EC2 host infrastructure has been updated with these new protections, and no customer action is required at the infrastructure level." "Meanwhile, we suggest using the stronger security and isolation properties of EC2 instances to separate any untrusted workloads." As I read it, it is talking about running code within your instance - if you have untrusted workloads, rather run them in a separate instance, so as not to encounter issues like this cross-process.
markdoubleyou 8y ago
Amazon offers "dedicated hosts" and "dedicated instances", which you can specify in the "tenancy" field when launching an EC2 instance. I bet that's the stronger isolation that they're referring to. Costs more, though. https://aws.amazon.com/ec2/dedicated-hosts/ https://aws.amazon.com/ec2/dedicated-hosts/
- _msw_ 8y ago
  This guidance is primarily for customers that are running untrusted code within their EC2 instances. The "strong isolation of EC2 instances" refers to the properties of isolation provided by EC2's virtualization compared to processes within an operating system. It is challenging to safely and securely run untrusted code within sandboxes and processes using general purpose software. However, the hypervisor and hardware based virtualization of EC2 instances is engineered to provide isolation between mutually untrusted instances. There are several reasons that customers may want to use dedicated instances or dedicated hosts, so we provide those tenancy options as well. The most common reason customers use dedicated hosts so that they can bring their own software licenses, which are often tied to a physical host. Disclosure: I work for AWS.
deathanatos 8y ago
> Given the hypervisor on EC2 is opaque to me, I'm not sure how I'm supposed to avoid co-tenanting in a risky fashion. In Linux, at least, the Xen hypervisor on EC2 exposes some information about itself at /sys/hypervisor. In particular, I think /sys/hypervisor/uuid would allow you to detect co-tenanting (between two VMs of your own). Not saying that I think you should do that, or that'd I'd want to — it'd be a PITA to coordinate amongst VMs, and I'm not sure it would matter (what if you're co-tentant w/ a malicious VM? even if you detect it, how do you get out of it?). That is, inside the VM seems like wholly the wrong place to attempt to deal with this. But I don't think many people realize /sys/hypervisor exists.