6 ms·

Perhaps someone might know better than me: why is hyperthreading necessarily bad? Can't you just keep it on and give your tenants cores with affinity? For exa

Perhaps someone might know better than me: why is hyperthreading necessarily bad? Can't you just keep it on and give your tenants cores with affinity? For example, some tenant wants two cores, you give them two vcores on the same core; some tenant wants four cores, you give them two vcores over two, etc.

gamache 7y ago
It's not necessarily bad, but it is not secure by default. That's why it ships turned off.
messe 7y ago
That's mostly fine, but it complicates the scheduler and doesn't necessarily help performance. IIRC hyperthreading performs best when the workload on each thread is different (not taking caches into account), so running threads from the same process on the same core can (although isn't always; I'd hesitate to claim anything concrete here without benchmarks) be detrimental. OpenBSD devs are likely open to it, but there are other inefficiencies in the kernel like locking that have priority.
throwaway2048 7y ago
Because unless you make hyperthreading a purely opt in thing, there is no way to know as a kernel if two things that are security sensitive relative to each other are being run on the same core as Hyperthreading neighbors. Two threads in a web browser that are assigned to different websites for instance.
- rrss 7y ago
  > there is no way to know as a kernel if two things that are security sensitive relative to each other The kernel has a way, and it is process isolation. The kernel doesn't care if you want two threads in the same process to be isolated from one another - that's your problem, not the kernel's. Anyway, thread isolation already doesn't work even without hyperthreading-specific attacks: "we have discovered that untrusted code can construct a universal read gadget to read all memory in the same address space through side-channels. In the face of this reality, we have shifted the security model of the Chrome web browser and V8 to process isolation" (from https://arxiv.org/pdf/1902.05178v1.pdf https://arxiv.org/pdf/1902.05178v1.pdf).
  - throwaway2048 7y ago
    Chrome doesn't exclusively use process isolation, several sites will share the same chrome process. This is only one example, another might be for instance, authenticating users on one thread, and sensitive authenticated data on another, which is an extremely common pattern.
    - rrss 7y ago
      Chrome turned on site isolation by default in chrome 67: https://security.googleblog.com/2018/07/mitigating-spectre-with-site-isolation.html https://security.googleblog.com/2018/07/mitigating-spectre-w... My point is that the OS has no responsibility to isolate threads from one another, regardless of how many applications may try to do it themselves anyway.
dnautics 7y ago
actually I just took a shower and answered my question (I think) - many of the hyperthreading bugs don't breach the process divide, they use errors in hyperthreading statefulness to use a side channel to breach memory divide, and the cores share memory, whether or not who's on what core, if any one core gets compromised, you could potentially access any of the core's memory.
- muricula 7y ago
  Close but not quite -- sibling hyperthreads (logical cores) share cache state. Physical cores do not share cache state. Different processes, threads, or VMs on sibling hyperthreads (by definition on the same physical core) can infer the other's memory state based on the cache state. If an attacker is pinned to one hyperthread, and the victim is pinned to another which isn't a sibling hyperthread, none of the spectre attacks will work since the cache state isn't shared. As an attacker with code exec on a core, you can theoretically play games with the OS scheduler until you're running on a sibling core with your victim thread/process/vm.
  - dnautics 7y ago
    That's not true. Spectre works due to speculative execution leaking memory data through a side channel exposed by hyperthreading. That memory can be in use by any of the threads, not just the ones on sibling threads.
    - muricula 7y ago
      The side channel for most of the spectre variants is the latency of misses on the cache lines. L1 and L2 cache lines are local to a physical core. As far as I know, nobody has made any of the spectre variants work by measuring the latency of L3 cache misses, which are local to NUMA nodes if I understand correctly, but I'd love to hear otherwise. The most recent round of spectre variants measured the latency of the line fill buffers and other parts which are local to a physical core's memory subsystem.
MPSimmons 7y ago
You have to be absolutely sure that your tenants are not executing arbitrary code (i.e. javascript) because then they can get compromised, too.
deleted 7y ago
[deleted]

gamache 7y ago

It's not necessarily bad, but it is not secure by default. That's why it ships turned off.

messe 7y ago

That's mostly fine, but it complicates the scheduler and doesn't necessarily help performance. IIRC hyperthreading performs best when the workload on each thread is different (not taking caches into account), so running threads from the same process on the same core can (although isn't always; I'd hesitate to claim anything concrete here without benchmarks) be detrimental. OpenBSD devs are likely open to it, but there are other inefficiencies in the kernel like locking that have priority.

throwaway2048 7y ago

Because unless you make hyperthreading a purely opt in thing, there is no way to know as a kernel if two things that are security sensitive relative to each other are being run on the same core as Hyperthreading neighbors. Two threads in a web browser that are assigned to different websites for instance.

rrss 7y ago
> there is no way to know as a kernel if two things that are security sensitive relative to each other The kernel has a way, and it is process isolation. The kernel doesn't care if you want two threads in the same process to be isolated from one another - that's your problem, not the kernel's. Anyway, thread isolation already doesn't work even without hyperthreading-specific attacks: "we have discovered that untrusted code can construct a universal read gadget to read all memory in the same address space through side-channels. In the face of this reality, we have shifted the security model of the Chrome web browser and V8 to process isolation" (from https://arxiv.org/pdf/1902.05178v1.pdf https://arxiv.org/pdf/1902.05178v1.pdf).
- throwaway2048 7y ago
  Chrome doesn't exclusively use process isolation, several sites will share the same chrome process. This is only one example, another might be for instance, authenticating users on one thread, and sensitive authenticated data on another, which is an extremely common pattern.
  - rrss 7y ago
    Chrome turned on site isolation by default in chrome 67: https://security.googleblog.com/2018/07/mitigating-spectre-with-site-isolation.html https://security.googleblog.com/2018/07/mitigating-spectre-w... My point is that the OS has no responsibility to isolate threads from one another, regardless of how many applications may try to do it themselves anyway.

dnautics 7y ago

actually I just took a shower and answered my question (I think) - many of the hyperthreading bugs don't breach the process divide, they use errors in hyperthreading statefulness to use a side channel to breach memory divide, and the cores share memory, whether or not who's on what core, if any one core gets compromised, you could potentially access any of the core's memory.

muricula 7y ago
Close but not quite -- sibling hyperthreads (logical cores) share cache state. Physical cores do not share cache state. Different processes, threads, or VMs on sibling hyperthreads (by definition on the same physical core) can infer the other's memory state based on the cache state. If an attacker is pinned to one hyperthread, and the victim is pinned to another which isn't a sibling hyperthread, none of the spectre attacks will work since the cache state isn't shared. As an attacker with code exec on a core, you can theoretically play games with the OS scheduler until you're running on a sibling core with your victim thread/process/vm.
- dnautics 7y ago
  That's not true. Spectre works due to speculative execution leaking memory data through a side channel exposed by hyperthreading. That memory can be in use by any of the threads, not just the ones on sibling threads.
  - muricula 7y ago
    The side channel for most of the spectre variants is the latency of misses on the cache lines. L1 and L2 cache lines are local to a physical core. As far as I know, nobody has made any of the spectre variants work by measuring the latency of L3 cache misses, which are local to NUMA nodes if I understand correctly, but I'd love to hear otherwise. The most recent round of spectre variants measured the latency of the line fill buffers and other parts which are local to a physical core's memory subsystem.

MPSimmons 7y ago

You have to be absolutely sure that your tenants are not executing arbitrary code (i.e. javascript) because then they can get compromised, too.

deleted 7y ago

[deleted]