5 ms·

As predicted Spectre/Meltdown were just the first fruit of a major new wave of investigations into these avenues of side-channel attacks in CPUs. Now we're seei

by xoa 8y ago

As predicted Spectre/Meltdown were just the first fruit of a major new wave of investigations into these avenues of side-channel attacks in CPUs. Now we're seeing more as it gets more attention, and it's interesting stuff. That said I think the researchers might go too far here: >"This is the main reason we released the exploit -- to show how reproducible it is," Brumley told us, "and help to kill off the SMT trend in chips." >"Security and SMT are mutually exclusive concepts," he added. "I hope our work encourages users to disable SMT in the BIOS or choose to spend their money on architectures not featuring SMT." I don't blame them for being security focused about all else at any cost and any layer, that's their gig. But I think the real response here is likely to be a lot more subtle and interesting. Of course perhaps SMT can in fact be fixed for this without a wholesale tossing in which case it'll just be a universal hardware revision somewhere down the line. This increasing level of public research and awareness of this specific class is still relatively early days after all. But taking it as a given for argument that there really is a fundamental conflict, the fact would remain that SMT can provide significant performance gains, and furthermore that we're still far from the point where SaaS/IaaS is everywhere. Lots of systems are still under single user local control, and in turn attackers being able to co-run their own arbitrary code on the same physical core isn't necessarily part of the threat model at all (and more specifically if attackers get that far least common denominator kicks in, they've already owned what's important). Even if it's desirable to run some risky code as well, hard core affinity for non-secure processes is a brute force solution in a local system context that seems like it shouldn't be a big deal given the a surfeit of physical/logical cores for many work loads. But perhaps this could be a leading edge of true processor level physical differentiation required between IaaS and more traditional deployments, and that might make for an interesting change to the competitive landscape there. It'd change the cost/benefit in some scenarios, or require more custom processor work to enact harder (and performance costing) boundaries that a traditional setup might take care of with machine isolation. I wonder if that could shift things back away from the Cloud and centralization trend in some instances, or at least create and more dynamic market?

mnw21cam 8y ago
The main problem with the view that a single-user system is safe from this is the point that even single-user systems run untrusted code. For example, your web browser probably by default runs Javascript, NaCl, or WebAsm code, which is supposedly sandboxed sufficiently to no longer be a threat to the other stuff running on your machine. This sort of exploit (which was kind of obviously going to happen at some point) breaks this sandbox to some extent.
- xoa 8y ago
  >For example, your web browser probably by default runs Javascript, NaCl, or WebAsm code, which is supposedly sandboxed sufficiently to no longer be a threat to the other stuff running on your machine. I think that would fall under "Even if it's desirable to run some risky code as well" and using hard core affinity wouldn't it? For performance oriented use cases 8-core and higher systems are not exactly super rare at this point, and AMD at least has been pushing the physical core counts pretty hard even on non-super high end chips. In a single owner/trusted dedicated metal case it's a lot easier to just say "core 1 is for running web code and that's it, if every core is needed for something else the web stuff will all be evicted and slept until that's over" which on a percentage basis doesn't incur that big a hit in a high core system. It's not like a IaaS system that's devoted to running dozens/hundreds of commercial client VMs and needs to be able to perform dynamic allocation with no manual intervention and untrusted code is the rule not the exception. IaaS is all about scaling, and that's a hard and interesting problem. But sometimes for individual instances it works fine to just pick a completely non-scalable solution that gets the job done and brute force it. I'm just thinking that it's worth remembering that most security is an economic equation, and sometimes performance is in fact worth it, or the security issue can be soft mitigated sufficiently in a restricted case. "Only trusted signed code may use SMT" for example, that won't work everywhere and it doesn't mitigate SMT still being a dangerous feature, but it may be plenty good enough in some instances. Or perhaps there is a server where the only truly sensitive piece is the private key, and that simply gets offloaded into an HSM blackbox and thus isn't available to be compromised anyway. Or the auth/signing servers disable it but other boxes do not. Basically, it just seems like it's too valuable a feature and the threat scenario not universal enough for it to be simply abandoned entirely as the authors suggest given the physical and economic realities we're facing right now with silicon.
bitwize 8y ago
Yeah, I don't think SMT is going away. It's one of the tricks CPU vendors have to eke out more performance, now that they can no longer significantly boost clock rates and shrink die sizes. So we will still have supercomputers and such with this technology enabled, but they will be air-gapped and acoustically isolated, or maybe just have their network perimeters beefed up. And then for mission-critical, sensitive server loads SMT will be disabled (or they will use SMTless CPUs).
mbell 8y ago
Does anyone have a tech brief on why SMT is more dangerous than any other type of shared resource? i.e. There are several resources that are shared between cores on a CPU, is there something inherent about SMT that makes the resources it shares more attackable or is this just a bug?
- throwaway2048 8y ago
  Basically timing attacks are completely unavoidable on SMT because SMT fundamentally depends on what the other thread is doing on the core, if the thread is using few core resources the attack thread gets more, or vice versa. This gives you a very fine grained view into what the other thread is doing timing wise, and is inherent to the concept of SMT (two threads running on a core at once, using resources not utilized by the other). I suspect this issue is completely impossible to resolve in anything except very constrained situations that negate the benefit of SMT entirely. Its different than say, a timing attack on the OS scheduler for two reasons 1. OS scheduling is very coarse (A process gets exclusive use of a core for tens of thousands or hundreds of thousands of cycles), with SMT you get a very fine grained near cycle-by-cycle view of core utilization by the other process. 2. you don't get much of a view into what precisely the other thread is doing such as is it doing lots of integer operations, floating point, memory fetches etc, which can be derived in SMT attacks based on how fast such things occur for the evil monitoring thread.
  - mbell 8y ago
    Is this not a problem with, for example, L3 cache that is shared across cores? Is there something inherent about it that makes timing attacks impossible or is it just that we're seeing people go after lower hanging fruit?
    - throwaway2048 8y ago
      Cache timing attacks are definitely a real thing, but cache can at least be isolated/partitioned (even if its not really currently).
- 8y ago
ams6110 8y ago
The OpenBSD project predicted more was coming after Spectre/Meltdown and disabled SMT by default.
- xoa 8y ago
  Sure, but OpenBSD has a very specific project goal and use cases (which is absolutely as it should be!) and their stance thus makes sense in context. But it doesn't mean their stance is universally applicable either, anymore then best practices for IaaS all apply everywhere. For an alternate feature example: some OpenBSD committers for years have reasonably rejected OpenZFS [1] requests because it was considered to be just plain too big, and in turn presented more threat surface and higher system resource consumption (big issue for embedded/firewall appliance applications) then was desirable for them. All of which is true and makes total sense there. But not for a lot of big servers and end user systems where I think automatic full data consistency/checks/regeneration, easy full replication, trivial cheap atomic snapshots, portability between many OS, and so on, address much more important threats then the extra load or any unknown potential vulns. No need for a monoculture ecosystem here rather then using the right specific tool for the job right? ----- 1: https://www.tedunangst.com/flak/post/ZFS-on-OpenBSD https://www.tedunangst.com/flak/post/ZFS-on-OpenBSD
keldaris 8y ago
His statement is particularly hyperbolic given that no one has shown a working side channel attack on a Ryzen CPU that relied on SMT. The article is littered with repetitions of suspicion, but there hasn't been any relevant code or results published anywhere to my knowledge.
jcranmer 8y ago
> Of course perhaps SMT can in fact be fixed for this without a wholesale tossing in which case it'll just be a universal hardware revision somewhere down the line. There's a very simple solution here. Don't schedule two threads with different trust domains on different SMT threads on the same core. No need to change any hardware, no need to disable anything, just accept (as has been known for at least a decade) that there is very likely to be a side channel attack if you look hard enough when SMT is involved.
- kbenson 8y ago
  At what point does the extra complexity and cost of including SMT in the design outweigh the gains is provides if you can only use it in specific situations, and if you don't err on the side of caution is may be a security problem? If it was clearly outlined as 8 cores that support hyperthreads or 10 cores that don't for around the same price, what do we suppose most people would choose? Part of the reason for the current status quo might be that hyperthreads are a big differentiator beteen AMD and Intel, and play towards Intel's strength (higher clock speeds at fewer cores). We have something close to the proposed scenario starting to play out with AMD's offerings that support more cores and no hyperthreading, but it's not a perfect experiment because AMD's cores are also lower clock speed, and there's a lot of brand name loyalty currently.
  - jcranmer 8y ago
    The point of SMT is that it is quite low overhead. You're not duplicating most of the core, only the physical register file and the TLB units (I think). The execution units and caches--where most of the die of a core goes--remain unreplicated. In terms of area tradeoff, you're not looking at "we'll give you 2 more cores if we disabled SMT." For the Pentium 4 (quickest I could get), adding SMT is a die overhead of 5%. Your realistic trade-off for area constraints is on the terms of SMT, or better out-of-order parallelism scavenging, or more L2 cache.
    - kbenson 8y ago
      > For the Pentium 4 (quickest I could get), adding SMT is a die overhead of 5%. Ah, that's what I was looking for. I'm aware that most the resources are shared, but I also assume there have been design choices in other components to make SMT easier or faster, and possibly that increases their die size a small amount, or in general just complicates their design. The question (which is ultimately unanswerable) is what would we have had Intel not chosen SMT as the path to pursue? If they had instead invested those resource into other areas (e.g. more cores) and never let SMT concerns enter into the discussion, how would those (theoretical) CPUs compare? That's the CPU comparison I was alluding to in the prior comment, and why I noted AMD vs Intel (even with all it's other differences) may be the closest match up of that idea we'll see. It wasn't meant as a rebuttal to anything, it was more a "wondering out loud" type of comment spurred by yours, about what could have been had different paths been taken.

by xoa 8y ago

mnw21cam 8y ago

The main problem with the view that a single-user system is safe from this is the point that even single-user systems run untrusted code. For example, your web browser probably by default runs Javascript, NaCl, or WebAsm code, which is supposedly sandboxed sufficiently to no longer be a threat to the other stuff running on your machine. This sort of exploit (which was kind of obviously going to happen at some point) breaks this sandbox to some extent.

xoa 8y ago
>For example, your web browser probably by default runs Javascript, NaCl, or WebAsm code, which is supposedly sandboxed sufficiently to no longer be a threat to the other stuff running on your machine. I think that would fall under "Even if it's desirable to run some risky code as well" and using hard core affinity wouldn't it? For performance oriented use cases 8-core and higher systems are not exactly super rare at this point, and AMD at least has been pushing the physical core counts pretty hard even on non-super high end chips. In a single owner/trusted dedicated metal case it's a lot easier to just say "core 1 is for running web code and that's it, if every core is needed for something else the web stuff will all be evicted and slept until that's over" which on a percentage basis doesn't incur that big a hit in a high core system. It's not like a IaaS system that's devoted to running dozens/hundreds of commercial client VMs and needs to be able to perform dynamic allocation with no manual intervention and untrusted code is the rule not the exception. IaaS is all about scaling, and that's a hard and interesting problem. But sometimes for individual instances it works fine to just pick a completely non-scalable solution that gets the job done and brute force it. I'm just thinking that it's worth remembering that most security is an economic equation, and sometimes performance is in fact worth it, or the security issue can be soft mitigated sufficiently in a restricted case. "Only trusted signed code may use SMT" for example, that won't work everywhere and it doesn't mitigate SMT still being a dangerous feature, but it may be plenty good enough in some instances. Or perhaps there is a server where the only truly sensitive piece is the private key, and that simply gets offloaded into an HSM blackbox and thus isn't available to be compromised anyway. Or the auth/signing servers disable it but other boxes do not. Basically, it just seems like it's too valuable a feature and the threat scenario not universal enough for it to be simply abandoned entirely as the authors suggest given the physical and economic realities we're facing right now with silicon.

bitwize 8y ago

Yeah, I don't think SMT is going away. It's one of the tricks CPU vendors have to eke out more performance, now that they can no longer significantly boost clock rates and shrink die sizes. So we will still have supercomputers and such with this technology enabled, but they will be air-gapped and acoustically isolated, or maybe just have their network perimeters beefed up. And then for mission-critical, sensitive server loads SMT will be disabled (or they will use SMTless CPUs).

mbell 8y ago

Does anyone have a tech brief on why SMT is more dangerous than any other type of shared resource? i.e. There are several resources that are shared between cores on a CPU, is there something inherent about SMT that makes the resources it shares more attackable or is this just a bug?

throwaway2048 8y ago
Basically timing attacks are completely unavoidable on SMT because SMT fundamentally depends on what the other thread is doing on the core, if the thread is using few core resources the attack thread gets more, or vice versa. This gives you a very fine grained view into what the other thread is doing timing wise, and is inherent to the concept of SMT (two threads running on a core at once, using resources not utilized by the other). I suspect this issue is completely impossible to resolve in anything except very constrained situations that negate the benefit of SMT entirely. Its different than say, a timing attack on the OS scheduler for two reasons 1. OS scheduling is very coarse (A process gets exclusive use of a core for tens of thousands or hundreds of thousands of cycles), with SMT you get a very fine grained near cycle-by-cycle view of core utilization by the other process. 2. you don't get much of a view into what precisely the other thread is doing such as is it doing lots of integer operations, floating point, memory fetches etc, which can be derived in SMT attacks based on how fast such things occur for the evil monitoring thread.
- mbell 8y ago
  Is this not a problem with, for example, L3 cache that is shared across cores? Is there something inherent about it that makes timing attacks impossible or is it just that we're seeing people go after lower hanging fruit?
  - throwaway2048 8y ago
    Cache timing attacks are definitely a real thing, but cache can at least be isolated/partitioned (even if its not really currently).
8y ago

ams6110 8y ago

The OpenBSD project predicted more was coming after Spectre/Meltdown and disabled SMT by default.

xoa 8y ago
Sure, but OpenBSD has a very specific project goal and use cases (which is absolutely as it should be!) and their stance thus makes sense in context. But it doesn't mean their stance is universally applicable either, anymore then best practices for IaaS all apply everywhere. For an alternate feature example: some OpenBSD committers for years have reasonably rejected OpenZFS [1] requests because it was considered to be just plain too big, and in turn presented more threat surface and higher system resource consumption (big issue for embedded/firewall appliance applications) then was desirable for them. All of which is true and makes total sense there. But not for a lot of big servers and end user systems where I think automatic full data consistency/checks/regeneration, easy full replication, trivial cheap atomic snapshots, portability between many OS, and so on, address much more important threats then the extra load or any unknown potential vulns. No need for a monoculture ecosystem here rather then using the right specific tool for the job right? ----- 1: https://www.tedunangst.com/flak/post/ZFS-on-OpenBSD https://www.tedunangst.com/flak/post/ZFS-on-OpenBSD

keldaris 8y ago

His statement is particularly hyperbolic given that no one has shown a working side channel attack on a Ryzen CPU that relied on SMT. The article is littered with repetitions of suspicion, but there hasn't been any relevant code or results published anywhere to my knowledge.

jcranmer 8y ago

> Of course perhaps SMT can in fact be fixed for this without a wholesale tossing in which case it'll just be a universal hardware revision somewhere down the line. There's a very simple solution here. Don't schedule two threads with different trust domains on different SMT threads on the same core. No need to change any hardware, no need to disable anything, just accept (as has been known for at least a decade) that there is very likely to be a side channel attack if you look hard enough when SMT is involved.

kbenson 8y ago
At what point does the extra complexity and cost of including SMT in the design outweigh the gains is provides if you can only use it in specific situations, and if you don't err on the side of caution is may be a security problem? If it was clearly outlined as 8 cores that support hyperthreads or 10 cores that don't for around the same price, what do we suppose most people would choose? Part of the reason for the current status quo might be that hyperthreads are a big differentiator beteen AMD and Intel, and play towards Intel's strength (higher clock speeds at fewer cores). We have something close to the proposed scenario starting to play out with AMD's offerings that support more cores and no hyperthreading, but it's not a perfect experiment because AMD's cores are also lower clock speed, and there's a lot of brand name loyalty currently.
- jcranmer 8y ago
  The point of SMT is that it is quite low overhead. You're not duplicating most of the core, only the physical register file and the TLB units (I think). The execution units and caches--where most of the die of a core goes--remain unreplicated. In terms of area tradeoff, you're not looking at "we'll give you 2 more cores if we disabled SMT." For the Pentium 4 (quickest I could get), adding SMT is a die overhead of 5%. Your realistic trade-off for area constraints is on the terms of SMT, or better out-of-order parallelism scavenging, or more L2 cache.
  - kbenson 8y ago
    > For the Pentium 4 (quickest I could get), adding SMT is a die overhead of 5%. Ah, that's what I was looking for. I'm aware that most the resources are shared, but I also assume there have been design choices in other components to make SMT easier or faster, and possibly that increases their die size a small amount, or in general just complicates their design. The question (which is ultimately unanswerable) is what would we have had Intel not chosen SMT as the path to pursue? If they had instead invested those resource into other areas (e.g. more cores) and never let SMT concerns enter into the discussion, how would those (theoretical) CPUs compare? That's the CPU comparison I was alluding to in the prior comment, and why I noted AMD vs Intel (even with all it's other differences) may be the closest match up of that idea we'll see. It wasn't meant as a rebuttal to anything, it was more a "wondering out loud" type of comment spurred by yours, about what could have been had different paths been taken.