10 ms·

Spectre is here to stay: An analysis of side-channels and speculative execution

169 points by matt_d 8y ago

phkahler 8y ago
>> These attacks leak information through micro-architectural side-channels which we show are not mere bugs, but in fact lie at the foundation of optimization. So we'll need to have non-speculative execution for cloud CPUs and stronger efforts to keep untrusted code off our high performance CPUs. This may even lead to chips with performance cores and trusted cores.
- bediger4000 8y ago
  Processor affinity would become a security-influenced decision with a CPU like that. That would allow for a whole new class of bugs.
  - phkahler 8y ago
    >> Processor affinity would become a security-influenced decision with a CPU like that. Yes it would. >> That would allow for a whole new class of bugs. I think it's necessary, but not easy. Actually what I think is necessary is for people to stop running code from random places - or even common places. Google could work without running stuff on my machine.
- bunderbunder 8y ago
  I'm curious if this leads to us re-visiting more Itanium-like CPU designs.
  - Tuna-Fish 8y ago
    No, it won't, because OoO with process isolation is still superior to Itanium-like in all respects.
- yardstick 8y ago
  I hope they still offer CPUs optimised for the non Cloud scenario, where all the programs running are trusted and so these sorts of attacks (which requires local access) are not applicable.
  - msbarnett 8y ago
    Outside of some very niche scenarios, does this usecase even exist? Certainly nothing running a javascript enabled browser, electron app, or in general any VM of any sort qualifies.
    - AnIdiotOnTheNet 8y ago
      All the VMs run on my employer's servers are running code we trust. None of them run arbitrary code from some third party, because we're not a cloud provider, nor are they used to browse the web. I don't want to slow them down to mitigate vulnerabilities that just aren't a serious risk or even applicable.
      - pjmlp 8y ago
        Do you guys audit the whole stack then?
    - berkut 8y ago
      Most HPC stuff likely (in my experience) fits this scenario.
      - msbarnett 8y ago
        Sure. I’m comfortable describing HPC and on-premises fully trusted computing (another response) as “very niche scenarios” though (compared to the much much much larger markets of large cloud farms on the server side, and consumer devices), to the point where I have to wonder whether or not it’s worth it for CPU vendors to cater specific SKUs to them without the silicon mitigations.
    - jcelerier 8y ago
- _jal 8y ago
  For some types of work, it means there's still a case for on-prem/"private cloud". Pick two: performance, safety, convenience.
  - djsumdog 8y ago
    The paper states: confidentiality, integrity, availability
    - _jal 8y ago
      ...when discussing something different than what I was.
- Tuna-Fish 8y ago
  No. The paper notes that Spectre can, and will in the future be able to defeat all programming language level techniques of isolation. With properly designed OoO, Spectre cannot defeat process isolation. The fundamental lesson that everyone must take to heart is that in the future, any code running on a system, including things like very high-level languages running on interpreters, always have full read access to the address space of the process they run in. Process isolation is the most granular security boundary that actually works. Or in other words, running javascript interpreters in the same address space as where you manage crypto is not something that can be done. Running code from two different privilege levels in the same VM is not something that can be done. Whenever you need to run untrusted code, you need to spin up a new OS-managed process for it.
  - downer52 8y ago
    Except there was a time before this class of exploitation. And so, now, there cannot be a time after? Doubtful.
  - elihu 8y ago
    What about languages that prevent the program from reading the current time or determining whether some local computation finished before or after some external event? (I'm thinking of something like Haskell code that's executing outside of the IO monad.) Perhaps we just need to have a more restricted idea about what untrusted code is allowed to do.
    - gizmo686 8y ago
      In concept this could work. In practice, you bassicly need to design the language (or at least compiler and runtime) with this in mind from the beggining, otherwise you might forget about some backdoor. Eg, if you do Haskell and just verify the function you are running is not in the IO monad, you might miss some usage of UnsafePerformIO. Even if you check their code, if you let them specify dependencies they might manages to sneak a buggy use of UnsafePerformIO into a library they submitted to hackage. Plus, your restriction is essentially: no clock, no contact with the outside world, no threading, and a carefully considered interface to the host program to prevent time leaks. For many usecases, this is not workable
jnordwick 8y ago
Has there been an in-the-wild exploit found for many of these micro-arch side channel attacks? I still have reservations that Specter can be actually exploited, and certainly not by JavaScript running in a browser it seems (even without the timing fixes -- just too much noise in the system to get a real world exploit). All the proof of concepts I saw effectively needed a running start (and a much lower bar to clear). I'm still not sure if we are just making too much out of many of these attacks. Also, many of these exploits require running native code to really even be possible, and a JIT offers a large degress of protection. Most of the machines I use, if you are already able to run native code, the battle has already been lost. I think the security community can sometimes have Chicken Little syndrome. I most definitely wouldn't want performance enhancing techniques that might have side-channel vulnerabilities to not be produced or used because they might be exploitable in a use case (eg, web serving and rendering) that doesn't fit most others (eg, internal servers).
- dwheeler 8y ago
  > I still have reservations that Specter can be actually exploited It's not easy, and many systems are more easily broken into by other means (e.g., unpatched known-vulnerable programs). But people are already demonstrating exploits; it's not theoretical. > and certainly not by JavaScript running in a browser it seems (even without the timing fixes -- just too much noise in the system to get a real world exploit). Your faith is unwarranted. The paper itself evaluated JavaScript as implemented by v8, which is used by Chrome: "as part of our defensive work, we implemented a number of the described mitigations in the V8 JavaScript virtual machine and evaluated their performance penalties. As we’ve noted, none of these mitigations provide comprehensive protection against Spectre, and so the mitigation space is a frustrating performance / protection trade-off." Also, modern statistics is pretty amazing at removing noise. Don't make claims about "too much noise" until you've tried them. > Most of the machines I use, if you are already able to run native code, the battle has already been lost. Perhaps, but many others are in a different circumstance. We've moved to a world where people run on clouds (shared machines with your enemy) and automatically run JavaScript provided by malicious parties. This isn't hopeless. You can run on clouds but run on temporarily-isolated CPUs so there's no leak. A lot of applications can be easily written to work without JavaScript (though many can't, and many developers have no idea that this might be desirable). For many, providing confidentiality when the CPUs always leak is a real problem. > I think the security community can sometimes have Chicken Little syndrome. That's certainly true. These are not easy to exploit today, especially given some mitigations. But think of these as warning shots - these are problems for which there is NO practical full mitigation today. It would be unwise to ignore the problem. Attackers are likely to gradually get better at exploiting this over time, and if that happens, the only solution available is to NOT share CPUs running programs (disable JavaScript and use more-expensive isolated-CPU clouds) OR replace most CPUs with CPUs that do not yet exist. > I most definitely wouldn't want performance enhancing techniques that might have side-channel vulnerabilities to not be produced or used because they might be exploitable in a use case (eg, web serving and rendering) that doesn't fit most others (eg, internal servers). Understandable, and that may be just fine for your situation. However, most systems try to use least privilege to reduce damage (e.g., if you break into one component, you don't get everything). This attack makes it easier to breach such isolation - so even in that case, you might have reason to be concerned. The sky is not falling, at least not today. But there are reasons to be concerned.
openasocket 8y ago
Side-channel attacks are some of the most terrifying to me, because fundamentally your code works fine, it's just some detail in the timing that gives away information. At least with buffer overflows there are automated ways to try and find them, like fuzz testing. It's not perfect, but at least it's something to try and validate you are doing the right thing. Is there some sort of fuzz testing that could potentially find side-channel timing attacks? Maybe some sort of statistical analysis on the timing results? I found this https://arxiv.org/pdf/1811.07005.pdf https://arxiv.org/pdf/1811.07005.pdf but it seems to be fairly recent, and I don't know how mature this area is.
- Strilanc 8y ago
  One of the interesting aspects of quantum computing is that it upgrades these issues into actual bugs. Bit flips are errors in the Z basis, and phase flips (information leaks) are errors in the X basis. [1][2] Trying to build large scale quantum computers could lead to styles of optimization that provably don't have side channel issues. [1]: https://www.scottaaronson.com/blog/?p=3327 https://www.scottaaronson.com/blog/?p=3327 [2]: https://www.youtube.com/watch?v=uPw9nkJAwDY https://www.youtube.com/watch?v=uPw9nkJAwDY
rolph 8y ago
I found some good reads here if anyone would like to review and get a good perspective: https://www.realworldtech.com/sandy-bridge/10/ https://www.realworldtech.com/sandy-bridge/10/ https://en.wikipedia.org/wiki/Out-of-order_execution https://en.wikipedia.org/wiki/Out-of-order_execution https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html https://googleprojectzero.blogspot.com/2018/01/reading-privi... https://bugs.chromium.org/p/project-zero/issues/detail?id=1272 https://bugs.chromium.org/p/project-zero/issues/detail?id=12...
rickmode 8y ago
How about CPUs without speculative execution and simultaneous multithreading (SMT / Hyper-Threading, which has similar issues)? We would, of course, need other optimizations to claw back the performance loss--an engineering problem I feel we can solve. I've wondered if the solution is more, simpler cores. We concentrate on smaller, faster cores, and the programming to utilize them better. Perhaps advances in memory architectures as well. Hardware isn't my specialty, so I'm just brainstorming here. Perhaps this is where ARM and even RISC-V based systems can step in. But I'm a software guy, so what do I know? I just know I'd feel more comfortable with systems based on simpler CPUs that just cannot be exploited by the recent side-channel attacks discovered, rather than trying playing whack-a-mole with patches, along with trying to reason when it might be safe to use CPUs with these optimizations.
- deleted 8y ago
  [deleted]
- DSingularity 8y ago
  It is a natural alternative. The "simpler but more cores" project works on paper (i.e. potential instruction throughput). In reality it falls apart for variety of reasons. The most fundamental is because of the difficulty of exploiting thread-level parallelism. The complex Out-of-Order cores do a really good job of improving throughput by finding independent instructions to execute in parallel. The path-to-parallelism is much easier at the granularity of instructions than at the granularity of cores. Parallel programming is hard. Amadahls law cannot be avoided except through ----- speculation, so we are back to complexity again.
  - rickmode 8y ago
    Amdahl's law for sure. We all know most software sucks, so in this vision (simpler cores), that reduces us to taking the performance hit and optimizing elsewhere. For example, older ARM-based systems do not use speculative execution, and can handle encryption and video transcoding though co-processors.
  - pcwalton 8y ago
    > In reality it falls apart for variety of reasons. On the contrary, the "many wimpy cores" approach has been very successful on GPUs (and other vector processors such as TPUs, etc.) What it hasn't been successful at is running existing software unmodified. The best solution (the one we use today, in fact) seems to be a hybrid approach. We have a few powerful cores to run sequential code and many weaker units (often vector units; e.g. GPUs and SIMD) to run parallel code. Not all algorithms can be parallelized, so we'll likely always need a few fast sequential cores. But a lot of code can.

169 points by matt_d 8y ago

>> These attacks leak information through micro-architectural side-channels which we show are not mere bugs, but in fact lie at the foundation of optimization. So we'll need to have non-speculative execution for cloud CPUs and stronger efforts to keep untrusted code off our high performance CPUs. This may even lead to chips with performance cores and trusted cores.

bediger4000 8y ago
Processor affinity would become a security-influenced decision with a CPU like that. That would allow for a whole new class of bugs.
- phkahler 8y ago
  >> Processor affinity would become a security-influenced decision with a CPU like that. Yes it would. >> That would allow for a whole new class of bugs. I think it's necessary, but not easy. Actually what I think is necessary is for people to stop running code from random places - or even common places. Google could work without running stuff on my machine.
bunderbunder 8y ago
I'm curious if this leads to us re-visiting more Itanium-like CPU designs.
- Tuna-Fish 8y ago
  No, it won't, because OoO with process isolation is still superior to Itanium-like in all respects.
yardstick 8y ago
I hope they still offer CPUs optimised for the non Cloud scenario, where all the programs running are trusted and so these sorts of attacks (which requires local access) are not applicable.
- msbarnett 8y ago
  Outside of some very niche scenarios, does this usecase even exist? Certainly nothing running a javascript enabled browser, electron app, or in general any VM of any sort qualifies.
  - AnIdiotOnTheNet 8y ago
    All the VMs run on my employer's servers are running code we trust. None of them run arbitrary code from some third party, because we're not a cloud provider, nor are they used to browse the web. I don't want to slow them down to mitigate vulnerabilities that just aren't a serious risk or even applicable.
    - pjmlp 8y ago
      Do you guys audit the whole stack then?
  - berkut 8y ago
    Most HPC stuff likely (in my experience) fits this scenario.
    - msbarnett 8y ago
      Sure. I’m comfortable describing HPC and on-premises fully trusted computing (another response) as “very niche scenarios” though (compared to the much much much larger markets of large cloud farms on the server side, and consumer devices), to the point where I have to wonder whether or not it’s worth it for CPU vendors to cater specific SKUs to them without the silicon mitigations.
  - jcelerier 8y ago
_jal 8y ago
For some types of work, it means there's still a case for on-prem/"private cloud". Pick two: performance, safety, convenience.
- djsumdog 8y ago
  The paper states: confidentiality, integrity, availability
  - _jal 8y ago
    ...when discussing something different than what I was.
Tuna-Fish 8y ago
No. The paper notes that Spectre can, and will in the future be able to defeat all programming language level techniques of isolation. With properly designed OoO, Spectre cannot defeat process isolation. The fundamental lesson that everyone must take to heart is that in the future, any code running on a system, including things like very high-level languages running on interpreters, always have full read access to the address space of the process they run in. Process isolation is the most granular security boundary that actually works. Or in other words, running javascript interpreters in the same address space as where you manage crypto is not something that can be done. Running code from two different privilege levels in the same VM is not something that can be done. Whenever you need to run untrusted code, you need to spin up a new OS-managed process for it.
- downer52 8y ago
  Except there was a time before this class of exploitation. And so, now, there cannot be a time after? Doubtful.
- elihu 8y ago
  What about languages that prevent the program from reading the current time or determining whether some local computation finished before or after some external event? (I'm thinking of something like Haskell code that's executing outside of the IO monad.) Perhaps we just need to have a more restricted idea about what untrusted code is allowed to do.
  - gizmo686 8y ago
    In concept this could work. In practice, you bassicly need to design the language (or at least compiler and runtime) with this in mind from the beggining, otherwise you might forget about some backdoor. Eg, if you do Haskell and just verify the function you are running is not in the IO monad, you might miss some usage of UnsafePerformIO. Even if you check their code, if you let them specify dependencies they might manages to sneak a buggy use of UnsafePerformIO into a library they submitted to hackage. Plus, your restriction is essentially: no clock, no contact with the outside world, no threading, and a carefully considered interface to the host program to prevent time leaks. For many usecases, this is not workable

jnordwick 8y ago

Has there been an in-the-wild exploit found for many of these micro-arch side channel attacks? I still have reservations that Specter can be actually exploited, and certainly not by JavaScript running in a browser it seems (even without the timing fixes -- just too much noise in the system to get a real world exploit). All the proof of concepts I saw effectively needed a running start (and a much lower bar to clear). I'm still not sure if we are just making too much out of many of these attacks. Also, many of these exploits require running native code to really even be possible, and a JIT offers a large degress of protection. Most of the machines I use, if you are already able to run native code, the battle has already been lost. I think the security community can sometimes have Chicken Little syndrome. I most definitely wouldn't want performance enhancing techniques that might have side-channel vulnerabilities to not be produced or used because they might be exploitable in a use case (eg, web serving and rendering) that doesn't fit most others (eg, internal servers).

dwheeler 8y ago
> I still have reservations that Specter can be actually exploited It's not easy, and many systems are more easily broken into by other means (e.g., unpatched known-vulnerable programs). But people are already demonstrating exploits; it's not theoretical. > and certainly not by JavaScript running in a browser it seems (even without the timing fixes -- just too much noise in the system to get a real world exploit). Your faith is unwarranted. The paper itself evaluated JavaScript as implemented by v8, which is used by Chrome: "as part of our defensive work, we implemented a number of the described mitigations in the V8 JavaScript virtual machine and evaluated their performance penalties. As we’ve noted, none of these mitigations provide comprehensive protection against Spectre, and so the mitigation space is a frustrating performance / protection trade-off." Also, modern statistics is pretty amazing at removing noise. Don't make claims about "too much noise" until you've tried them. > Most of the machines I use, if you are already able to run native code, the battle has already been lost. Perhaps, but many others are in a different circumstance. We've moved to a world where people run on clouds (shared machines with your enemy) and automatically run JavaScript provided by malicious parties. This isn't hopeless. You can run on clouds but run on temporarily-isolated CPUs so there's no leak. A lot of applications can be easily written to work without JavaScript (though many can't, and many developers have no idea that this might be desirable). For many, providing confidentiality when the CPUs always leak is a real problem. > I think the security community can sometimes have Chicken Little syndrome. That's certainly true. These are not easy to exploit today, especially given some mitigations. But think of these as warning shots - these are problems for which there is NO practical full mitigation today. It would be unwise to ignore the problem. Attackers are likely to gradually get better at exploiting this over time, and if that happens, the only solution available is to NOT share CPUs running programs (disable JavaScript and use more-expensive isolated-CPU clouds) OR replace most CPUs with CPUs that do not yet exist. > I most definitely wouldn't want performance enhancing techniques that might have side-channel vulnerabilities to not be produced or used because they might be exploitable in a use case (eg, web serving and rendering) that doesn't fit most others (eg, internal servers). Understandable, and that may be just fine for your situation. However, most systems try to use least privilege to reduce damage (e.g., if you break into one component, you don't get everything). This attack makes it easier to breach such isolation - so even in that case, you might have reason to be concerned. The sky is not falling, at least not today. But there are reasons to be concerned.

openasocket 8y ago

Side-channel attacks are some of the most terrifying to me, because fundamentally your code works fine, it's just some detail in the timing that gives away information. At least with buffer overflows there are automated ways to try and find them, like fuzz testing. It's not perfect, but at least it's something to try and validate you are doing the right thing. Is there some sort of fuzz testing that could potentially find side-channel timing attacks? Maybe some sort of statistical analysis on the timing results? I found this https://arxiv.org/pdf/1811.07005.pdf https://arxiv.org/pdf/1811.07005.pdf but it seems to be fairly recent, and I don't know how mature this area is.

Strilanc 8y ago
One of the interesting aspects of quantum computing is that it upgrades these issues into actual bugs. Bit flips are errors in the Z basis, and phase flips (information leaks) are errors in the X basis. [1][2] Trying to build large scale quantum computers could lead to styles of optimization that provably don't have side channel issues. [1]: https://www.scottaaronson.com/blog/?p=3327 https://www.scottaaronson.com/blog/?p=3327 [2]: https://www.youtube.com/watch?v=uPw9nkJAwDY https://www.youtube.com/watch?v=uPw9nkJAwDY

rolph 8y ago

I found some good reads here if anyone would like to review and get a good perspective: https://www.realworldtech.com/sandy-bridge/10/ https://www.realworldtech.com/sandy-bridge/10/ https://en.wikipedia.org/wiki/Out-of-order_execution https://en.wikipedia.org/wiki/Out-of-order_execution https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html https://googleprojectzero.blogspot.com/2018/01/reading-privi... https://bugs.chromium.org/p/project-zero/issues/detail?id=1272 https://bugs.chromium.org/p/project-zero/issues/detail?id=12...

rickmode 8y ago

How about CPUs without speculative execution and simultaneous multithreading (SMT / Hyper-Threading, which has similar issues)? We would, of course, need other optimizations to claw back the performance loss--an engineering problem I feel we can solve. I've wondered if the solution is more, simpler cores. We concentrate on smaller, faster cores, and the programming to utilize them better. Perhaps advances in memory architectures as well. Hardware isn't my specialty, so I'm just brainstorming here. Perhaps this is where ARM and even RISC-V based systems can step in. But I'm a software guy, so what do I know? I just know I'd feel more comfortable with systems based on simpler CPUs that just cannot be exploited by the recent side-channel attacks discovered, rather than trying playing whack-a-mole with patches, along with trying to reason when it might be safe to use CPUs with these optimizations.

deleted 8y ago
[deleted]
DSingularity 8y ago
It is a natural alternative. The "simpler but more cores" project works on paper (i.e. potential instruction throughput). In reality it falls apart for variety of reasons. The most fundamental is because of the difficulty of exploiting thread-level parallelism. The complex Out-of-Order cores do a really good job of improving throughput by finding independent instructions to execute in parallel. The path-to-parallelism is much easier at the granularity of instructions than at the granularity of cores. Parallel programming is hard. Amadahls law cannot be avoided except through ----- speculation, so we are back to complexity again.
- rickmode 8y ago
  Amdahl's law for sure. We all know most software sucks, so in this vision (simpler cores), that reduces us to taking the performance hit and optimizing elsewhere. For example, older ARM-based systems do not use speculative execution, and can handle encryption and video transcoding though co-processors.
- pcwalton 8y ago
  > In reality it falls apart for variety of reasons. On the contrary, the "many wimpy cores" approach has been very successful on GPUs (and other vector processors such as TPUs, etc.) What it hasn't been successful at is running existing software unmodified. The best solution (the one we use today, in fact) seems to be a hybrid approach. We have a few powerful cores to run sequential code and many weaker units (often vector units; e.g. GPUs and SIMD) to run parallel code. Not all algorithms can be parallelized, so we'll likely always need a few fast sequential cores. But a lot of code can.