11 ms·

Completely dropping all forms of speculative execution means dropping overall performance to a tenth of today. There are really hard limits on how fast any oper

Completely dropping all forms of speculative execution means dropping overall performance to a tenth of today. There are really hard limits on how fast any operation, especially memory operations, can be done. The way we have made our CPUs faster is by making them do more operations in parallel, at all levels. At the lowest level, in straight line code, this very often requires speculation to achieve. Speculation is not fundamentally incompatible with security. It's just that literally everyone in the industry never though that leaking information out of speculative context was possible -- and so there is no hardening anywhere. Now that it was proven possible, people are rushing to find all the ways this can be exploited. New CPUs currently being designed will fix all these, and then eventually we will have speculation without security issues. Except for Spectre variant 1. That will always stay with us, because there is no sensible fix for it. The only real solution to that is to accept that branches cannot be used as a security boundary. This is mostly relevant to people implementing secure sandboxes and language runtimes. Going forward, the only reasonable assumption is that if you let a third party run their code in a process, no matter how you verify accesses or otherwise try to contain that code, you should assume it has a read access to the entire process. Any real security requires you to make use of the proper OS-provided isolation.

vbezhenar 8y ago
I wonder why do we have to rely on CPU to parallelize code. Surely compiler could do better job and CPU should just offer transistors without any smart logic. It's not backwards-compatible and I'm aware about Itanium fiasco, but I'm not compelled that it's a wrong way.
- tylerhou 8y ago
  A quote I once heard from a friend: "Don't turn your normal problem into a distributed systems problem."
  - mokus 8y ago
    Arguably, the reason for Spectre, etc., is that people have failed to realize that at the scale of modern intel CPUs, these problems already are distributed systems problems.
- jcranmer 8y ago
  As someone who works in compilers: no, it's not possible to make the compiler do a better job. There's a reason why VLIW architectures keep getting proposed and keep dying. The instruction-level parallelism that a CPU can extract is primarily a dynamic kind of parallelism. You can, say, have a branch that's true 1000 times, then false 1000 times, then true 1000 times, then false 1000 times, etc.--as a compiler, telling the hardware to predict true or false is going to guarantee a 50% hit rate, but the stupid simple dynamic branch predictor will get 99% on that branch.
  - kinghajj 8y ago
    What are your thoughts on the Mill CPU team's claims re: their ISA allowing compilers feasibly to schedule operations statically?
    - Taniwha 8y ago
      you can do this to some degree on most CPU's - moving loads away from their results being used - compilers, esp on RISC machines with lots of registers, do this today VLIW machines allow you to provide hints about instruction level parallelism without all the superscalar on-the-fly analysis of instruction level data dependencies (so the hardware can do all that rescheduling on the fly). I think that if interlock-free software scheduled CPUs allowed us to reach 20GHz clock speeds where complex superscalar machines were stuck at 3GHz we'd all be jumping ship - but they're not
      - AstralStorm 8y ago
        Essentially the CPU is running a JIT branch optimization and speculative loads which cannot ever be matched by static optimization. The interesting part is that any JIT compiler that is so dynamic in performance will have these problems. The solution might be to actually prevent some kind of security important code from being optimized in this way. Say, forcing full cache sync in-order execution for parts of code with no resource sharing between cores.
        8y ago
- Tuna-Fish 8y ago
  The compiler cannot compete with the CPU because the CPU has strictly more information. Any memory access can take between 4 and thousands of cycles of time to execute, and the compiler won't know how long any of them take. The CPU can reorder based on exact knowledge.
  - Symmetry 8y ago
    Using profile guided optimization can help a lot with this in practice, though it isn't perfect.
kentonv 8y ago
> Going forward, the only reasonable assumption is that if you let a third party run their code in a process, no matter how you verify accesses or otherwise try to contain that code, you should assume it has a read access to the entire process. If this is true, it is unbelievably bad for the future of security and computing in general. People throwing around this assertion are, in my opinion, not appreciating how bad it is. We need to try a lot harder before we give up. Here's why: 1. Finer-grained isolation makes security better, because it allows us to apply the Principle of Least Authority to each component of the system, and protect components of a system from bugs in other components. To make meaningful gains in security going forward, we need to encourage more fine-grained isolation. If process-level isolation is the finest grain we'll ever have, we can't make these advances. 2. The scalability of edge computing requires finer-grained isolation than process isolation. The trend is towards pushing more and more code to run directly on devices or edge networks rather than centralized servers. That means that the places where code runs need to be able to handle orders of magnitude more tenants than before -- because everyone wants their code to run in every location. If we can't achieve high multi-tenancy securely -- by which I mean 10,000 or more independently isolated pieces of code running on the same machine -- then the only solution will be to limit these resources to big, trusted players. Small companies will but shut out from "the edge". That's bad. Luckily, the "process isolation is the only isolation" claim is wrong. It's true that we need to evolve our approach to make sub-process isolation secure, but it's not impossible at all. In fact, it's possible to design a sandbox where you can trivially prove that code running inside it cannot observe side channels. Here's how that works: Thank about Haskell, or another purely-functional language. An attacker provides you with a pure function to execute. Because it's a pure function, for some given set of inputs, it will always produce exactly the same output, no matter what machine you run it on, or what else is going on in the background. Therefore, the output cannot possibly incorporate observations from side channels, no matter what the code does internally. So: It is possible to run attacker-provided code without exposing secrets from the process's memory space. The question is, of course, how do we build a useful sandbox that relies on this property. There is a lot of work to be done there. Luckily, we don't really have to use a purely-functional language, we only need to use a deterministic language. It turns out that the world's most popular language, JavaScript, is actually highly deterministic, in large part due to its single-threaded nature. We do need to remove access to timers, or find a way to make them not useful. We also need to prevent attackers from being able to time their code's execution remotely. Basically, we need to think carefully about the I/O surface while paying attention to timing. But sandboxing has always been about thinking carefully about the I/O surface, and we have a lot of control there. We just have a new aspect that we need to account for. I think it's doable.
yuhong 8y ago
I think Spectre is even less severe then Meltdown though.
cmsimike 8y ago
Thank you for your response!
JudasGoat 8y ago
I know the Arm A-53 isn't cutting edge fast, but hardly one tenth. Seeing as the A-53 is unaffected by both Meltdown and Spectre, my assumption was that it didn't use speculative execution.
- baobrien 8y ago
  Basically every CPU with a pipeline -- even simple, in-order cores -- speculate by way of branch prediction. I think with the Cortex A53 the pipeline is short enough and the branch predictor simple enough that you can't build a useful spectre attack. It's also common in simpler in-order cores to speculate a little bit around memory accesses.
  - dnautics 8y ago
    There are architectural choices for pipelined systems you can make that don't specex, but it requires compiler level optimization to take advantage of them, so it's a chicken and egg problem to get adoption
    - ajross 8y ago
      Been there, done that, didn't work. VLIW has been tried a dozen times and failed every time.
      - dnautics 8y ago
        machine learning has been tried a dozen times and failed every time. Other developments catch up, like compiler technology. Not directly related to branch prediction, but: We are much better at polyhedral optimization than the last time people tried VLIW, for example. Besides, the last major VLIW push (itanium) DID have SpecEx AND Branch Prediction, and did NOT have delay slots.
        ajross 8y ago
        Shrug. To paraphrase, your argument for "this can be done" is "THIS time will be different I swear!". What will actually happen, of course, is that everyone will put a ASID/PCID as a tag word into all the relevant caches, they'll stop being probe-able from other contexts, and we'll all keep our deep pipelines and speculation and the cache crisis of 2018 will be just a story we tell our grandkids. Much cheaper than a paradigm shift based on long-since tried and rejected technology.
caf 8y ago
Except for Spectre variant 1. That will always stay with us, because there is no sensible fix for it. Is it not a feasible fix for Spectre-v1 to have loaded cachelines wait in a staging area, and not actually update the cache hierarchy until the load instruction is retired?
- bdonlan 8y ago
  Loading those cache lines might invalidate an exclusively held cache line on another core. When that core tries to write to that cache line, it can observe a slower write.
  - deleted 8y ago
    [deleted]
- _chris_ 8y ago
  Yes, you can buffer up and kill any misspeculated updates. You have to do this for ALL shared structures, not just data caches. Annoying, but not impossible. The scarier part is managing bandwidth contention to shared structures.
- nhaehnle 8y ago
  Is it theoretically possible? Maybe. Is it practical? Almost certainly not. Keep in mind that it's not just the L1 cache you potentially need to worry about, but also all the higher level caches, which are really like remote nodes in a distributed system. Even the last level cache would have to be aware of the current speculation state of all threads in the system, which seems like it would be really expensive. What's perhaps even worse, you wouldn't be able to hit on speculatively loaded cache lines until they have "committed" (because the resulting change in external bandwidth utilization may be observable from another thread), which would probably kill a lot of the benefits of the cache in the first place.
forapurpose 8y ago
> everyone in the industry never though that leaking information out of speculative context was possible Everyone overlooked it, or everyone had a good reason to think it was impossible? The information is there; it's a target that needs to be secured.

vbezhenar 8y ago

I wonder why do we have to rely on CPU to parallelize code. Surely compiler could do better job and CPU should just offer transistors without any smart logic. It's not backwards-compatible and I'm aware about Itanium fiasco, but I'm not compelled that it's a wrong way.

tylerhou 8y ago
A quote I once heard from a friend: "Don't turn your normal problem into a distributed systems problem."
- mokus 8y ago
  Arguably, the reason for Spectre, etc., is that people have failed to realize that at the scale of modern intel CPUs, these problems already are distributed systems problems.
jcranmer 8y ago
As someone who works in compilers: no, it's not possible to make the compiler do a better job. There's a reason why VLIW architectures keep getting proposed and keep dying. The instruction-level parallelism that a CPU can extract is primarily a dynamic kind of parallelism. You can, say, have a branch that's true 1000 times, then false 1000 times, then true 1000 times, then false 1000 times, etc.--as a compiler, telling the hardware to predict true or false is going to guarantee a 50% hit rate, but the stupid simple dynamic branch predictor will get 99% on that branch.
- kinghajj 8y ago
  What are your thoughts on the Mill CPU team's claims re: their ISA allowing compilers feasibly to schedule operations statically?
  - Taniwha 8y ago
    you can do this to some degree on most CPU's - moving loads away from their results being used - compilers, esp on RISC machines with lots of registers, do this today VLIW machines allow you to provide hints about instruction level parallelism without all the superscalar on-the-fly analysis of instruction level data dependencies (so the hardware can do all that rescheduling on the fly). I think that if interlock-free software scheduled CPUs allowed us to reach 20GHz clock speeds where complex superscalar machines were stuck at 3GHz we'd all be jumping ship - but they're not
    - AstralStorm 8y ago
      Essentially the CPU is running a JIT branch optimization and speculative loads which cannot ever be matched by static optimization. The interesting part is that any JIT compiler that is so dynamic in performance will have these problems. The solution might be to actually prevent some kind of security important code from being optimized in this way. Say, forcing full cache sync in-order execution for parts of code with no resource sharing between cores.
      - 8y ago
Tuna-Fish 8y ago
The compiler cannot compete with the CPU because the CPU has strictly more information. Any memory access can take between 4 and thousands of cycles of time to execute, and the compiler won't know how long any of them take. The CPU can reorder based on exact knowledge.
- Symmetry 8y ago
  Using profile guided optimization can help a lot with this in practice, though it isn't perfect.

kentonv 8y ago

> Going forward, the only reasonable assumption is that if you let a third party run their code in a process, no matter how you verify accesses or otherwise try to contain that code, you should assume it has a read access to the entire process. If this is true, it is unbelievably bad for the future of security and computing in general. People throwing around this assertion are, in my opinion, not appreciating how bad it is. We need to try a lot harder before we give up. Here's why: 1. Finer-grained isolation makes security better, because it allows us to apply the Principle of Least Authority to each component of the system, and protect components of a system from bugs in other components. To make meaningful gains in security going forward, we need to encourage more fine-grained isolation. If process-level isolation is the finest grain we'll ever have, we can't make these advances. 2. The scalability of edge computing requires finer-grained isolation than process isolation. The trend is towards pushing more and more code to run directly on devices or edge networks rather than centralized servers. That means that the places where code runs need to be able to handle orders of magnitude more tenants than before -- because everyone wants their code to run in every location. If we can't achieve high multi-tenancy securely -- by which I mean 10,000 or more independently isolated pieces of code running on the same machine -- then the only solution will be to limit these resources to big, trusted players. Small companies will but shut out from "the edge". That's bad. Luckily, the "process isolation is the only isolation" claim is wrong. It's true that we need to evolve our approach to make sub-process isolation secure, but it's not impossible at all. In fact, it's possible to design a sandbox where you can trivially prove that code running inside it cannot observe side channels. Here's how that works: Thank about Haskell, or another purely-functional language. An attacker provides you with a pure function to execute. Because it's a pure function, for some given set of inputs, it will always produce exactly the same output, no matter what machine you run it on, or what else is going on in the background. Therefore, the output cannot possibly incorporate observations from side channels, no matter what the code does internally. So: It is possible to run attacker-provided code without exposing secrets from the process's memory space. The question is, of course, how do we build a useful sandbox that relies on this property. There is a lot of work to be done there. Luckily, we don't really have to use a purely-functional language, we only need to use a deterministic language. It turns out that the world's most popular language, JavaScript, is actually highly deterministic, in large part due to its single-threaded nature. We do need to remove access to timers, or find a way to make them not useful. We also need to prevent attackers from being able to time their code's execution remotely. Basically, we need to think carefully about the I/O surface while paying attention to timing. But sandboxing has always been about thinking carefully about the I/O surface, and we have a lot of control there. We just have a new aspect that we need to account for. I think it's doable.

yuhong 8y ago

I think Spectre is even less severe then Meltdown though.

cmsimike 8y ago

Thank you for your response!

JudasGoat 8y ago

I know the Arm A-53 isn't cutting edge fast, but hardly one tenth. Seeing as the A-53 is unaffected by both Meltdown and Spectre, my assumption was that it didn't use speculative execution.

baobrien 8y ago
Basically every CPU with a pipeline -- even simple, in-order cores -- speculate by way of branch prediction. I think with the Cortex A53 the pipeline is short enough and the branch predictor simple enough that you can't build a useful spectre attack. It's also common in simpler in-order cores to speculate a little bit around memory accesses.
- dnautics 8y ago
  There are architectural choices for pipelined systems you can make that don't specex, but it requires compiler level optimization to take advantage of them, so it's a chicken and egg problem to get adoption
  - ajross 8y ago
    Been there, done that, didn't work. VLIW has been tried a dozen times and failed every time.
    - dnautics 8y ago
      machine learning has been tried a dozen times and failed every time. Other developments catch up, like compiler technology. Not directly related to branch prediction, but: We are much better at polyhedral optimization than the last time people tried VLIW, for example. Besides, the last major VLIW push (itanium) DID have SpecEx AND Branch Prediction, and did NOT have delay slots.
      - ajross 8y ago
        Shrug. To paraphrase, your argument for "this can be done" is "THIS time will be different I swear!". What will actually happen, of course, is that everyone will put a ASID/PCID as a tag word into all the relevant caches, they'll stop being probe-able from other contexts, and we'll all keep our deep pipelines and speculation and the cache crisis of 2018 will be just a story we tell our grandkids. Much cheaper than a paradigm shift based on long-since tried and rejected technology.

caf 8y ago

Except for Spectre variant 1. That will always stay with us, because there is no sensible fix for it. Is it not a feasible fix for Spectre-v1 to have loaded cachelines wait in a staging area, and not actually update the cache hierarchy until the load instruction is retired?

bdonlan 8y ago
Loading those cache lines might invalidate an exclusively held cache line on another core. When that core tries to write to that cache line, it can observe a slower write.
- deleted 8y ago
  [deleted]
_chris_ 8y ago
Yes, you can buffer up and kill any misspeculated updates. You have to do this for ALL shared structures, not just data caches. Annoying, but not impossible. The scarier part is managing bandwidth contention to shared structures.
nhaehnle 8y ago
Is it theoretically possible? Maybe. Is it practical? Almost certainly not. Keep in mind that it's not just the L1 cache you potentially need to worry about, but also all the higher level caches, which are really like remote nodes in a distributed system. Even the last level cache would have to be aware of the current speculation state of all threads in the system, which seems like it would be really expensive. What's perhaps even worse, you wouldn't be able to hit on speculatively loaded cache lines until they have "committed" (because the resulting change in external bandwidth utilization may be observable from another thread), which would probably kill a lot of the benefits of the cache in the first place.

forapurpose 8y ago

> everyone in the industry never though that leaking information out of speculative context was possible Everyone overlooked it, or everyone had a good reason to think it was impossible? The information is there; it's a target that needs to be secured.