3 ms·

I really hate to see this rush to "make sure our code doesn't use branch prediction". It may be the best fix we have available now, but it's going to create a l

I really hate to see this rush to "make sure our code doesn't use branch prediction". It may be the best fix we have available now, but it's going to create a legacy of bizarre inefficient machine code that will last forever.

zerkten 9y ago
How will this last forever? Browsers seem like a relatively competitive and performance-oriented space.
pizlonator 9y ago
Our mitigations do not prevent branch prediction.
- Game_Ender 9y ago
  You are removing branches, and adding slightly less clear code to do it. If this resulted in faster or cleaner code you would already be doing it. So the OP’s point holds: if these security holes are fixed at the CPU level, there will still be a generation of software compiled with these mitigation’s in and no way for a user on a newer CPU to disable them. EDIT: I just wanted to clarify here that I did not mean “all branches” just some of them, hence the name of the technique which is “branchless security” (which is a really good piece of engineering). The frustration is basically we have to bake in CPU model specific mitigation’s into binaries. You can of course patch it out, but the affected CPUs will in theory be in the wild for a long time to come even once corrected ones are released.
  - om2 9y ago
    The mitigations are largely in code produced by the JIT. It's not hard to patch the JIT's codgen to back this out (perhaps even based on current CPU) if it becomes unnecessary. It would be nice if CPU architectures never had this fundamental vulnerability, but they do. So this seems like the best response for the reality we actually live in.
    - Game_Ender 9y ago
      That is super cool. I had no idea a lot of this was JIT side, and I do agree this is one of the best fixes I have seen for this yet.
  - pizlonator 9y ago
    We are not removing branches. To fix this in hardware without software changes you would have to disable branch prediction. That would perform worse than what we are doing.
    - Game_Ender 9y ago
      Sorry maybe I missed something but you had some really cool techniques to avoid using branches to do things like array length checks and type checks? So you replaced an if branch with a new, potentially slower, and harder to understand concept. I was not trying to imply you removed all branches. It is a really cool piece of engineering but we can still be sad we have to implement it. (On the hardware side I was assuming sommeone clever at Intel will come up with some security non Perf killing change to make branches trustable again).
      - pizlonator 9y ago
        Now we are doing both bounds check branches and index masking. The index masking is purely additive. We didn't replace branches. Similarly, where we use pointer poisoning, it is in addition to some other security check, which is almost always a branch.
        iainmerrick 9y ago
        Also, security aside, removing (mispredicted) branches is a legitimate way to try to speed up code!
alien_at_work 9y ago
I feel the opposite. I'd like nothing better than this to be the end of branch prediciton. I can think of no other technology that's been so deterimental to high level languages than branch prediction. How many elegant, beutiful algorithms are replaced by bizarre data structures, etc., because avoiding pipeline stalls will trump any other optimization you can make. For most things the CPU is doing, we can ignore in high level languages because the compiler/optimizer can turn our high level code into the appropriate byte code to take advantage of deep hardwar magic. Not branch prediction, though, no matter how high level my language, I still must consider pipeline size, cache alignment and so on when designing a novel new container or algorithm or risk it being inexplicitly slow. Branch prediction is, and always was a hack. Let it die.
- ambulancechaser 9y ago
  but removing branch prediction would be to restore those stalls and inefficiencies as the way things are when you don't use those strange data structures, right? The effect of this seems to be "I wish everything was slower and then the naive implementation would be the best implementation".
  - alien_at_work 9y ago
    >restore those stalls and inefficiencies as the way things are when you don't use those strange data structures, right No, a stall is only a thing because branch prediction exists. Get rid of it and I can go back to not caring about the deep underlying architecture of the hardware I may potentially be running on. And not doing branch prediction isn't inefficient... that's a bizare statement. Branch prediction and speculative execution are themselves inefficiencies (i.e. spending power to do things that will turn out to not be needed fairly often). >the naive implementation would be the best implementation Not the naive implementation. A well thought out, developed implementation that is defeated by deep hardware details. The promise of high level languages was always to avoid having to know these sorts of details. In designing an algorithm I should be worried about things like how often I access data (e.g. can I cut down on how many accesses with some clever structure?) and the like. Now that's all trumped by deeply knowing the low level implementation.

zerkten 9y ago

How will this last forever? Browsers seem like a relatively competitive and performance-oriented space.

pizlonator 9y ago

Our mitigations do not prevent branch prediction.

Game_Ender 9y ago
You are removing branches, and adding slightly less clear code to do it. If this resulted in faster or cleaner code you would already be doing it. So the OP’s point holds: if these security holes are fixed at the CPU level, there will still be a generation of software compiled with these mitigation’s in and no way for a user on a newer CPU to disable them. EDIT: I just wanted to clarify here that I did not mean “all branches” just some of them, hence the name of the technique which is “branchless security” (which is a really good piece of engineering). The frustration is basically we have to bake in CPU model specific mitigation’s into binaries. You can of course patch it out, but the affected CPUs will in theory be in the wild for a long time to come even once corrected ones are released.
- om2 9y ago
  The mitigations are largely in code produced by the JIT. It's not hard to patch the JIT's codgen to back this out (perhaps even based on current CPU) if it becomes unnecessary. It would be nice if CPU architectures never had this fundamental vulnerability, but they do. So this seems like the best response for the reality we actually live in.
  - Game_Ender 9y ago
    That is super cool. I had no idea a lot of this was JIT side, and I do agree this is one of the best fixes I have seen for this yet.
- pizlonator 9y ago
  We are not removing branches. To fix this in hardware without software changes you would have to disable branch prediction. That would perform worse than what we are doing.
  - Game_Ender 9y ago
    Sorry maybe I missed something but you had some really cool techniques to avoid using branches to do things like array length checks and type checks? So you replaced an if branch with a new, potentially slower, and harder to understand concept. I was not trying to imply you removed all branches. It is a really cool piece of engineering but we can still be sad we have to implement it. (On the hardware side I was assuming sommeone clever at Intel will come up with some security non Perf killing change to make branches trustable again).
    - pizlonator 9y ago
      Now we are doing both bounds check branches and index masking. The index masking is purely additive. We didn't replace branches. Similarly, where we use pointer poisoning, it is in addition to some other security check, which is almost always a branch.
      - iainmerrick 9y ago
        Also, security aside, removing (mispredicted) branches is a legitimate way to try to speed up code!

alien_at_work 9y ago

I feel the opposite. I'd like nothing better than this to be the end of branch prediciton. I can think of no other technology that's been so deterimental to high level languages than branch prediction. How many elegant, beutiful algorithms are replaced by bizarre data structures, etc., because avoiding pipeline stalls will trump any other optimization you can make. For most things the CPU is doing, we can ignore in high level languages because the compiler/optimizer can turn our high level code into the appropriate byte code to take advantage of deep hardwar magic. Not branch prediction, though, no matter how high level my language, I still must consider pipeline size, cache alignment and so on when designing a novel new container or algorithm or risk it being inexplicitly slow. Branch prediction is, and always was a hack. Let it die.

ambulancechaser 9y ago
but removing branch prediction would be to restore those stalls and inefficiencies as the way things are when you don't use those strange data structures, right? The effect of this seems to be "I wish everything was slower and then the naive implementation would be the best implementation".
- alien_at_work 9y ago
  >restore those stalls and inefficiencies as the way things are when you don't use those strange data structures, right No, a stall is only a thing because branch prediction exists. Get rid of it and I can go back to not caring about the deep underlying architecture of the hardware I may potentially be running on. And not doing branch prediction isn't inefficient... that's a bizare statement. Branch prediction and speculative execution are themselves inefficiencies (i.e. spending power to do things that will turn out to not be needed fairly often). >the naive implementation would be the best implementation Not the naive implementation. A well thought out, developed implementation that is defeated by deep hardware details. The promise of high level languages was always to avoid having to know these sorts of details. In designing an algorithm I should be worried about things like how often I access data (e.g. can I cut down on how many accesses with some clever structure?) and the like. Now that's all trumped by deeply knowing the low level implementation.