5 ms·

People sometimes talk about waiting for a Spectre "fix" at a hardware or software level, but it seems that the only fix is to lose Speculative execution entirel

People sometimes talk about waiting for a Spectre "fix" at a hardware or software level, but it seems that the only fix is to lose Speculative execution entirely (and with it, its performance gains). On my Haswell CPU, the performance hit was about 5% in various benchmarks, although newer architectures suffer less. Given that Spectre attacks still appear to be very difficult to perform in practice, I wonder if we will start to see Speculative execution at the hardware level brought back as an option for high performance and closed environment settings.

nicoburns 7y ago
I wonder whether you could mitigate by speculatively executing all/both branches, or somehow making it constant time...
- jnordwick 7y ago
  The issue is more that you are causing a calculated cache line to be fetched and then the timing difference between the L1 cache and L3 or MM fetch shows what the calculated value was. The only way to make those appear at the same speed is to slow L1 down on a speculative fetch to MM speed. It isn't like a simple branch with two possible values. The cache line could be one of 256 (if scanning a byte at a time).
microcolonel 7y ago
I don't think you are actually disabling speculative execution on your Haswell.
- Ragib_Zaman 7y ago
  Perhaps not at the hardware level, I'm not sure. I downloaded InSpectre (a windows application) which informs me that I had been protected against Spectre and Meltdown attacks through Windows updates. I also used the option to disable the protection, which is how I figured out how much of a performance hit I was taking.
  - jcranmer 7y ago
    The hardware mitigation at this point is disabling some of the advanced indirect branch prediction in some circumstances (I don't remember the exact set of circumstances). It is nowhere close to disabling all speculative execution.
  - gpderetta 7y ago
    That's probably meltdown and spectre V2 (i.e. cross process poisoning of indirect branch predictor). There is no hardware/firmware fix yet (and possibly ever) for spectre v1, only software mitigations.
0815test 7y ago
There are architecture-level alternatives to Speculative execution for high-performance compute - namely VLIW and the "tweaks" on it that efforts like The Mill are working on right now. The main problem with them is that they make the processor ISA inherently dependent on the actual details of its hardware configuration (you don't have a single ISA like with RISC, which can seamlessly scale from tiny micro-controller hardware to supercomputer- or datacenter-scale). But once you're willing to bite that bullet, and perhaps the vast majority of the software you use gets distributed in either source-code or arch-independent "VM" format, it can become a very attractive choice. Yes we used to think that compilers would never be good enough to make use of the implied parallelism in VLIW, but it might be time to revisit that now that both languages and compilation platforms have improved.
- gpderetta 7y ago
  The only existing VLIW that is able to run general purpose code in a reasonably efficient way is Nvidia Denver. Which is also vulnerable to spectre Edit: that I know, at least.
lallysingh 7y ago
I think you can do it by tracking caches pulled in during speculation and dumping them if speculation failed. Perhaps on a new barrier instruction that's lighter weight than lfence (as you're just enabling a recording mode for cache snooping). Ultimately I don't think many speculative instructions actually pull in cache lines, so we can either clean up when they do, or stop speculating when a line would need to be loaded.
- titzer 7y ago
  The right mental model is that, in modern out-of-order CPUs, there is no difference between executing "in speculation" and "not in speculation". There is just a huge pile of u-ops (~200 on big Intel chips) in the reorder buffer, and some of them may be unexecuted branches. The fetch unit plows right on through branch instructions using branch prediction and jams code "from the future" into the reorder buffer. The CPU keeps on executing whatever instructions are ready (i.e. have data dependencies met), irrespective of the in-flight branches. When a mispredicted branch is retired (~0.1-1% of the time), the CPU can throw away the entire reorder buffer and start over. A CPU can also abort-on-execute, throwing away only work related to the given mispredicted branch. So probably 99% of all cycles are spent with at least one unexecuted branch somewhere in the reorder buffer--the CPU is essentially always speculating.
- ghusbands 7y ago
  You also need to somehow restore the cache lines that got removed, else there's still information disclosure.
  - lallysingh 7y ago
    Fine point. So really, speculative instructions can't pull in cache lines.
- tntn 7y ago
  I don't see how this would really work. As soon as one core pulls in a cache line, all the other cores know about it, so somehow you would have to roll back all the coherency messaging as well.
jnordwick 7y ago
I see you are getting down too for a very congenial, decent post. While HN does have an issue with differing opinions, for some reason this (and a few others) are socially censored vigorously. Shrug. Not sure why.
- sciurus 7y ago
  "the only fix is to lose Speculative execution entirely" As mentioned elsewhere, this is incorrect and does not describe what CPU designers have done. That probably explains the downvotes.
- msbarnett 7y ago
  Because it's dead wrong, laughably so, in the assertion that speculative execution is a ~5% performance hit (it would, in reality, result in a couple of orders of magnitude slowdown on desktop workloads). As noted elsewhere OP completely misunderstood the fact that his CPU has received some of Intel's Spectre mitigation firmware patches and came to the silly conclusion that this meant his CPU was no longer doing speculative execution of any sort. HN is still a (vaguely) computer-savvy forum. Being extremely wrong about the fundamentals of how a modern CPU works is going to get you downvoted.
gpderetta 7y ago
The hit from disabling speculative execution is not a few percentage points, but a couple of orders of magnitude.

nicoburns 7y ago

I wonder whether you could mitigate by speculatively executing all/both branches, or somehow making it constant time...

jnordwick 7y ago
The issue is more that you are causing a calculated cache line to be fetched and then the timing difference between the L1 cache and L3 or MM fetch shows what the calculated value was. The only way to make those appear at the same speed is to slow L1 down on a speculative fetch to MM speed. It isn't like a simple branch with two possible values. The cache line could be one of 256 (if scanning a byte at a time).

microcolonel 7y ago

I don't think you are actually disabling speculative execution on your Haswell.

Ragib_Zaman 7y ago
Perhaps not at the hardware level, I'm not sure. I downloaded InSpectre (a windows application) which informs me that I had been protected against Spectre and Meltdown attacks through Windows updates. I also used the option to disable the protection, which is how I figured out how much of a performance hit I was taking.
- jcranmer 7y ago
  The hardware mitigation at this point is disabling some of the advanced indirect branch prediction in some circumstances (I don't remember the exact set of circumstances). It is nowhere close to disabling all speculative execution.
- gpderetta 7y ago
  That's probably meltdown and spectre V2 (i.e. cross process poisoning of indirect branch predictor). There is no hardware/firmware fix yet (and possibly ever) for spectre v1, only software mitigations.

0815test 7y ago

There are architecture-level alternatives to Speculative execution for high-performance compute - namely VLIW and the "tweaks" on it that efforts like The Mill are working on right now. The main problem with them is that they make the processor ISA inherently dependent on the actual details of its hardware configuration (you don't have a single ISA like with RISC, which can seamlessly scale from tiny micro-controller hardware to supercomputer- or datacenter-scale). But once you're willing to bite that bullet, and perhaps the vast majority of the software you use gets distributed in either source-code or arch-independent "VM" format, it can become a very attractive choice. Yes we used to think that compilers would never be good enough to make use of the implied parallelism in VLIW, but it might be time to revisit that now that both languages and compilation platforms have improved.

gpderetta 7y ago
The only existing VLIW that is able to run general purpose code in a reasonably efficient way is Nvidia Denver. Which is also vulnerable to spectre Edit: that I know, at least.

lallysingh 7y ago

I think you can do it by tracking caches pulled in during speculation and dumping them if speculation failed. Perhaps on a new barrier instruction that's lighter weight than lfence (as you're just enabling a recording mode for cache snooping). Ultimately I don't think many speculative instructions actually pull in cache lines, so we can either clean up when they do, or stop speculating when a line would need to be loaded.

titzer 7y ago
The right mental model is that, in modern out-of-order CPUs, there is no difference between executing "in speculation" and "not in speculation". There is just a huge pile of u-ops (~200 on big Intel chips) in the reorder buffer, and some of them may be unexecuted branches. The fetch unit plows right on through branch instructions using branch prediction and jams code "from the future" into the reorder buffer. The CPU keeps on executing whatever instructions are ready (i.e. have data dependencies met), irrespective of the in-flight branches. When a mispredicted branch is retired (~0.1-1% of the time), the CPU can throw away the entire reorder buffer and start over. A CPU can also abort-on-execute, throwing away only work related to the given mispredicted branch. So probably 99% of all cycles are spent with at least one unexecuted branch somewhere in the reorder buffer--the CPU is essentially always speculating.
ghusbands 7y ago
You also need to somehow restore the cache lines that got removed, else there's still information disclosure.
- lallysingh 7y ago
  Fine point. So really, speculative instructions can't pull in cache lines.
tntn 7y ago
I don't see how this would really work. As soon as one core pulls in a cache line, all the other cores know about it, so somehow you would have to roll back all the coherency messaging as well.

jnordwick 7y ago

I see you are getting down too for a very congenial, decent post. While HN does have an issue with differing opinions, for some reason this (and a few others) are socially censored vigorously. Shrug. Not sure why.

sciurus 7y ago
"the only fix is to lose Speculative execution entirely" As mentioned elsewhere, this is incorrect and does not describe what CPU designers have done. That probably explains the downvotes.
msbarnett 7y ago
Because it's dead wrong, laughably so, in the assertion that speculative execution is a ~5% performance hit (it would, in reality, result in a couple of orders of magnitude slowdown on desktop workloads). As noted elsewhere OP completely misunderstood the fact that his CPU has received some of Intel's Spectre mitigation firmware patches and came to the silly conclusion that this meant his CPU was no longer doing speculative execution of any sort. HN is still a (vaguely) computer-savvy forum. Being extremely wrong about the fundamentals of how a modern CPU works is going to get you downvoted.

gpderetta 7y ago

The hit from disabling speculative execution is not a few percentage points, but a couple of orders of magnitude.