2 ms·

My understanding is that branch prediction got better in the ‘10s and a bunch of techniques that didn’t work before do now.

pizlonator 2y ago
The modern VM technique looks almost exactly like what the original PIC papers talked about in the 90s. There are some details that are different, but I'm not sure that the details come down to exploiting changes in branch prediction efficiency. I think the things that changed come mostly down to the fact that the original PIC paper was a first stab by a small team whereas modern VMs involve decades of engineering by larger teams (so everything that could get more complex as a consequence of tuning did get more complex). So, while it's true that microarches changed in a lot of ways, the overall implications for how you build VMs are not so big.
- hinkley 2y ago
  Are you still using a threaded interpreter main loop? That didn't really come around until the mid 90's and I've been hearing for about ten years now that it's not a clear win anymore due to predictors being able to read through two levels of indirection.
  - pizlonator 2y ago
    The last time I ran the experiment of having a single jump, it was slower than jump-per-opcode-handler. It's true that predictors are able to see through multiple levels, but a threaded interpreter gives them plus one level, and that ends up mattering as much as it always did.
  - titzer 2y ago
    Threaded dispatch is absolutely worth it. Wizard's interpreter gets anywhere from 10-35% performance improvement from threaded dispatch.
gopalv 2y ago
> that branch prediction got better in the ‘10s and a bunch of techniques that didn’t work before do now. They got better than they had any right to be, but then we found out that Spectre & Meltdown were vulnerabilities rather than optimizations. For example, a switch based interpreter was fast as a CGOTO one for a brief period between 2012 and 2018, but suddenly got slower again as the CPUs could no longer rely on branch prediction to do prefetching.
- titzer 2y ago
  While better predictors allow the speculation window to be larger on average, the the real culprit is that large speculation window. Even if the branch predictor weren't very smart, it will still do well on a program with stable, predictable branches, thus allowing a large speculation window to open up. The vulnerability is that some of those branches guard really important things, like not going out-of-bounds of an array. So a Spectre attack, which works by exploiting a mispredicted branch, is a constructive attack where the gadget is tuned for the branch predictor anyway. The other part of an attack, the windowing gadget, just relies on making a really slow input into a branch. Neither of them would be particularly harder with a dumb predictor.

pizlonator 2y ago

The modern VM technique looks almost exactly like what the original PIC papers talked about in the 90s. There are some details that are different, but I'm not sure that the details come down to exploiting changes in branch prediction efficiency. I think the things that changed come mostly down to the fact that the original PIC paper was a first stab by a small team whereas modern VMs involve decades of engineering by larger teams (so everything that could get more complex as a consequence of tuning did get more complex). So, while it's true that microarches changed in a lot of ways, the overall implications for how you build VMs are not so big.

hinkley 2y ago
Are you still using a threaded interpreter main loop? That didn't really come around until the mid 90's and I've been hearing for about ten years now that it's not a clear win anymore due to predictors being able to read through two levels of indirection.
- pizlonator 2y ago
  The last time I ran the experiment of having a single jump, it was slower than jump-per-opcode-handler. It's true that predictors are able to see through multiple levels, but a threaded interpreter gives them plus one level, and that ends up mattering as much as it always did.
- titzer 2y ago
  Threaded dispatch is absolutely worth it. Wizard's interpreter gets anywhere from 10-35% performance improvement from threaded dispatch.

gopalv 2y ago

> that branch prediction got better in the ‘10s and a bunch of techniques that didn’t work before do now. They got better than they had any right to be, but then we found out that Spectre & Meltdown were vulnerabilities rather than optimizations. For example, a switch based interpreter was fast as a CGOTO one for a brief period between 2012 and 2018, but suddenly got slower again as the CPUs could no longer rely on branch prediction to do prefetching.

titzer 2y ago
While better predictors allow the speculation window to be larger on average, the the real culprit is that large speculation window. Even if the branch predictor weren't very smart, it will still do well on a program with stable, predictable branches, thus allowing a large speculation window to open up. The vulnerability is that some of those branches guard really important things, like not going out-of-bounds of an array. So a Spectre attack, which works by exploiting a mispredicted branch, is a constructive attack where the gadget is tuned for the branch predictor anyway. The other part of an attack, the windowing gadget, just relies on making a really slow input into a branch. Neither of them would be particularly harder with a dumb predictor.