6 ms·

So you are questioning the effect of direct threading code? Because I assure you, that has a significant impact compared to the naive switch-eval loop, on the o

by kaba0 3y ago

So you are questioning the effect of direct threading code? Because I assure you, that has a significant impact compared to the naive switch-eval loop, on the order of 2-4x from my experience. (Source: have written a JVM interpreter with the naive approach that is approximately that much slower than the OpenJDK with disabled JIT compiler. It was later also confirmed by people working on the actual JVM that that’s roughly the order of difference between the two approaches. Unfortunately I can’t find a great article on the topic right now (google is beyond useless recently)). The article is about a further small improvement over this much larger and commonly implemented one.

hinkley 3y ago
The last new information I had on this was that direct threading stopped being a distinct perf advantage when branch predictors got more sophisticated (able to predict indirect branches). Do you have benchmarks on current gen hardware that suggest that the code complexity results in a measurable difference in outcomes?
- kaba0 3y ago
  I wrote that interpreter roughly a year ago — my computer is definitely not the most up-to-date (the CPU is an intel i5-4690), but I would be surprised if the difference would have disappeared. Also, I don’t think that it could even in principle be efficiently branch predicted — you are basically doing a memory read, jump conditionally based on that, jump back, do another memory read vs memory read, jump there, memory read, jump there. There is one less jump involved here, so any form of improved branch prediction would also enhance this second form as well. I found this, you might be able to benchmark it yourself if you have better hardware: https://github.com/vshymanskyy/interp https://github.com/vshymanskyy/interp
- ghusbands 3y ago
  Even if it were only the same speed on current gen hardware, most users are on older hardware, anyway, so it would still be a win unless it actually reduced performance.
  - hinkley 3y ago
    That was pre COVID news, if not more on the order of five years ago. Although now that I think of it, with Spectre and Meltdown mitigations perhaps the clock has been turned back on those improvements.
_a_a_a_ 3y ago
> So you are questioning the effect of direct threading code? No, cos I said "the benchmarks speak for themselves". But I am trying to understand why it's better and under what circumstances.
- kaba0 3y ago
  I meant it in terms of whether your question is about the reordering or the direct threading vs naive switch-loop.
  - _a_a_a_ 3y ago
    The direct threading is what interests me. From a link provided above by @ghusbands https://eli.thegreenplace.net/2012/07/12/computed-goto-for-efficient-dispatch-tables https://eli.thegreenplace.net/2012/07/12/computed-goto-for-e... "...and the goto version is 25% faster than the switch version... Comments inside the CPython implementation note that using computed goto made the Python VM 15-20% faster" (Which isn't the 2-4x speed up that you found. Maybe you meant 20-40%?). In that link they say that part of the reason is because of the bounds checking which is automatically done by the switch statement. What I'm struggling with which is no doubt down to my lack of knowledge, is why it should be significantly faster (if you take out or disable the bounds checking added by the switch). I can see there would be some advantage in spreading out the jumps so the branch predictor could work a bit better, but I am surprised at the speed up. I guess mispredicted branches are so expensive that doing just a little bit better could account for the difference, but I am trying to understand it.

by kaba0 3y ago

hinkley 3y ago

The last new information I had on this was that direct threading stopped being a distinct perf advantage when branch predictors got more sophisticated (able to predict indirect branches). Do you have benchmarks on current gen hardware that suggest that the code complexity results in a measurable difference in outcomes?

kaba0 3y ago
I wrote that interpreter roughly a year ago — my computer is definitely not the most up-to-date (the CPU is an intel i5-4690), but I would be surprised if the difference would have disappeared. Also, I don’t think that it could even in principle be efficiently branch predicted — you are basically doing a memory read, jump conditionally based on that, jump back, do another memory read vs memory read, jump there, memory read, jump there. There is one less jump involved here, so any form of improved branch prediction would also enhance this second form as well. I found this, you might be able to benchmark it yourself if you have better hardware: https://github.com/vshymanskyy/interp https://github.com/vshymanskyy/interp
ghusbands 3y ago
Even if it were only the same speed on current gen hardware, most users are on older hardware, anyway, so it would still be a win unless it actually reduced performance.
- hinkley 3y ago
  That was pre COVID news, if not more on the order of five years ago. Although now that I think of it, with Spectre and Meltdown mitigations perhaps the clock has been turned back on those improvements.

_a_a_a_ 3y ago

> So you are questioning the effect of direct threading code? No, cos I said "the benchmarks speak for themselves". But I am trying to understand why it's better and under what circumstances.

kaba0 3y ago
I meant it in terms of whether your question is about the reordering or the direct threading vs naive switch-loop.
- _a_a_a_ 3y ago
  The direct threading is what interests me. From a link provided above by @ghusbands https://eli.thegreenplace.net/2012/07/12/computed-goto-for-efficient-dispatch-tables https://eli.thegreenplace.net/2012/07/12/computed-goto-for-e... "...and the goto version is 25% faster than the switch version... Comments inside the CPython implementation note that using computed goto made the Python VM 15-20% faster" (Which isn't the 2-4x speed up that you found. Maybe you meant 20-40%?). In that link they say that part of the reason is because of the bounds checking which is automatically done by the switch statement. What I'm struggling with which is no doubt down to my lack of knowledge, is why it should be significantly faster (if you take out or disable the bounds checking added by the switch). I can see there would be some advantage in spreading out the jumps so the branch predictor could work a bit better, but I am surprised at the speed up. I guess mispredicted branches are so expensive that doing just a little bit better could account for the difference, but I am trying to understand it.