3 ms·

A standard way to handle branching in gpu code is with masking, like so (where x is a vector, and broadcasting is implied): M = x > 0 y = M * f(x) + (1-M) * g(x

A standard way to handle branching in gpu code is with masking, like so (where x is a vector, and broadcasting is implied): M = x > 0 y = M * f(x) + (1-M) * g(x) So you end up evaluating both sides of the decision branch and adding the results. But this is fine if you've got a dumb number of cores. And often traditional cpus wind up evaluating both branches anyway.

monocasa 5y ago
> And often traditional cpus wind up evaluating both branches anyway. That's actually really overstated. Evaluating both sides isn't really something CPUs tend to do, but instead predict one path and roll back on mispredict. This is because the out of order hardware isn't a tree fundamentally, but generally better thought of a ring buffer where uncommitted state is what's between the head and tail. Storing diverging paths is incredibly expensive there. I'm not going to say something as strong as "it's never been done", but but I certainly don't know of an general purpose CPU arch that'll compute both sides of an architectural branch, instead relying an making good predictions down one instruction stream, then rolling back and restarting when it's clear you mispredicted.
- jasonwatkinspdx 5y ago
  It's not even about the expense of implementing diverging paths in hardware. This concept, of exploring like a tree vs a path was explored under the name Disjoint Eager Execution. You know what killed it? Branch predictors. In a world where branch predictors are maybe only 75% effective, DEE could make sense. We live in a world where branch predictors are far better than that. So it just isn't worth speculating off the predicted most likely path.
  - monocasa 5y ago
    What killed it was more the effectiveness of Tomasulo style out of order machinery, and the real problem not being control hazards, but data hazards. DEE was thought of in a day where memory was about as fast as the processor. That's why it's always being compared with cores like R3000.

monocasa 5y ago

> And often traditional cpus wind up evaluating both branches anyway. That's actually really overstated. Evaluating both sides isn't really something CPUs tend to do, but instead predict one path and roll back on mispredict. This is because the out of order hardware isn't a tree fundamentally, but generally better thought of a ring buffer where uncommitted state is what's between the head and tail. Storing diverging paths is incredibly expensive there. I'm not going to say something as strong as "it's never been done", but but I certainly don't know of an general purpose CPU arch that'll compute both sides of an architectural branch, instead relying an making good predictions down one instruction stream, then rolling back and restarting when it's clear you mispredicted.

jasonwatkinspdx 5y ago
It's not even about the expense of implementing diverging paths in hardware. This concept, of exploring like a tree vs a path was explored under the name Disjoint Eager Execution. You know what killed it? Branch predictors. In a world where branch predictors are maybe only 75% effective, DEE could make sense. We live in a world where branch predictors are far better than that. So it just isn't worth speculating off the predicted most likely path.
- monocasa 5y ago
  What killed it was more the effectiveness of Tomasulo style out of order machinery, and the real problem not being control hazards, but data hazards. DEE was thought of in a day where memory was about as fast as the processor. That's why it's always being compared with cores like R3000.