5 ms·

I wish there was a good way of knowing when an if forces an actual branch rather than when it doesn't. The reason people do potentially more expensive mix/lerps

I wish there was a good way of knowing when an if forces an actual branch rather than when it doesn't. The reason people do potentially more expensive mix/lerps is because while it might cost a tiny overhead, they are scared of making it a branch. I do like that the most obvious v = x > y ? a : b; actually works, but it's also concerning that we have syntax where an if is some times a branch and some times not. In a context where you really can't branch, you'd almost like branch-if and non-branching-if to be different keywords. The non-branching one would fail compilation if the compiler couldn't do it without branching. The branching one would warn if it could be done with branching.

deleted 2y ago
[deleted]
ajross 2y ago
> it's also concerning that we have syntax where an if is some times a branch and some times not. That's true on scalar CPUs too though. The CMOV instruction arrived with the P6 core in 1995, for example. Branches are expensive everywhere, even in scalar architectures, and compilers do their best to figure out when they should use an alternative strategy. And sometimes get it wrong, but not very often.
- masklinn 2y ago
  For scalar CPUs, historically CMOV used to be relatively slow on x86, and notably for reliable branching patterns (>75% reliable) branches could be a lot faster. cmov also has dependencies on all three inputs, so if there's a high level of bias towards the unlikely input having a much higher latency than the likely one a cmov can cost a fair amount of waiting. Finally cmov were absolutely terrible on P4 (10-ish cycles), and it's likely that a lot of their lore dates back to that.
chrisjj 2y ago
The good way is to inspect the code :) > it's also concerning that we have syntax where an if is some times a branch and some times not. It would be more concerning if we didn't. We might get a branch on one GPU and none on another.
- phkahler 2y ago
  >> The good way is to inspect the code :) The best way is to profile the code. Time is what we are after, so measure that.
  - chrisjj 2y ago
    For sure.
plagiarist 2y ago
I think that capability in the shader language would be interesting to have. One might even want it to two-color all functions in the code. Anything annotated nonbranching must have if statements compile down to conditional moves and must only call nonbranching functions.
- catlifeonmars 2y ago
  This is also very relevant for cryptography use cases, where branching is a potential side channel for leaking secret information.
nice_byte 2y ago
godbolt has rga compiler now, you can always paste in hlsl and look at the actual rdna instructions that are generated (what GPU actually runs, not spirv)
mpreda 2y ago
Exactly. Consider this example: a = f(z); b = g(z); v = x > y ? a : b; Assuming computing the two function calls f() and g() is relativelly expensive, it becomes a trade-off whether to emit conditional code or to compute both followed by a select. So it's not a simple choice, and the decision is made by the compiler.
- dragontamer 2y ago
  This is a GPU focused article. The GPU will almost always execute f and g due to GPU differences vs CPU. You can avoid the f vs g if you can ensure a scalar Boolean / if statement that is consistent across the warp. So it's not 'always' but requires incredibly specific coding patterns to 'force' the optimizer + GPU compiler into making the branch.
  - justsid 2y ago
    It depends. If the code flow is uniform for the warp, only side of the branch needs to be evaluated. But you could still end up with pessimistic register allocation because the compiler can’t know it is uniform. It’s sometimes weirdly hard to reason about how exactly code will end up executing on the GPU.
  - danybittel 2y ago
    f or g may have side effects too. Like writing to memory. Now a conditional has a different meaning. You could also have some fun stuff, where f and g return a boolean, because thanks to short circuit evaluation && || are actually also conditionals in disguise.
    - account42 2y ago
      Side effects will be masked, the GPU is still executing exactly the same code for the entire workgroup.
pandaman 2y ago
>The reason people do potentially more expensive mix/lerps is because while it might cost a tiny overhead, they are scared of making it a branch. And the reason for that is the confusing documentation from NVidia and its cg/CUDA compilers. I believe they did not want to scare programmers at first and hid the execution model, talking about "threads" and then they kept using that abstraction to hype up their GPUs ("it has 100500 CUDA threads!"). The result is people coding for GPUs with some bizarre superstitions though. You actually want branches in the the code. Those are quick. The problem is that you cannot have a branch off a SIMD way so, instead of a branch the compiler will emit code for both branches and the results will be masked out based on the branch's condition. So, to answer your question - any computation based on shader inputs (vertices, computer shader indices and what not) cannot and won't branch. It will all be executed sequentially with masking. Even in the TFA example, both values of ? operator are computed, the same happens with any conditional on an SIMD value. There can be shortcut branches emitted by the compiler to quickly bypass computations when all ways are the same value but in general case everything will be computed for every condition being true as well as being false. Only conditionals based on scalar registers (shader constants/unform values) will generate branches and those are super quick.
- ryao 2y ago
  You can always have the compiler dump the assembly output so you can examine it. I suspect few do that.
  - vanderZwan 2y ago
    Does this also apply for shaders? And is it even useful given the enormous variation in hardware capabilities out there. My impression was that it's all JIT compiled unless you know which hardware you're targeting, e.g. Valve precompiling highly optimized shaders for the Steam Deck (I'm not a grapics programmer, mind you, so please correct any misunderstandings on my end)
    - swiftcoder 2y ago
      It's all JIT'd based on the specific driver/GPU, but the intermediate assembly language is sufficient to inspect things like branches and loop unrolling.
mwkaufma 2y ago
One can do the precisely how it's done in the article -- inspect the assembly.
NohatCoder 2y ago
But you don't generally need to care if the shader code contains a few branches, modern GPUs handles those reasonably well, and the compiler will probably make a reasonable guess about what is fastest.
- account42 2y ago
  You do need to care about large non-uniform branches as in the general case the GPU will have to execute both sides.
  - NohatCoder 2y ago
    A non-branching version of the same algorithm will also run code equivalent to both branches. The branching version may sometimes skip one of the branches, the non-branching version can't. So if the functionality you want is best described by a branch, then use a branch.
account42 2y ago
You got this the wrong way around: For GPUs conditional moves are the default and real branches are a performance optimization possible only if the branch is uniform (=same side taken for the entire workgroup).
grg0 2y ago
The good way of knowing is to look at the assembly generated by the compiler. Maybe not a completely satisfying answer given that the result is heavily vendor-dependent, but unless the high-level language exposes some way of explicitly controlling it, then assembly is what you got. > The reason people do potentially more expensive mix/lerps is because while it might cost a tiny overhead, they are scared of making it a branch. This is a problem, though. People shouldn't do things potentially, they should look at the actual code that is generated and executed.

deleted 2y ago

[deleted]

ajross 2y ago

> it's also concerning that we have syntax where an if is some times a branch and some times not. That's true on scalar CPUs too though. The CMOV instruction arrived with the P6 core in 1995, for example. Branches are expensive everywhere, even in scalar architectures, and compilers do their best to figure out when they should use an alternative strategy. And sometimes get it wrong, but not very often.

masklinn 2y ago
For scalar CPUs, historically CMOV used to be relatively slow on x86, and notably for reliable branching patterns (>75% reliable) branches could be a lot faster. cmov also has dependencies on all three inputs, so if there's a high level of bias towards the unlikely input having a much higher latency than the likely one a cmov can cost a fair amount of waiting. Finally cmov were absolutely terrible on P4 (10-ish cycles), and it's likely that a lot of their lore dates back to that.

chrisjj 2y ago

The good way is to inspect the code :) > it's also concerning that we have syntax where an if is some times a branch and some times not. It would be more concerning if we didn't. We might get a branch on one GPU and none on another.

phkahler 2y ago
>> The good way is to inspect the code :) The best way is to profile the code. Time is what we are after, so measure that.
- chrisjj 2y ago
  For sure.

plagiarist 2y ago

I think that capability in the shader language would be interesting to have. One might even want it to two-color all functions in the code. Anything annotated nonbranching must have if statements compile down to conditional moves and must only call nonbranching functions.

catlifeonmars 2y ago
This is also very relevant for cryptography use cases, where branching is a potential side channel for leaking secret information.

nice_byte 2y ago

godbolt has rga compiler now, you can always paste in hlsl and look at the actual rdna instructions that are generated (what GPU actually runs, not spirv)

mpreda 2y ago

Exactly. Consider this example: a = f(z); b = g(z); v = x > y ? a : b; Assuming computing the two function calls f() and g() is relativelly expensive, it becomes a trade-off whether to emit conditional code or to compute both followed by a select. So it's not a simple choice, and the decision is made by the compiler.

dragontamer 2y ago
This is a GPU focused article. The GPU will almost always execute f and g due to GPU differences vs CPU. You can avoid the f vs g if you can ensure a scalar Boolean / if statement that is consistent across the warp. So it's not 'always' but requires incredibly specific coding patterns to 'force' the optimizer + GPU compiler into making the branch.
- justsid 2y ago
  It depends. If the code flow is uniform for the warp, only side of the branch needs to be evaluated. But you could still end up with pessimistic register allocation because the compiler can’t know it is uniform. It’s sometimes weirdly hard to reason about how exactly code will end up executing on the GPU.
- danybittel 2y ago
  f or g may have side effects too. Like writing to memory. Now a conditional has a different meaning. You could also have some fun stuff, where f and g return a boolean, because thanks to short circuit evaluation && || are actually also conditionals in disguise.
  - account42 2y ago
    Side effects will be masked, the GPU is still executing exactly the same code for the entire workgroup.

pandaman 2y ago

>The reason people do potentially more expensive mix/lerps is because while it might cost a tiny overhead, they are scared of making it a branch. And the reason for that is the confusing documentation from NVidia and its cg/CUDA compilers. I believe they did not want to scare programmers at first and hid the execution model, talking about "threads" and then they kept using that abstraction to hype up their GPUs ("it has 100500 CUDA threads!"). The result is people coding for GPUs with some bizarre superstitions though. You actually want branches in the the code. Those are quick. The problem is that you cannot have a branch off a SIMD way so, instead of a branch the compiler will emit code for both branches and the results will be masked out based on the branch's condition. So, to answer your question - any computation based on shader inputs (vertices, computer shader indices and what not) cannot and won't branch. It will all be executed sequentially with masking. Even in the TFA example, both values of ? operator are computed, the same happens with any conditional on an SIMD value. There can be shortcut branches emitted by the compiler to quickly bypass computations when all ways are the same value but in general case everything will be computed for every condition being true as well as being false. Only conditionals based on scalar registers (shader constants/unform values) will generate branches and those are super quick.

ryao 2y ago
You can always have the compiler dump the assembly output so you can examine it. I suspect few do that.
- vanderZwan 2y ago
  Does this also apply for shaders? And is it even useful given the enormous variation in hardware capabilities out there. My impression was that it's all JIT compiled unless you know which hardware you're targeting, e.g. Valve precompiling highly optimized shaders for the Steam Deck (I'm not a grapics programmer, mind you, so please correct any misunderstandings on my end)
  - swiftcoder 2y ago
    It's all JIT'd based on the specific driver/GPU, but the intermediate assembly language is sufficient to inspect things like branches and loop unrolling.

mwkaufma 2y ago

One can do the precisely how it's done in the article -- inspect the assembly.

NohatCoder 2y ago

But you don't generally need to care if the shader code contains a few branches, modern GPUs handles those reasonably well, and the compiler will probably make a reasonable guess about what is fastest.

account42 2y ago
You do need to care about large non-uniform branches as in the general case the GPU will have to execute both sides.
- NohatCoder 2y ago
  A non-branching version of the same algorithm will also run code equivalent to both branches. The branching version may sometimes skip one of the branches, the non-branching version can't. So if the functionality you want is best described by a branch, then use a branch.

account42 2y ago

You got this the wrong way around: For GPUs conditional moves are the default and real branches are a performance optimization possible only if the branch is uniform (=same side taken for the entire workgroup).

grg0 2y ago

The good way of knowing is to look at the assembly generated by the compiler. Maybe not a completely satisfying answer given that the result is heavily vendor-dependent, but unless the high-level language exposes some way of explicitly controlling it, then assembly is what you got. > The reason people do potentially more expensive mix/lerps is because while it might cost a tiny overhead, they are scared of making it a branch. This is a problem, though. People shouldn't do things potentially, they should look at the actual code that is generated and executed.