4 ms·

Thanks Inigo ! The second wrong thing with the supposedly optimizer version is that it actually runs much slower than the original version. The reason is tha

Thanks Inigo ! The second wrong thing with the supposedly optimizer version is that it actually runs much slower than the original version. The reason is that the step() function is actually implemented like this: float step( float x, float y ) { return x < y ? 1.0 : 0.0; } How are we supposed to know what OpenGL functions are emulated rather than calling GPU primitives ?

Const-me 2y ago
The only way is do what OP did – compile your shader, disassemble, and read the assembly. I do that quite often with my HLSL shaders, learned a lot about that virtual instruction set. For example, it’s interesting GPUs have instruction sincos, but inverse trigonometry is emulated while compiling.
Waterluvian 2y ago
This is a great question that I see everywhere in programming and I think it is core to why you measure first when optimizing. You generally shouldn’t know or care how a built in is implemented. If do care, you’re probably thinking about optimization. At that point the answer is “measure and find out what works better.”
TeMPOraL 2y ago
EDIT: I see my source of confusion must be that "branch" must have a well-understood hardware-specific meaning that goes beyond the meaning I grew up with, which is that a conditional is a branch, because the path control takes (at the machine code level) is chosen at runtime. This makes a conditional jump a branch by definition. > How are we supposed to know what OpenGL functions are emulated rather than calling GPU primitives? To me the problem was obvious, but then again I'm having trouble with both your and author's statements about it. The problem I saw was, obviously by going for a step() function, people aren't turning logic into arithmetic, they're just hiding logic in a library function call. Just because step() is a built-in or something you'd find used in mathematical paper doesn't mean anything; the definition of step() in mathematics is literally a conditional too. Now, the way to optimize it properly to have no conditionals, is you have to take a continuous function that resembles your desired outcome (which in the problem in question isn't step() but the thing it was used for!), and tune its parameters to get as close as it can to your target. I.e. typically you'd pick some polynomial and run the standard iterative approximation on it. Then you'd just have an f(x) that has no branching, just a bunch of extra additions and multiplications and some "weirdly specific" constants. Where I don't get the author is in insisting that conditional move isn't "branching". I don't see how that would be except in some special cases, where lack of branching is well-known but very special implementation detail - like where the author says: > also note that the abs() call does not become a GPU instruction and instead becomes an instruction modifier, which is free. That's because we standardized on two's complement representation for ints, which has the convenient quality of isolating sign as the most significant bit, and for floats the representation (IEEE-754) was just straight up designed to achieve the same. So in both cases, abs() boils down to unconditionally setting the most significant bit to 0 - or, equivalently, masking it off for the instruction that's reading it. step() isn't like that, nor any other arbitrary ternary operation construct, and nor is - as far as I know - a conditional move instruction. As for where I don't get 'ttoinou: > How are we supposed to know what OpenGL functions are emulated rather than calling GPU primitives The basics like abs() and sqrt() and basic trigonometry are standard knowledge, the rest... does it even matter? step() obviously has to branch somewhere; whether you do it yourself, let a library do it, or let the hardware do it, shouldn't change the fundamental nature.
- ttoinou 2y ago
  It kinda does when you’re wondering what’s going on in backstage and working with shaders on multiple OS, drivers and hardware. Now, the way to optimize it properly to have no conditionals, is you have to take a continuous I suspect that we shaders authors really like Clean Math and that’s also why we like to think such “optimizations” with the step function is a nice modification :-)
- mymoomin 2y ago
  A "branch" here is a conditional jump. This has the issues the article mentions, which branchless programming avoids: Again, there is no branching - the instruction pointer isn't manipulated, there's no branch prediction involved, no instruction cache to invalidate, no nothing. This has nothing to do with whether the behaviour of some instruction depends on its arguments. Looking at the Microsoft compiler output from the article, the iadd (signed add) instruction will get different results depending on its arguments, and the movc (conditional move) will store different values depending on its arguments, but after each the instruction pointer will just move onto the next instruction, so there are no branches.
- burch45 2y ago
  Branching is different instruction paths, so it requires reading the instructions from different memory that causes a delay jumping to those new instructions rather than plowing ahead on the current stream of instructions. So a conditional jump is a branch but a conditional move is just an instruction that moves one of two values into a register but doesn’t affect what code is executed next.
- dahart 2y ago
  > the meaning I grew up with, which is that a conditional is a branch A conditional jump is a branch. But a branch has always had a different meaning than a generic “conditional”. There are conditional instructions that don’t jump, e.g. CMP, and the distinction is very important. Branch or conditional jump means the PC can be set to something other than ‘next instruction’. A conditional, such a conditional select or conditional move, one that doesn’t change the PC, is not a branch. > take a continuous function […] Then you’d just have an f(x) that has no branching One can easily implement conditional functions without branching. You can use a compare instruction followed by a Heaviside function on the result, evaluate both sides of the result, and sum it up with a 2D dot product (against the compare result and its negation). That is occasionally (but certainly not always) faster on a GPU than using if/else, but only if the compiler is otherwise going to produce real branch instructions.
SideQuark 2y ago
I’ve never seen a GPU with special primitives for any functions than you’d see in pc style assembly. Every time I’ve looked at a decompiled shader, it’s always been pretty much what you think of in C. Aldo specs like OpenGL specify many intrinsic behavior, which is then implemented as the spec, using standard assembly instructions. Find an online site that decompiles to various architectures.
account42 2y ago
Why are you supposed to know? Because you care about performance? step being implemented as a libray function on top of a conditional doesn't really say anything about its performance vs being a dedicated instruction. Don't worry about the implementation. Because you are curious about GPU architectures? Look at disassembly, (open source) driver code (including LLVM) and/or ISA documentation.

Const-me 2y ago

The only way is do what OP did – compile your shader, disassemble, and read the assembly. I do that quite often with my HLSL shaders, learned a lot about that virtual instruction set. For example, it’s interesting GPUs have instruction sincos, but inverse trigonometry is emulated while compiling.

Waterluvian 2y ago

This is a great question that I see everywhere in programming and I think it is core to why you measure first when optimizing. You generally shouldn’t know or care how a built in is implemented. If do care, you’re probably thinking about optimization. At that point the answer is “measure and find out what works better.”

TeMPOraL 2y ago

EDIT: I see my source of confusion must be that "branch" must have a well-understood hardware-specific meaning that goes beyond the meaning I grew up with, which is that a conditional is a branch, because the path control takes (at the machine code level) is chosen at runtime. This makes a conditional jump a branch by definition. > How are we supposed to know what OpenGL functions are emulated rather than calling GPU primitives? To me the problem was obvious, but then again I'm having trouble with both your and author's statements about it. The problem I saw was, obviously by going for a step() function, people aren't turning logic into arithmetic, they're just hiding logic in a library function call. Just because step() is a built-in or something you'd find used in mathematical paper doesn't mean anything; the definition of step() in mathematics is literally a conditional too. Now, the way to optimize it properly to have no conditionals, is you have to take a continuous function that resembles your desired outcome (which in the problem in question isn't step() but the thing it was used for!), and tune its parameters to get as close as it can to your target. I.e. typically you'd pick some polynomial and run the standard iterative approximation on it. Then you'd just have an f(x) that has no branching, just a bunch of extra additions and multiplications and some "weirdly specific" constants. Where I don't get the author is in insisting that conditional move isn't "branching". I don't see how that would be except in some special cases, where lack of branching is well-known but very special implementation detail - like where the author says: > also note that the abs() call does not become a GPU instruction and instead becomes an instruction modifier, which is free. That's because we standardized on two's complement representation for ints, which has the convenient quality of isolating sign as the most significant bit, and for floats the representation (IEEE-754) was just straight up designed to achieve the same. So in both cases, abs() boils down to unconditionally setting the most significant bit to 0 - or, equivalently, masking it off for the instruction that's reading it. step() isn't like that, nor any other arbitrary ternary operation construct, and nor is - as far as I know - a conditional move instruction. As for where I don't get 'ttoinou: > How are we supposed to know what OpenGL functions are emulated rather than calling GPU primitives The basics like abs() and sqrt() and basic trigonometry are standard knowledge, the rest... does it even matter? step() obviously has to branch somewhere; whether you do it yourself, let a library do it, or let the hardware do it, shouldn't change the fundamental nature.

ttoinou 2y ago
It kinda does when you’re wondering what’s going on in backstage and working with shaders on multiple OS, drivers and hardware. Now, the way to optimize it properly to have no conditionals, is you have to take a continuous I suspect that we shaders authors really like Clean Math and that’s also why we like to think such “optimizations” with the step function is a nice modification :-)
mymoomin 2y ago
A "branch" here is a conditional jump. This has the issues the article mentions, which branchless programming avoids: Again, there is no branching - the instruction pointer isn't manipulated, there's no branch prediction involved, no instruction cache to invalidate, no nothing. This has nothing to do with whether the behaviour of some instruction depends on its arguments. Looking at the Microsoft compiler output from the article, the iadd (signed add) instruction will get different results depending on its arguments, and the movc (conditional move) will store different values depending on its arguments, but after each the instruction pointer will just move onto the next instruction, so there are no branches.
burch45 2y ago
Branching is different instruction paths, so it requires reading the instructions from different memory that causes a delay jumping to those new instructions rather than plowing ahead on the current stream of instructions. So a conditional jump is a branch but a conditional move is just an instruction that moves one of two values into a register but doesn’t affect what code is executed next.
dahart 2y ago
> the meaning I grew up with, which is that a conditional is a branch A conditional jump is a branch. But a branch has always had a different meaning than a generic “conditional”. There are conditional instructions that don’t jump, e.g. CMP, and the distinction is very important. Branch or conditional jump means the PC can be set to something other than ‘next instruction’. A conditional, such a conditional select or conditional move, one that doesn’t change the PC, is not a branch. > take a continuous function […] Then you’d just have an f(x) that has no branching One can easily implement conditional functions without branching. You can use a compare instruction followed by a Heaviside function on the result, evaluate both sides of the result, and sum it up with a 2D dot product (against the compare result and its negation). That is occasionally (but certainly not always) faster on a GPU than using if/else, but only if the compiler is otherwise going to produce real branch instructions.

SideQuark 2y ago

I’ve never seen a GPU with special primitives for any functions than you’d see in pc style assembly. Every time I’ve looked at a decompiled shader, it’s always been pretty much what you think of in C. Aldo specs like OpenGL specify many intrinsic behavior, which is then implemented as the spec, using standard assembly instructions. Find an online site that decompiles to various architectures.

account42 2y ago

Why are you supposed to know? Because you care about performance? step being implemented as a libray function on top of a conditional doesn't really say anything about its performance vs being a dedicated instruction. Don't worry about the implementation. Because you are curious about GPU architectures? Look at disassembly, (open source) driver code (including LLVM) and/or ISA documentation.