4 ms·

Has it really been 9 years since I started working on Ubershaders? I'm a little surprised no better solution has come along. Vulkan didn't even exist back then

by phire 2y ago

Has it really been 9 years since I started working on Ubershaders? I'm a little surprised no better solution has come along. Vulkan didn't even exist back then (and DirectX 12 had only just released) but instead of making things better, it digs it's feet even deeper into the assumption that all shaders will be known ahead of time (resulting in long "shader recompilation" dialogs on startup on many games). I've been tempted to build my own fast shader compiler into Dolphin for many common GPU architectures. Hell, it wouldn't even be a proper compiler, more of a templated emitter as all shaders fit a pattern. Register allocation and scheduling could all be pre-calculated. But that would be even more insane than ubershaders, as it would be one backend per gpu arch. And some drivers (like Nvidia) don't provide a way to inject pre-compiled shader binaries. On the positive side, ubershaders do solve the problem, and modern GPU drivers do a much better job at accepting ubershaders than they did 9 years ago. Though that's primarily because (as far as I'm aware) examples of Dolphin's ubershader have made their way into every single shader compiler test suite.

Miksel12 2y ago
Don't you think intermediate representation like SPIR-V would suffice in mostly eliminating stutter? Yuzu used that and shader stutter seemed to be minimal and I can image that the shaders generated by Yuzu are much more complex than Dolphin.
- phire 2y ago
  The only step that SPIR-V replaces is parsing the GLSL to an AST tree, and that's only a small part of the total time to compile a shader. Usually the bottleneck is Register allocation or scheduling. Back when Vulkan was developed, there were a bunch of OpenGL drivers out there which had random AST parsing bugs (Dolphin even has a bunch of workarounds for them); So a large chunk of the motivation for SPIR-V was avoiding the need for every driver to implement their own GLSL parser and the associated bugs. The problem for Dolphin is not the complexity of the shader, but the quantity. Shaders in modern games are usually written manually (or authored in a shader node editor by an artist), so it's rare for a game to have more than a few thousand total. Better games might only have a few dozen for the entire game. But because gamecube/Wii games configure the TEV pixel pipeline though a dynamic API, some games use that API in a pattern where Dolphin can find itself generating hundreds of shaders per second. Some games even manage to generate new shaders continually as you play, because they append junk state to their pixel pixeline state which dolphin doesn't detect as a duplicate.
  - Jasper_ 2y ago
    Shaving off the frontend costs is not going to be nothing. I don't know if Dolphin is still using FXC/D3DCompile or if they've switched to DXC, but FXC is infamously slow, even for very simple shaders. Dolphin's shaders are medium-complexity IIRC, so I'd expect removing the frontend to be a decent win. The driver PSO compilers aren't amazing but they're also not terrible. Most games do some form of hash-n-cache for PSO compilation and while stutters are still an issue, it's not the worst in the world. With the frontend gone, I'd expect ~50 shaders per second to be roughly stutter-free. Being smarter about specialization is probably a good idea -- having a blend between "GPU interpreter" and "full specialized pipeline" is where I think you should head. Several of the weirder TEV features could probably be moved to branching on dynamic buffer contents. Not to mention using newer features like bindless to merge draw calls. I always wanted to do that but got too busy before I stopped working on Dolphin :)
    - phire 2y ago
      > so I'd expect removing the frontend to be a decent win I did some testing before working on ubershaders, and my modified build which cached the bytecode output of FXC/D3DCompile (whatever dolphin was using at the time) didn't reduce the stuttering by enough to be worth the effort of optimising the frontend. My conclusion was that it's simply wasn't worth any effort to optimise for slightly smaller stutters, as they were still very perceivable to users. And Hybrid Ubershaders can hide any compile delays without any issue. And this testing was with FXC/D3DCompile which does a bunch of optimisations. The fact that SPIR-V comes in (potentially) unoptimised means any vulkan compiler has to send it though all optimisation passes. Though I have been very tempted to do dead code removal before submitting the shaders, partly to make the shaders more readable to humans and partly to reduce the amount of code going though the various compiler passes. > Being smarter about specialization is probably a good idea -- having a blend between "GPU interpreter" and "full specialized pipeline" is where I think you should head. Yeah, that was always next on the list. Start with just ubershaders and then incrementally specialise on a background thread for the correct balance of shaders. Dolphin's current specialised shaders are no-where near fully specialized. Need to go further by baking some of the constants and lookup textures into the shader.
naikrovek 2y ago
I still don’t understand why you didn’t use the precompiled shaders packed with the games… you’re emulating the GameCube or Wii GPU, and it’s never going to change, and the games provide precompiled shaders.
- mappu 2y ago
  They're precompiled for the console GPU architecture, not the PC architecture, so they can't be used directly and still need to be emulated - I think those precompiled shaders are the input to the ubershader.
  - naikrovek 2y ago
    The GAMES THEMSELVES are precompiled for the PowerPC architecture, not the PC architecture, though. That didn’t stop anyone from creating Dolphin. GPUs (I’m told) have far fewer instructions to emulate than a CPU, so I’d think that low level emulation of the Flipper shaders would be no trouble. Can’t translate or transpile them to PC GPUs though because those instruction sets are somewhat secret, I think. I know nothing about this stuff but I am a developer so perhaps I know enough to ask the most stupid questions possible. It’s gotta be a performance thing, why they didn’t emulate Flipper at a low enough level to use the precompiled shaders directly.
    - saagarjha 2y ago
      Yes, that’s exactly the point though. This is the same question as why you can’t emulate a game by precompiling its code, and this doesn’t work because that information isn’t available until you try to run the game. That’s why Dolphin has an interpreter/JIT.
      - HideousKojima 2y ago
        >This is the same question as why you can’t emulate a game by precompiling its code, and this doesn’t work because that information isn’t available until you try to run the game. I mean technically you can, but it generally requires a bunch of inefficient jump tables, or alternatively a way to fall back to an interpreter or JIT for self modifying code.
oppositelock 2y ago
Very cool work! I had to solve a similar problem years ago, during the transition from fixed function to shaders, when shaders weren't as fast or powerful as today. We started out with an ubershader approximating the DX9/OpenGL 1.2 fixed functions, but that was too slow. People in those days thought of rendering state being stored in a tree, like the transform hierarchy, and you ended up having unpredictable state at the leaf nodes, sometimes leading to a very high permutation of possible states. At the time, I decomposed all possible pipeline state into atomic pieces, eg, one light, fog function, texenv, etc. These were all annotated with inputs and outputs, and based on the state graph traversal, we'd generate a minimal shader for each particular material automatically, while giving old tools the semblance of being able to compose fixed function states. As for you, doing this on-demand resulted in stuttering, but a single game only has so many possible states - from what I've seen, it's on the order of a few hundred to a few thousand. Once all shaders are generated, you can cache the generated shaders and compile them all at startup time. I wonder if something like this would work for emulating a Gamecube. You can definitely compute a signature for a game executable, and as you encounter new shaders, you can associate them with the game. Over time, you'll discover all the possible state, and if it's cached, you can compile all the cached shaders at startup. Anyhow, fun stuff. I used to love work like this. I've implemented 3DFx's Glide API on top of DX ages ago to play Voodoo games on my Nvidia cards, and contributed some code to an N64 emulator named UltraHLE.
- M4v3R 2y ago
  > contributed some code to an N64 emulator named UltraHLE That's a blast from the past, I distinctly remember reading up about UltraHLE way back when and then trying it our and for the first time being able to play Ocarina of Time on my middle class PC with almost no issues, that was magical.
Akronymus 2y ago
>On the positive side, ubershaders do solve the problem, and modern GPU drivers do a much better job at accepting ubershaders than they did 9 years ago. Though that's primarily because (as far as I'm aware) examples of Dolphin's ubershader have made their way into every single shader compiler test suite. How'd that come to be? Just interesting code for test suites or did you guys advocate for it to be included?

by phire 2y ago

Miksel12 2y ago

Don't you think intermediate representation like SPIR-V would suffice in mostly eliminating stutter? Yuzu used that and shader stutter seemed to be minimal and I can image that the shaders generated by Yuzu are much more complex than Dolphin.

phire 2y ago
The only step that SPIR-V replaces is parsing the GLSL to an AST tree, and that's only a small part of the total time to compile a shader. Usually the bottleneck is Register allocation or scheduling. Back when Vulkan was developed, there were a bunch of OpenGL drivers out there which had random AST parsing bugs (Dolphin even has a bunch of workarounds for them); So a large chunk of the motivation for SPIR-V was avoiding the need for every driver to implement their own GLSL parser and the associated bugs. The problem for Dolphin is not the complexity of the shader, but the quantity. Shaders in modern games are usually written manually (or authored in a shader node editor by an artist), so it's rare for a game to have more than a few thousand total. Better games might only have a few dozen for the entire game. But because gamecube/Wii games configure the TEV pixel pipeline though a dynamic API, some games use that API in a pattern where Dolphin can find itself generating hundreds of shaders per second. Some games even manage to generate new shaders continually as you play, because they append junk state to their pixel pixeline state which dolphin doesn't detect as a duplicate.
- Jasper_ 2y ago
  Shaving off the frontend costs is not going to be nothing. I don't know if Dolphin is still using FXC/D3DCompile or if they've switched to DXC, but FXC is infamously slow, even for very simple shaders. Dolphin's shaders are medium-complexity IIRC, so I'd expect removing the frontend to be a decent win. The driver PSO compilers aren't amazing but they're also not terrible. Most games do some form of hash-n-cache for PSO compilation and while stutters are still an issue, it's not the worst in the world. With the frontend gone, I'd expect ~50 shaders per second to be roughly stutter-free. Being smarter about specialization is probably a good idea -- having a blend between "GPU interpreter" and "full specialized pipeline" is where I think you should head. Several of the weirder TEV features could probably be moved to branching on dynamic buffer contents. Not to mention using newer features like bindless to merge draw calls. I always wanted to do that but got too busy before I stopped working on Dolphin :)
  - phire 2y ago
    > so I'd expect removing the frontend to be a decent win I did some testing before working on ubershaders, and my modified build which cached the bytecode output of FXC/D3DCompile (whatever dolphin was using at the time) didn't reduce the stuttering by enough to be worth the effort of optimising the frontend. My conclusion was that it's simply wasn't worth any effort to optimise for slightly smaller stutters, as they were still very perceivable to users. And Hybrid Ubershaders can hide any compile delays without any issue. And this testing was with FXC/D3DCompile which does a bunch of optimisations. The fact that SPIR-V comes in (potentially) unoptimised means any vulkan compiler has to send it though all optimisation passes. Though I have been very tempted to do dead code removal before submitting the shaders, partly to make the shaders more readable to humans and partly to reduce the amount of code going though the various compiler passes. > Being smarter about specialization is probably a good idea -- having a blend between "GPU interpreter" and "full specialized pipeline" is where I think you should head. Yeah, that was always next on the list. Start with just ubershaders and then incrementally specialise on a background thread for the correct balance of shaders. Dolphin's current specialised shaders are no-where near fully specialized. Need to go further by baking some of the constants and lookup textures into the shader.

naikrovek 2y ago

I still don’t understand why you didn’t use the precompiled shaders packed with the games… you’re emulating the GameCube or Wii GPU, and it’s never going to change, and the games provide precompiled shaders.

mappu 2y ago
They're precompiled for the console GPU architecture, not the PC architecture, so they can't be used directly and still need to be emulated - I think those precompiled shaders are the input to the ubershader.
- naikrovek 2y ago
  The GAMES THEMSELVES are precompiled for the PowerPC architecture, not the PC architecture, though. That didn’t stop anyone from creating Dolphin. GPUs (I’m told) have far fewer instructions to emulate than a CPU, so I’d think that low level emulation of the Flipper shaders would be no trouble. Can’t translate or transpile them to PC GPUs though because those instruction sets are somewhat secret, I think. I know nothing about this stuff but I am a developer so perhaps I know enough to ask the most stupid questions possible. It’s gotta be a performance thing, why they didn’t emulate Flipper at a low enough level to use the precompiled shaders directly.
  - saagarjha 2y ago
    Yes, that’s exactly the point though. This is the same question as why you can’t emulate a game by precompiling its code, and this doesn’t work because that information isn’t available until you try to run the game. That’s why Dolphin has an interpreter/JIT.
    - HideousKojima 2y ago
      >This is the same question as why you can’t emulate a game by precompiling its code, and this doesn’t work because that information isn’t available until you try to run the game. I mean technically you can, but it generally requires a bunch of inefficient jump tables, or alternatively a way to fall back to an interpreter or JIT for self modifying code.

oppositelock 2y ago

Very cool work! I had to solve a similar problem years ago, during the transition from fixed function to shaders, when shaders weren't as fast or powerful as today. We started out with an ubershader approximating the DX9/OpenGL 1.2 fixed functions, but that was too slow. People in those days thought of rendering state being stored in a tree, like the transform hierarchy, and you ended up having unpredictable state at the leaf nodes, sometimes leading to a very high permutation of possible states. At the time, I decomposed all possible pipeline state into atomic pieces, eg, one light, fog function, texenv, etc. These were all annotated with inputs and outputs, and based on the state graph traversal, we'd generate a minimal shader for each particular material automatically, while giving old tools the semblance of being able to compose fixed function states. As for you, doing this on-demand resulted in stuttering, but a single game only has so many possible states - from what I've seen, it's on the order of a few hundred to a few thousand. Once all shaders are generated, you can cache the generated shaders and compile them all at startup time. I wonder if something like this would work for emulating a Gamecube. You can definitely compute a signature for a game executable, and as you encounter new shaders, you can associate them with the game. Over time, you'll discover all the possible state, and if it's cached, you can compile all the cached shaders at startup. Anyhow, fun stuff. I used to love work like this. I've implemented 3DFx's Glide API on top of DX ages ago to play Voodoo games on my Nvidia cards, and contributed some code to an N64 emulator named UltraHLE.

M4v3R 2y ago
> contributed some code to an N64 emulator named UltraHLE That's a blast from the past, I distinctly remember reading up about UltraHLE way back when and then trying it our and for the first time being able to play Ocarina of Time on my middle class PC with almost no issues, that was magical.

Akronymus 2y ago

>On the positive side, ubershaders do solve the problem, and modern GPU drivers do a much better job at accepting ubershaders than they did 9 years ago. Though that's primarily because (as far as I'm aware) examples of Dolphin's ubershader have made their way into every single shader compiler test suite. How'd that come to be? Just interesting code for test suites or did you guys advocate for it to be included?