4 ms·

Thanks, and thanks for the other long reply :) One thing that confuses me: we were talking about "compiled code", then your answer jumps to talking about shade

Thanks, and thanks for the other long reply :) One thing that confuses me: we were talking about "compiled code", then your answer jumps to talking about shaders. Can all OpenGL code be represented as shaders? Or is the OpenGL code needing hardware-specific compilation shaders only? When the above poster said that Mesa "compiles code from OpenGL to some target backend", were they referring to shaders already? Said differently: if I import a C GL lib and use it to make a call to draw a triangle, is this a shader, and will Mesa do any hardware-specific compilation? Or is this compilation only happening when shaders are involved? I guess I'm confused about what shaders exactly are, and when they are used. I think I understand them as programs able to fiddle at each frame with how stuff (surface, pixels) is rendered, but for example, geometry (putting triangles in space) is OpenGL's job even though it isn't about shaders, is it? By the way, do you (or other passerbys) have courses recommendations to study this? (ideally interactive / tutorial-like / with exercises)

raphlinus 6y ago
This is a pretty deep topic. The best resource I've found is: https://fgiesen.wordpress.com/2011/07/09/a-trip-through-the-graphics-pipeline-2011-index/ https://fgiesen.wordpress.com/2011/07/09/a-trip-through-the-... I'll also do my best to answer your questions, though I won't be able to do them justice in this space. A pretty good model for GPUs is remote procedure calls. When you want to do something like draw some triangles, you call a function in the GL API on the client side, and what the driver actually does under the hood is serialize the parameters of that call into some binary sequence, then at some point that "command buffer" is uploaded to the GPU, and a combination of hardware and software on the GPU side decodes it and uses it to set up the hardware pipeline that actually draws a triangle. There's a lot more to it than that, obviously. A very big deal is that if you did an actual RPC for every triangle draw, it would be hopelessly inefficient. So a large part of what OpenGL drivers do is use heuristics for batching up a bunch of calls into one actual request. In OpenGL, the details of that batching, and the way the command buffer is encoded, are completely hidden from the application. Shaders are basically small pieces of code that get run on the GPU hardware as part of the rendering pipeline. A vertex shader is a program that gets run every vertex in a mesh, and a fragment shader is another program that gets run for every fragment (pixel) that gets rasterized. There are other shaders that are run for other tasks, but those are the two biggies. In the early days of OpenGL, lighting calculations and so on were done by "fixed function" hardware, but these days it's programmable. In classic OpenGL, you call a function to compile a shader from a string containing GLSL source code, and the driver compiles it to machine language for the GPU hardware. You can use Radeon Shader Analyzer (available online at http://shader-playground.timjones.io/ http://shader-playground.timjones.io/) to see what that assembly language looks like. Even just to understand OpenGL better, it might make sense to learn about Vulkan. A good (though somewhat daunting) resource is "API without Secrets: Introduction to Vulkan": https://software.intel.com/content/www/us/en/develop/articles/api-without-secrets-introduction-to-vulkan-part-1.html https://software.intel.com/content/www/us/en/develop/article... . In Vulkan, the batching and other aspects of resource management are much more explicit and under control of the application, though other details are still abstracted by the driver. For example, there is no one standard binary format for encoding command lists. Also, instead of the driver compiling shaders all the way from source, the application is responsible for compiling the shader into an intermediate language (SPIR-V), then the driver compiles that to the actual GPU machine language. There are some other low level GPU resources here: https://raphlinus.github.io/gpu/2020/02/12/gpu-resources.html https://raphlinus.github.io/gpu/2020/02/12/gpu-resources.htm... Best of luck, I find GPUs to be a fascinating journey!
- ronjouch 6y ago
  Awesome. Thanks for the detailed answer and links :)
wtallis 6y ago
> we were talking about "compiled code", then your answer jumps to talking about shaders. Can all OpenGL code be represented as shaders? Or is the OpenGL code needing hardware-specific compilation shaders only? When the above poster said that Mesa "compiles code from OpenGL to some target backend", were they referring to shaders already? OpenGL code consists of both code running on the CPU, and shader programs running on the GPU. Most of the heavy number crunching is handled by the shader programs and the CPU acts mostly as the control plane, handling resource allocation and dispatching data and work items to the GPU. Early versions of OpenGL predated programmable GPUs and had no concept of shader programs. Instead, various GPU mode registers were programmed to set up the rendering pipeline, which was then fed vertex and texture data to render. Modern OpenGL largely abandons that beginner-friendly immediate mode paradigm and requires you to provide shader programs and do various other setup work to get anything on screen; you can't simply make a DrawTriangle kind of function call on the CPU. Newer APIs like Vulkan strip away even more abstractions.
- ronjouch 6y ago
  Thanks for the clarification! On "Newer APIs like Vulkan strip away even more abstractions", another comment links to multi-part article https://software.intel.com/content/www/us/en/develop/articles/api-without-secrets-introduction-to-vulkan-part-1.html https://software.intel.com/content/www/us/en/develop/article... , and uuuuh yes indeed it's a lot of setup code to draw a triangle :D
LukeShu 6y ago
(You're making me remember things I haven't thought about in a long time :) my apologies if some details are off) The simple definition of a shader is "a shader is a program that runs on the GPU instead of the CPU" (or at least is intended to run on the GPU instead of the CPU; if you write a shader that makes use of a feature your GPU doesn't support, Mesa will actually use LLVM to compile it to run on the CPU instead. Performance will be terrible, but at least the program will run). The compilation is only involved when shaders are involved, though on many graphics cards the driver will implement various apparently-non-shader things as shaders under the hood. In the early days of OpenGL, it had what we now call a "fixed pipeline" where the GPU isn't very configurable, you feed it geometry and textures, configure a few knobs, and then let it run. You could feed it geometry (either polynomial curves or polygon vertices), then the first stage of the pipeline would approximate all the curves as polygon vertices. Then the second stage would take that, and you'd feed it some lighting information and feed it some textures, and would render each polygon to an individual fragment. Then the third stage would composite those fragments together in to the framebuffer. Which it would then presumably display on your screen. You didn't quite have to do things exactly that way, but it was a set of large-ish "blocks" that weren't terribly configurable. And then very creative people would say "I wish I could change this little aspect of how it processes vertices" or "... of how it processes fragments". And so some hardware vendor would implement another knob on their GPU to configure a specific aspect of one of the stages in the pipeline, make up their own OpenGL extension to configure that knob, and if the OpenGL Architecture Review Board liked that extension, it might become a core part of the next minor release of OpenGL. So it progressed toward a "configurable pipeline". Then they ended up with enough knobs that they said "Let's just let them write code that will run on the GPU to process the vertices and fragments, instead of adding more and more knobs". So in OpenGL 2.0, we got vertex shaders and fragment shaders. The OpenGL spec specified a C-like language, GLSL, which the graphics card driver would compile in to code that would run directly on the GPU. And so you could run your own "vertex shader" instead of using the vertex-processing behavior that was baked-in to the graphics card in OpenGL 1. You could write the original baked-in behavior as a shader; and so that's what is happening under-the-hood in the drivers for most modern graphics cards. So now we have a "programmable pipeline". At this point they were called shaders because largely what they allowed you to accomplish was, well, fancy shading. And then very creative people wanted to be able to specify the behavior at more parts of the pipeline, and so OpenGL grew geometry shaders and pixel shaders and whatnot. And so that's the general direction that OpenGL is going, carving out more and more pieces of what used to be "part of" OpenGL and letting you replace it with your own shader code. And then we even got compute shaders that don't even have anything to do with graphics, but let you run general-purpose computation on the GPU instead of the CPU. So the word "shader" a bit of a misnomer these days. I got a lot out of CS 334 with Voicu Popescu at Purdue, but I'm not sure if much course material from that is online. Also, I find the OpenGL specs to be surprisingly approachable, but I spend a lot of time reading specs so maybe that's just me.

raphlinus 6y ago

This is a pretty deep topic. The best resource I've found is: https://fgiesen.wordpress.com/2011/07/09/a-trip-through-the-graphics-pipeline-2011-index/ https://fgiesen.wordpress.com/2011/07/09/a-trip-through-the-... I'll also do my best to answer your questions, though I won't be able to do them justice in this space. A pretty good model for GPUs is remote procedure calls. When you want to do something like draw some triangles, you call a function in the GL API on the client side, and what the driver actually does under the hood is serialize the parameters of that call into some binary sequence, then at some point that "command buffer" is uploaded to the GPU, and a combination of hardware and software on the GPU side decodes it and uses it to set up the hardware pipeline that actually draws a triangle. There's a lot more to it than that, obviously. A very big deal is that if you did an actual RPC for every triangle draw, it would be hopelessly inefficient. So a large part of what OpenGL drivers do is use heuristics for batching up a bunch of calls into one actual request. In OpenGL, the details of that batching, and the way the command buffer is encoded, are completely hidden from the application. Shaders are basically small pieces of code that get run on the GPU hardware as part of the rendering pipeline. A vertex shader is a program that gets run every vertex in a mesh, and a fragment shader is another program that gets run for every fragment (pixel) that gets rasterized. There are other shaders that are run for other tasks, but those are the two biggies. In the early days of OpenGL, lighting calculations and so on were done by "fixed function" hardware, but these days it's programmable. In classic OpenGL, you call a function to compile a shader from a string containing GLSL source code, and the driver compiles it to machine language for the GPU hardware. You can use Radeon Shader Analyzer (available online at http://shader-playground.timjones.io/ http://shader-playground.timjones.io/) to see what that assembly language looks like. Even just to understand OpenGL better, it might make sense to learn about Vulkan. A good (though somewhat daunting) resource is "API without Secrets: Introduction to Vulkan": https://software.intel.com/content/www/us/en/develop/articles/api-without-secrets-introduction-to-vulkan-part-1.html https://software.intel.com/content/www/us/en/develop/article... . In Vulkan, the batching and other aspects of resource management are much more explicit and under control of the application, though other details are still abstracted by the driver. For example, there is no one standard binary format for encoding command lists. Also, instead of the driver compiling shaders all the way from source, the application is responsible for compiling the shader into an intermediate language (SPIR-V), then the driver compiles that to the actual GPU machine language. There are some other low level GPU resources here: https://raphlinus.github.io/gpu/2020/02/12/gpu-resources.html https://raphlinus.github.io/gpu/2020/02/12/gpu-resources.htm... Best of luck, I find GPUs to be a fascinating journey!

ronjouch 6y ago
Awesome. Thanks for the detailed answer and links :)

wtallis 6y ago

> we were talking about "compiled code", then your answer jumps to talking about shaders. Can all OpenGL code be represented as shaders? Or is the OpenGL code needing hardware-specific compilation shaders only? When the above poster said that Mesa "compiles code from OpenGL to some target backend", were they referring to shaders already? OpenGL code consists of both code running on the CPU, and shader programs running on the GPU. Most of the heavy number crunching is handled by the shader programs and the CPU acts mostly as the control plane, handling resource allocation and dispatching data and work items to the GPU. Early versions of OpenGL predated programmable GPUs and had no concept of shader programs. Instead, various GPU mode registers were programmed to set up the rendering pipeline, which was then fed vertex and texture data to render. Modern OpenGL largely abandons that beginner-friendly immediate mode paradigm and requires you to provide shader programs and do various other setup work to get anything on screen; you can't simply make a DrawTriangle kind of function call on the CPU. Newer APIs like Vulkan strip away even more abstractions.

ronjouch 6y ago
Thanks for the clarification! On "Newer APIs like Vulkan strip away even more abstractions", another comment links to multi-part article https://software.intel.com/content/www/us/en/develop/articles/api-without-secrets-introduction-to-vulkan-part-1.html https://software.intel.com/content/www/us/en/develop/article... , and uuuuh yes indeed it's a lot of setup code to draw a triangle :D

LukeShu 6y ago

(You're making me remember things I haven't thought about in a long time :) my apologies if some details are off) The simple definition of a shader is "a shader is a program that runs on the GPU instead of the CPU" (or at least is intended to run on the GPU instead of the CPU; if you write a shader that makes use of a feature your GPU doesn't support, Mesa will actually use LLVM to compile it to run on the CPU instead. Performance will be terrible, but at least the program will run). The compilation is only involved when shaders are involved, though on many graphics cards the driver will implement various apparently-non-shader things as shaders under the hood. In the early days of OpenGL, it had what we now call a "fixed pipeline" where the GPU isn't very configurable, you feed it geometry and textures, configure a few knobs, and then let it run. You could feed it geometry (either polynomial curves or polygon vertices), then the first stage of the pipeline would approximate all the curves as polygon vertices. Then the second stage would take that, and you'd feed it some lighting information and feed it some textures, and would render each polygon to an individual fragment. Then the third stage would composite those fragments together in to the framebuffer. Which it would then presumably display on your screen. You didn't quite have to do things exactly that way, but it was a set of large-ish "blocks" that weren't terribly configurable. And then very creative people would say "I wish I could change this little aspect of how it processes vertices" or "... of how it processes fragments". And so some hardware vendor would implement another knob on their GPU to configure a specific aspect of one of the stages in the pipeline, make up their own OpenGL extension to configure that knob, and if the OpenGL Architecture Review Board liked that extension, it might become a core part of the next minor release of OpenGL. So it progressed toward a "configurable pipeline". Then they ended up with enough knobs that they said "Let's just let them write code that will run on the GPU to process the vertices and fragments, instead of adding more and more knobs". So in OpenGL 2.0, we got vertex shaders and fragment shaders. The OpenGL spec specified a C-like language, GLSL, which the graphics card driver would compile in to code that would run directly on the GPU. And so you could run your own "vertex shader" instead of using the vertex-processing behavior that was baked-in to the graphics card in OpenGL 1. You could write the original baked-in behavior as a shader; and so that's what is happening under-the-hood in the drivers for most modern graphics cards. So now we have a "programmable pipeline". At this point they were called shaders because largely what they allowed you to accomplish was, well, fancy shading. And then very creative people wanted to be able to specify the behavior at more parts of the pipeline, and so OpenGL grew geometry shaders and pixel shaders and whatnot. And so that's the general direction that OpenGL is going, carving out more and more pieces of what used to be "part of" OpenGL and letting you replace it with your own shader code. And then we even got compute shaders that don't even have anything to do with graphics, but let you run general-purpose computation on the GPU instead of the CPU. So the word "shader" a bit of a misnomer these days. I got a lot out of CS 334 with Voicu Popescu at Purdue, but I'm not sure if much course material from that is online. Also, I find the OpenGL specs to be surprisingly approachable, but I spend a lot of time reading specs so maybe that's just me.