7 ms·

From scratch OpenGL and shaders with raw Xlib

97 points by libcheet 3y ago

bun_terminator 3y ago
As a fellow OpenGL enjoyer, this should really target 4.5 or 4.6. DSA is a gamechanger
- jsheard 3y ago
  Honestly it's hard to make the case for learning OpenGL at all at this point, unless you need to maintain legacy code. It's not a good API, and it's a technical dead-end with Khronos no longer developing it and Apple abandoning it even earlier. While Vulkan has the vertical learning curve problem there's now native WebGPU libaries providing a sane middle ground that follows modern principles without getting bogged down in minutiae like Vulkan does.
  - tempodox 3y ago
    One might argue that getting bogged down in minutiae is not the hallmark of a good API either. Vulkan is not available on Apple HW and Metal is not available elsewhere, while OpenGL is at least somewhat portable.
    - jsheard 3y ago
      That's why I'm saying that people new to graphics should probably start with Dawn or wgpu nowadays. They follow the same shape as the modern native APIs, but in a more streamlined manner, and are portable to all of the major platforms without relying on Apples long-abandoned OpenGL implementation which is missing foundational features like compute shaders. If you go down the Rust route with wgpu then it doesn't even require delving into unsafe code. From that starting point there's a natural progression to tackling raw Vulkan, D3D12 or Metal if you outgrow what the WebGPU libraries are capable of. If you start from OpenGL instead then you have to unlearn all the nonsensical abstractions it teaches you.
    - cogman10 3y ago
      Vulkan is available via MoltenVK. If you have sufficiently low level API that works similarly, then the benefit of two low level APIs is relatively direct translations are possible.
    - pjmlp 3y ago
      Portable only to certain extent, the moment you start junggling extensions it could be a complete different API for all pratical purposes, depending on how much extensions one needs to take care of, and the different semantics across them, or similar extensions from different vendors. Also, OpenGL never really quite made it into game consoles, only in some ways not fully compatible, so if they are a target, one already needs to handle multiple APIs anyway.
      - my123 3y ago
        The Nintendo Switch has a full GL4.6 implementation, and has Vulkan too. But that's an anomaly in the console world. (SW stack provided by NVIDIA)
        pjmlp 3y ago
        It has, but you forgot to mention that the main API that actually exposes everything, is NVN.
  - bun_terminator 3y ago
    OpenGL does a lot of work you'd have to do manually on Vulkan. That's why it's beloved. I use it to write a game currently. It's really half an engine. The danger of deprecation is there though. The API itself is fine IMO, with DSA of course. Without DSA it's a shitty state machine.
    - bitwize 3y ago
      We'll always have Zink, my man.
  - sham1 3y ago
    Then again, it's hard to make a case for not learning OGL. You still need to learn all the concepts whether you'd end up using Vulkan or Metal or whatnot. At worst you'd end up having to use an OpenGL middleware thing in the middle of you and the lower abstraction level API. Like yes, OpenGL is crufty. It's also just not good in many ways, but in many ways it's simple. The lessons learnt from it also informed Khronos on the design of Vulkan, so even for that, it's nice to be able to see just where some of the design decisions come from. Also, not everyone develops for Apple and so their deprecation is less compelling than, say, Khronos officially stopping development on the API ;)
    - jsheard 3y ago
      Part of the problem is that many of the concepts OpenGL teaches you have no bearing on how modern hardware actually works, so you end up having to unlearn bad habits which OpenGLs messy abstractions enable. OpenGL won't teach you to think in terms of PSOs, for example. > Also, not everyone develops for Apple and so their deprecation is less compelling than, say, Khronos officially stopping development on the API ;) Have they not stopped? The last major update to OpenGL was six years ago, around the time Vulkan went public. I recall there initially being talk of OpenGL continuing to be developed alongside Vulkan but that just hasn't happened.
      - winterismute 3y ago
        > Part of the problem is that many of the concepts OpenGL teaches you have no bearing on how modern hardware actually works, so you end up having to unlearn bad habits which OpenGLs messy abstractions enable. OpenGL won't teach you to think in terms of PSOs, for example. While this is true, for somebody who is starting from scratch there is a lot to learn before getting to the level at which thinking in terms of PSO is important, and it can be easier to get there via OpenGL, which btw still teaches you a decent chunk of GPU-friendly patterns (assuming of course we are talking about "modern" OpenGL and not display lists and such...). Also, with a good command of OpenGL, one can start trying to understand and re-implement rendering techniques spanning from deferred/forward+/clustered lighting, the various shadowing techniques and even HW raytracing eg. via the GLSL_NV_ray_tracingextension, which is - in my opinion - the more important side of learning GPU-accelerated rendering.
  - pjmlp 3y ago
    The way to go on native platforms is via middleware, WebGPU will always be constrained by the Web. If you bring extensions to WebGPU for native into the picture, then it is no better than the spaghetti way of dealing with extensions in OpenGL and Vulkan.
    - deleted 3y ago
      [deleted]
    - jsheard 3y ago
      If you ship an application using a native WebGPU library then you're in control of which implementation you use, unlike native APIs where you're at the mercy of the platform. It doesn't particularly matter if you use an extension that exists on Dawn but not on wgpu, if every build of your app comes packaged with Dawn.
      - pjmlp 3y ago
        Just like any middleware engine, no added benefit, with less capable tooling.
        pcwalton 3y ago
        The benefit of wgpu is that you don't have to mess with synchronization the way you do in Vulkan and DX12. This is an enormous productivity booster. Besides, if you pick one native API (say, Vulkan), then you're still going to go through translation layers on the platforms it isn't native to, so wgpu isn't any different. Or you can write multiple backends for different platforms, which multiplies the amount of work you have to do. Either way, saying that wgpu gives you "no added benefit" is silly.
        pjmlp 3y ago
        Just like middleware engines, with much better tooling.
  - somethingsome 3y ago
    I was teaching OpenGL for several years, we did C++ and OpenGL and then, during covid, we switched to web solutions.. What a pain. Not the WebGPU or WebGL per se, but teaching engineers not familiar with Javascript or web development, local servers, CORS, etc.. adds a whole new level of difficulty.
    - jsheard 3y ago
      "WebGPU" is a misnomer, one nice thing about it is that it's not actually married to the web. Dawn, the C++ implementation used in Chrome, and wgpu, the Rust implementation used in Firefox, are both standalone libraries that you can link into a native application and use their portable abstraction without touching Javascript or Electron or any other web tech.
      - somethingsome 3y ago
        It is maintained by the w3c, not used extensively by the businesses that will employ the students, it's easy to go from OpenGL to Vulkan. I can reconsider it and happily use it if I have more arguments in favor of it, but from what I see it doesn't seem groundbreaking and adds a layer of complexity in comparison to OpenGL 4.x without really improving the learning curve or the capabilities.
      - pjmlp 3y ago
        It is driven by Web browsers requirements, anything beyond that are non portable extensions.
  - Pannoniae 3y ago
    The only problem that WebGPU itself (as a desktop API) is outright horrible. For someone coming from OpenGL - with all its warts - the sheer bloat and confusion coming from there is a downgrade. Just see the whole fiasco around WGSL - a shading language which is kind of like SPIR-V but also not really because it's theoretically human-writable, the syntax is worse than anything the C++ committee could dream up, etc.... I don't think there is any good replacement for OpenGL as of now (2024). I hoped there would be higher-level Vulkan wrappers coming out to bridge the gap but even AMD's attempt (V-EZ) got abandoned fairly soon.
    - jsheard 3y ago
      FWIW the native implementations do let you opt-out of the WGSL fiasco by ingesting SPIR-V instead. From what I gather Apple were the ones who lobbied for a new shading language, so Google and Mozilla kept the door open to existing languages in their implementations.
    - Jasper_ 3y ago
      What are you confused about? I'd perhaps say that you'd have the same confusion with any other API. Your confusion is perhaps about unlearning OpenGL rather than learning WebGPU.
  - phendrenad2 3y ago
    How is it a bad API? OpenGL 4.5+ is almost exactly the same as Metal or Vulkan (you just don't have to manage buffers). It's not going anywhere, either, as thousands of games rely on it.
  - bobajeff 3y ago
    I keep seeing WebGPU being pitched as a successor to OpenGL. However as someone who's used it, it's not yet the obvious choice. The native API is still not stable yet. Meanwhile we have things like Bgfx and Diligent that do the same things but are already mature. Maybe webgpu will win because Apple Google and Microsoft will have to have good support for it but today it's not even available in all browsers yet. When it is released it will be missing features the other middleware solutions have and we'll have to wait for WebGPU Next to be available.
nialv7 3y ago
aww, I thought this is going to be about implementing OpenGL functionalities by directly interacting with the GPU and X.
- mananaysiempre 3y ago
  I don’t think that exists (I sure would like for it to), but until it does you could amuse yourself with: - A 500-line (non-OpenGL-compatible) software rasterizer: https://github.com/ssloy/tinyrenderer/wiki https://github.com/ssloy/tinyrenderer/wiki. - A “hello Wayland” app written in C without libwayland or anything else: https://gaultier.github.io/blog/wayland_from_scratch.html https://gaultier.github.io/blog/wayland_from_scratch.html. - A “hello X11” app written in x86-64 assembly(!) without libX11, libxcb, or anything else: https://gaultier.github.io/blog/x11_x64.html https://gaultier.github.io/blog/x11_x64.html.
- dartos 3y ago
  You’re looking for driver programming. IIRC OpenGL is implemented at the driver level. I’m sure you could find a, potentially dated, walkthrough of mesa’s code if you looked hard enough.
Aransentin 3y ago
It should be noted that Xlib has been a compatibility wrapper around XCB – the "modern" X11 client library replacement – for about 15 years now, so it might make more sense to just target that. (The only thing on the top of my head that absolutely requires Xlib is some niche Vulkan stuff; e.g. VK_EXT_acquire_xlib_display unfortunately has no XCB analogue.)
- 0x_rs 3y ago
  It's been a very long while since I played with any of this, and I don't want to rely too much on the widely understood to be lacking documentation, but are Xlib and XCB still not in a very weird situation in which one depends on another for different reasons? In this specific case, at least multiple years ago, glx wouldn't play nice without Xlib, if it would be possible at all without a lot of pain. I don't expect it to have changed for obvious reasons, XCB never being complete and X interest slowly waning. https://xcb.freedesktop.org/opengl/#index3h1 https://xcb.freedesktop.org/opengl/#index3h1
  - qetuoadgjlxvn 3y ago
    These days you can use EGL to get an OpenGL context in a pure XCB application without needing to use Xlib.
- premysl 3y ago
  It should also be noted that in general, Xlib can do a lot more than XCB with a /lot/ less effort and bugs, and has an actual documentation, tutorials, books. Xlib certainly is quirky, in particular around error handling, though.
- engeljohnb 3y ago
  I haven't done anything with X11 in a few years, but last I looked made no sense to do anything with XCB because the documentation is half complete, whereas Xlib has everything you might need documented.
  - toast0 3y ago

97 points by libcheet 3y ago

bun_terminator 3y ago

As a fellow OpenGL enjoyer, this should really target 4.5 or 4.6. DSA is a gamechanger

jsheard 3y ago
Honestly it's hard to make the case for learning OpenGL at all at this point, unless you need to maintain legacy code. It's not a good API, and it's a technical dead-end with Khronos no longer developing it and Apple abandoning it even earlier. While Vulkan has the vertical learning curve problem there's now native WebGPU libaries providing a sane middle ground that follows modern principles without getting bogged down in minutiae like Vulkan does.
- tempodox 3y ago
  One might argue that getting bogged down in minutiae is not the hallmark of a good API either. Vulkan is not available on Apple HW and Metal is not available elsewhere, while OpenGL is at least somewhat portable.
  - jsheard 3y ago
    That's why I'm saying that people new to graphics should probably start with Dawn or wgpu nowadays. They follow the same shape as the modern native APIs, but in a more streamlined manner, and are portable to all of the major platforms without relying on Apples long-abandoned OpenGL implementation which is missing foundational features like compute shaders. If you go down the Rust route with wgpu then it doesn't even require delving into unsafe code. From that starting point there's a natural progression to tackling raw Vulkan, D3D12 or Metal if you outgrow what the WebGPU libraries are capable of. If you start from OpenGL instead then you have to unlearn all the nonsensical abstractions it teaches you.
  - cogman10 3y ago
    Vulkan is available via MoltenVK. If you have sufficiently low level API that works similarly, then the benefit of two low level APIs is relatively direct translations are possible.
  - pjmlp 3y ago
    Portable only to certain extent, the moment you start junggling extensions it could be a complete different API for all pratical purposes, depending on how much extensions one needs to take care of, and the different semantics across them, or similar extensions from different vendors. Also, OpenGL never really quite made it into game consoles, only in some ways not fully compatible, so if they are a target, one already needs to handle multiple APIs anyway.
    - my123 3y ago
      The Nintendo Switch has a full GL4.6 implementation, and has Vulkan too. But that's an anomaly in the console world. (SW stack provided by NVIDIA)
      - pjmlp 3y ago
        It has, but you forgot to mention that the main API that actually exposes everything, is NVN.
- bun_terminator 3y ago
  OpenGL does a lot of work you'd have to do manually on Vulkan. That's why it's beloved. I use it to write a game currently. It's really half an engine. The danger of deprecation is there though. The API itself is fine IMO, with DSA of course. Without DSA it's a shitty state machine.
  - bitwize 3y ago
    We'll always have Zink, my man.
- sham1 3y ago
  Then again, it's hard to make a case for not learning OGL. You still need to learn all the concepts whether you'd end up using Vulkan or Metal or whatnot. At worst you'd end up having to use an OpenGL middleware thing in the middle of you and the lower abstraction level API. Like yes, OpenGL is crufty. It's also just not good in many ways, but in many ways it's simple. The lessons learnt from it also informed Khronos on the design of Vulkan, so even for that, it's nice to be able to see just where some of the design decisions come from. Also, not everyone develops for Apple and so their deprecation is less compelling than, say, Khronos officially stopping development on the API ;)
  - jsheard 3y ago
    Part of the problem is that many of the concepts OpenGL teaches you have no bearing on how modern hardware actually works, so you end up having to unlearn bad habits which OpenGLs messy abstractions enable. OpenGL won't teach you to think in terms of PSOs, for example. > Also, not everyone develops for Apple and so their deprecation is less compelling than, say, Khronos officially stopping development on the API ;) Have they not stopped? The last major update to OpenGL was six years ago, around the time Vulkan went public. I recall there initially being talk of OpenGL continuing to be developed alongside Vulkan but that just hasn't happened.
    - winterismute 3y ago
      > Part of the problem is that many of the concepts OpenGL teaches you have no bearing on how modern hardware actually works, so you end up having to unlearn bad habits which OpenGLs messy abstractions enable. OpenGL won't teach you to think in terms of PSOs, for example. While this is true, for somebody who is starting from scratch there is a lot to learn before getting to the level at which thinking in terms of PSO is important, and it can be easier to get there via OpenGL, which btw still teaches you a decent chunk of GPU-friendly patterns (assuming of course we are talking about "modern" OpenGL and not display lists and such...). Also, with a good command of OpenGL, one can start trying to understand and re-implement rendering techniques spanning from deferred/forward+/clustered lighting, the various shadowing techniques and even HW raytracing eg. via the GLSL_NV_ray_tracingextension, which is - in my opinion - the more important side of learning GPU-accelerated rendering.
- pjmlp 3y ago
  The way to go on native platforms is via middleware, WebGPU will always be constrained by the Web. If you bring extensions to WebGPU for native into the picture, then it is no better than the spaghetti way of dealing with extensions in OpenGL and Vulkan.
  - deleted 3y ago
    [deleted]
  - jsheard 3y ago
    If you ship an application using a native WebGPU library then you're in control of which implementation you use, unlike native APIs where you're at the mercy of the platform. It doesn't particularly matter if you use an extension that exists on Dawn but not on wgpu, if every build of your app comes packaged with Dawn.
    - pjmlp 3y ago
      Just like any middleware engine, no added benefit, with less capable tooling.
      - pcwalton 3y ago
        The benefit of wgpu is that you don't have to mess with synchronization the way you do in Vulkan and DX12. This is an enormous productivity booster. Besides, if you pick one native API (say, Vulkan), then you're still going to go through translation layers on the platforms it isn't native to, so wgpu isn't any different. Or you can write multiple backends for different platforms, which multiplies the amount of work you have to do. Either way, saying that wgpu gives you "no added benefit" is silly.
        pjmlp 3y ago
        Just like middleware engines, with much better tooling.
- somethingsome 3y ago
  I was teaching OpenGL for several years, we did C++ and OpenGL and then, during covid, we switched to web solutions.. What a pain. Not the WebGPU or WebGL per se, but teaching engineers not familiar with Javascript or web development, local servers, CORS, etc.. adds a whole new level of difficulty.
  - jsheard 3y ago
    "WebGPU" is a misnomer, one nice thing about it is that it's not actually married to the web. Dawn, the C++ implementation used in Chrome, and wgpu, the Rust implementation used in Firefox, are both standalone libraries that you can link into a native application and use their portable abstraction without touching Javascript or Electron or any other web tech.
    - somethingsome 3y ago
      It is maintained by the w3c, not used extensively by the businesses that will employ the students, it's easy to go from OpenGL to Vulkan. I can reconsider it and happily use it if I have more arguments in favor of it, but from what I see it doesn't seem groundbreaking and adds a layer of complexity in comparison to OpenGL 4.x without really improving the learning curve or the capabilities.
    - pjmlp 3y ago
      It is driven by Web browsers requirements, anything beyond that are non portable extensions.
- Pannoniae 3y ago
  The only problem that WebGPU itself (as a desktop API) is outright horrible. For someone coming from OpenGL - with all its warts - the sheer bloat and confusion coming from there is a downgrade. Just see the whole fiasco around WGSL - a shading language which is kind of like SPIR-V but also not really because it's theoretically human-writable, the syntax is worse than anything the C++ committee could dream up, etc.... I don't think there is any good replacement for OpenGL as of now (2024). I hoped there would be higher-level Vulkan wrappers coming out to bridge the gap but even AMD's attempt (V-EZ) got abandoned fairly soon.
  - jsheard 3y ago
    FWIW the native implementations do let you opt-out of the WGSL fiasco by ingesting SPIR-V instead. From what I gather Apple were the ones who lobbied for a new shading language, so Google and Mozilla kept the door open to existing languages in their implementations.
  - Jasper_ 3y ago
    What are you confused about? I'd perhaps say that you'd have the same confusion with any other API. Your confusion is perhaps about unlearning OpenGL rather than learning WebGPU.
- phendrenad2 3y ago
  How is it a bad API? OpenGL 4.5+ is almost exactly the same as Metal or Vulkan (you just don't have to manage buffers). It's not going anywhere, either, as thousands of games rely on it.
- bobajeff 3y ago
  I keep seeing WebGPU being pitched as a successor to OpenGL. However as someone who's used it, it's not yet the obvious choice. The native API is still not stable yet. Meanwhile we have things like Bgfx and Diligent that do the same things but are already mature. Maybe webgpu will win because Apple Google and Microsoft will have to have good support for it but today it's not even available in all browsers yet. When it is released it will be missing features the other middleware solutions have and we'll have to wait for WebGPU Next to be available.

nialv7 3y ago

aww, I thought this is going to be about implementing OpenGL functionalities by directly interacting with the GPU and X.

mananaysiempre 3y ago
I don’t think that exists (I sure would like for it to), but until it does you could amuse yourself with: - A 500-line (non-OpenGL-compatible) software rasterizer: https://github.com/ssloy/tinyrenderer/wiki https://github.com/ssloy/tinyrenderer/wiki. - A “hello Wayland” app written in C without libwayland or anything else: https://gaultier.github.io/blog/wayland_from_scratch.html https://gaultier.github.io/blog/wayland_from_scratch.html. - A “hello X11” app written in x86-64 assembly(!) without libX11, libxcb, or anything else: https://gaultier.github.io/blog/x11_x64.html https://gaultier.github.io/blog/x11_x64.html.
dartos 3y ago
You’re looking for driver programming. IIRC OpenGL is implemented at the driver level. I’m sure you could find a, potentially dated, walkthrough of mesa’s code if you looked hard enough.

Aransentin 3y ago

It should be noted that Xlib has been a compatibility wrapper around XCB – the "modern" X11 client library replacement – for about 15 years now, so it might make more sense to just target that. (The only thing on the top of my head that absolutely requires Xlib is some niche Vulkan stuff; e.g. VK_EXT_acquire_xlib_display unfortunately has no XCB analogue.)

0x_rs 3y ago
It's been a very long while since I played with any of this, and I don't want to rely too much on the widely understood to be lacking documentation, but are Xlib and XCB still not in a very weird situation in which one depends on another for different reasons? In this specific case, at least multiple years ago, glx wouldn't play nice without Xlib, if it would be possible at all without a lot of pain. I don't expect it to have changed for obvious reasons, XCB never being complete and X interest slowly waning. https://xcb.freedesktop.org/opengl/#index3h1 https://xcb.freedesktop.org/opengl/#index3h1
- qetuoadgjlxvn 3y ago
  These days you can use EGL to get an OpenGL context in a pure XCB application without needing to use Xlib.
premysl 3y ago
It should also be noted that in general, Xlib can do a lot more than XCB with a /lot/ less effort and bugs, and has an actual documentation, tutorials, books. Xlib certainly is quirky, in particular around error handling, though.
engeljohnb 3y ago
I haven't done anything with X11 in a few years, but last I looked made no sense to do anything with XCB because the documentation is half complete, whereas Xlib has everything you might need documented.
- toast0 3y ago