3 ms·

After working with OpenGL for many years and then moving to higher-level systems more recently (e.g. three.js), I found a useful way of viewing the 'essence' of

After working with OpenGL for many years and then moving to higher-level systems more recently (e.g. three.js), I found a useful way of viewing the 'essence' of the API: moving data around. It's not so much a '3D graphics API' as it is a wrapper for interacting with a a GPU. (A notable exception is shader code, which is largely doing custom geometric or pixel transformations. But the rest of it is pretty much an API for moving blocks of data in and out of the GPU.) I think this is important to know, because the common perception is that OpenGL, as an API for creating 3D graphics, would give you primitives for talking about 3D objects/spaces—which for the most part it doesn't, aside from essentially utility classes (e.g. vectors/matrices for working with 3D spaces). In consideration of that, it's a pretty awful experience if you're interested in describing 3D scenes, as opposed to developing new rendering techniques or optimizing old ones or whatever. So instead, it's an API well-suited to building actual 3D graphics APIs on top—scene graphs, game engines, etc. My understanding is that Vulkan/Metal take this to a further extreme, and I see it commonly mentioned in that context—but it's true of OpenGL too! It should be more common to warn beginners: learn OpenGL so that you can understand the underpinnings of some other system you will eventually use to describe 3D graphics—but this isn't the endpoint, don't build all your games or whatever directly on OpenGL. You will have so much more fun with a good scene graph.

dahart 8y ago
> I found a useful way of viewing the ‘essence’ of the API: moving data around. It’s not so much a ‘3D graphics API’ as it is a wrapper for interacting with a GPU. It is very true that most of the WebGL functions just move data in & out of the GPU, but to me this feels similar to saying that WebGL is a 2D API. The thing is, all APIs essentially move data around, if we break it down. It feels maybe too reductionist. But it depends on your goal, right? For a student who wants to know how WebGL works, it is helpful to break down WebGL into components: move data into GPU, rasterize into buffer, view or optionally copy data out. Where’s the 3d? You get support for 3d data & operations, but (unlike OpenGL 1) you now have to design your 3d->2d projection yourself. For everyone else, for people who want to know what WebGL is for or why it was built, and for people to want to know how to make 3d graphics, it’s not that helpful to say that WebGL is 2d or that it moves data around. It’s more helpful to know that WebGL is the API you need, if your goal is 3d graphics in a browser. In any case, it’s problematic to insist that either interpretation is the complete picture, or that one of them is wrong, agreed? > So instead, it’s an API well-suited to building actual 3D graphics APIs on top—scene graphs, game engines, etc. Or, OpenGL/Vulkan/Metal/WebGL are actual APIs for 3D that are good for building scene graph APIs and game engine APIs on top of? These 3d graphics APIs intentionally don’t provide scenes graphs, for very good reasons. That would compromise their generality and speed.
- westoncb 8y ago
  > The thing is, all APIs essentially move data around Technically true, but misleading to bring up. OpenGL is different because the primitives in it are explicitly about collections of data of specific types. Whereas an API that was 'about' 3D graphics would offer primitives related to the shape, position, and other aspects of the appearance of 3D forms and how they change in time. > It’s more helpful to know that WebGL is the API you need, if your goal is 3d graphics in a browser. I disagree. I think for almost everyone's needs, the API they need is something more like three.js. If you are writing a new scene graph or whatever, then WebGL is what you're looking for. More importantly though, it's about managing expectations of new users: yes, you can use OpenGL/WebGL to create moving 3D pictures—but it's going to be a linguistically awkward, and yet (potentially) performant, way of doing it. > These 3d graphics APIs intentionally don’t provide scenes graphs, for very good reasons. That would compromise their generality and speed. To be clear, I am not saying that OpenGL, Vulkan, etc. should provide scene graphs. I'm saying they are essentially hardware access layers with some special utilities for doing graphics stuff. They need to exist, and they should not be doing irrelevant things like providing scene graphs.
  - dahart 8y ago
    > OpenGL is different because the primitives in it are explicitly about collections of data of specific types. Different than what? Three.js? OpenCV? Firebase? Instagram? All of these can be described as being explicitly about collections of data of specific types. This isn’t differentiating, you’re still describing all APIs. > Whereas an API that was ‘about’ 3D graphics would offer primitives related to the shape, position, and other aspects of the appearance of 3D forms and how they change in time. That is what WebGL provides. You’re trying to split a hair that doesn’t exist. I realize you’re trying to talk about the difference between immediate mode and retained state graphics APIs. Both are “about” 3D, and retained mode is not more 3D or more “about” 3D than immediate mode. I’d argue the other way around. Three.js is built on top of WebGL, it doesn’t do 3d graphics without WebGL, and the only thing it adds is retained mode state, i.e., a specific type of data. Three.js doesn’t do it’s own drawing, and it doesn’t add any 3D to WebGL that wasn’t already there, it only adds management of some host side memory so it can call WebGL on the same data repeatedly. By your own metric, three.js is less “about” 3d than WebGL. Anyway, none of this really relates to the original point. Saying that WebGL is about data is neither supporting the argument that WebGL is about 2D graphics nor refuting the argument that WebGL is about 3D graphics. > To be clear, I am not saying that OpenGL, Vulkan, etc. should provide scene graphs. Right. They shouldn’t because their purpose is 2D or 3D drawing in any form, not management of a node graph or scheduling of draw calls.
deleted 8y ago
[deleted]
MaxBarraclough 8y ago
> It's not so much a '3D graphics API' as it is a wrapper for interacting with a a GPU. Mandatory link to Zed Shaw's blog post on the difference between an 'indirection layer' and an 'abstraction layer'. Abstraction is used to reduce complexity. Indirection is used to reduce coupling or dependence. * https://zedshaw.com/archive/indirection-is-not-abstraction/ https://zedshaw.com/archive/indirection-is-not-abstraction/
- westoncb 8y ago
  Hmm, but I didn't say it was an indirection layer nor an abstraction layer...
  - MaxBarraclough 8y ago
    If I'm understanding you and Zed correctly, you pretty much did, you just didn't use Zed's terminology. As you say, the intent of modern graphics APIs like Vulkan isn't to spare the graphics programmer the hard work of writing low-level code. Instead the intent is to provide a minimal indirection layer which does just enough work to let graphics code work on any vendor's hardware, whilst preserving the graphics programmer's ability to do low-level cleverness. So game-engines and scenegraph systems are in the business of abstraction, but graphics APIs are in the business of indirection.
    - westoncb 8y ago
      Okay, I see your point. But indirection only enters the picture once you introduce the possibility of multiple graphics cards which are being communicated with the same API—and I agree, that is an example of indirection. However, it's not the role I was attempting to highlight. I was considering the role of OpenGL/WebGL in relation to a single graphics card: if you consider that situation, the API is still necessary and what it's doing is neither (much) abstraction nor indirection. Instead, it's main task is essentially conversion: the graphics card needs data in one format, javascript programs assemble their data in another (I'm using data very loosely here; I mean to include even the program source, since it's in a 'format' we can just give directly to a GPU). That said, even something like Vulkan does abstract a bit, and once you bring in the possibility of multiple cards, the value of its role as an indirection layer becomes clear.
deleted 8y ago
[deleted]

dahart 8y ago

> I found a useful way of viewing the ‘essence’ of the API: moving data around. It’s not so much a ‘3D graphics API’ as it is a wrapper for interacting with a GPU. It is very true that most of the WebGL functions just move data in & out of the GPU, but to me this feels similar to saying that WebGL is a 2D API. The thing is, all APIs essentially move data around, if we break it down. It feels maybe too reductionist. But it depends on your goal, right? For a student who wants to know how WebGL works, it is helpful to break down WebGL into components: move data into GPU, rasterize into buffer, view or optionally copy data out. Where’s the 3d? You get support for 3d data & operations, but (unlike OpenGL 1) you now have to design your 3d->2d projection yourself. For everyone else, for people who want to know what WebGL is for or why it was built, and for people to want to know how to make 3d graphics, it’s not that helpful to say that WebGL is 2d or that it moves data around. It’s more helpful to know that WebGL is the API you need, if your goal is 3d graphics in a browser. In any case, it’s problematic to insist that either interpretation is the complete picture, or that one of them is wrong, agreed? > So instead, it’s an API well-suited to building actual 3D graphics APIs on top—scene graphs, game engines, etc. Or, OpenGL/Vulkan/Metal/WebGL are actual APIs for 3D that are good for building scene graph APIs and game engine APIs on top of? These 3d graphics APIs intentionally don’t provide scenes graphs, for very good reasons. That would compromise their generality and speed.

westoncb 8y ago
> The thing is, all APIs essentially move data around Technically true, but misleading to bring up. OpenGL is different because the primitives in it are explicitly about collections of data of specific types. Whereas an API that was 'about' 3D graphics would offer primitives related to the shape, position, and other aspects of the appearance of 3D forms and how they change in time. > It’s more helpful to know that WebGL is the API you need, if your goal is 3d graphics in a browser. I disagree. I think for almost everyone's needs, the API they need is something more like three.js. If you are writing a new scene graph or whatever, then WebGL is what you're looking for. More importantly though, it's about managing expectations of new users: yes, you can use OpenGL/WebGL to create moving 3D pictures—but it's going to be a linguistically awkward, and yet (potentially) performant, way of doing it. > These 3d graphics APIs intentionally don’t provide scenes graphs, for very good reasons. That would compromise their generality and speed. To be clear, I am not saying that OpenGL, Vulkan, etc. should provide scene graphs. I'm saying they are essentially hardware access layers with some special utilities for doing graphics stuff. They need to exist, and they should not be doing irrelevant things like providing scene graphs.
- dahart 8y ago
  > OpenGL is different because the primitives in it are explicitly about collections of data of specific types. Different than what? Three.js? OpenCV? Firebase? Instagram? All of these can be described as being explicitly about collections of data of specific types. This isn’t differentiating, you’re still describing all APIs. > Whereas an API that was ‘about’ 3D graphics would offer primitives related to the shape, position, and other aspects of the appearance of 3D forms and how they change in time. That is what WebGL provides. You’re trying to split a hair that doesn’t exist. I realize you’re trying to talk about the difference between immediate mode and retained state graphics APIs. Both are “about” 3D, and retained mode is not more 3D or more “about” 3D than immediate mode. I’d argue the other way around. Three.js is built on top of WebGL, it doesn’t do 3d graphics without WebGL, and the only thing it adds is retained mode state, i.e., a specific type of data. Three.js doesn’t do it’s own drawing, and it doesn’t add any 3D to WebGL that wasn’t already there, it only adds management of some host side memory so it can call WebGL on the same data repeatedly. By your own metric, three.js is less “about” 3d than WebGL. Anyway, none of this really relates to the original point. Saying that WebGL is about data is neither supporting the argument that WebGL is about 2D graphics nor refuting the argument that WebGL is about 3D graphics. > To be clear, I am not saying that OpenGL, Vulkan, etc. should provide scene graphs. Right. They shouldn’t because their purpose is 2D or 3D drawing in any form, not management of a node graph or scheduling of draw calls.

deleted 8y ago

[deleted]

MaxBarraclough 8y ago

> It's not so much a '3D graphics API' as it is a wrapper for interacting with a a GPU. Mandatory link to Zed Shaw's blog post on the difference between an 'indirection layer' and an 'abstraction layer'. Abstraction is used to reduce complexity. Indirection is used to reduce coupling or dependence. * https://zedshaw.com/archive/indirection-is-not-abstraction/ https://zedshaw.com/archive/indirection-is-not-abstraction/

westoncb 8y ago
Hmm, but I didn't say it was an indirection layer nor an abstraction layer...
- MaxBarraclough 8y ago
  If I'm understanding you and Zed correctly, you pretty much did, you just didn't use Zed's terminology. As you say, the intent of modern graphics APIs like Vulkan isn't to spare the graphics programmer the hard work of writing low-level code. Instead the intent is to provide a minimal indirection layer which does just enough work to let graphics code work on any vendor's hardware, whilst preserving the graphics programmer's ability to do low-level cleverness. So game-engines and scenegraph systems are in the business of abstraction, but graphics APIs are in the business of indirection.
  - westoncb 8y ago
    Okay, I see your point. But indirection only enters the picture once you introduce the possibility of multiple graphics cards which are being communicated with the same API—and I agree, that is an example of indirection. However, it's not the role I was attempting to highlight. I was considering the role of OpenGL/WebGL in relation to a single graphics card: if you consider that situation, the API is still necessary and what it's doing is neither (much) abstraction nor indirection. Instead, it's main task is essentially conversion: the graphics card needs data in one format, javascript programs assemble their data in another (I'm using data very loosely here; I mean to include even the program source, since it's in a 'format' we can just give directly to a GPU). That said, even something like Vulkan does abstract a bit, and once you bring in the possibility of multiple cards, the value of its role as an indirection layer becomes clear.

deleted 8y ago

[deleted]