10 ms·

It's important to point out why games use immediate mode GUIs: 1. The GUI needs to be overlaid on the game image (OpenGL/DirectX). This is difficult with tradi

by 781 7y ago

It's important to point out why games use immediate mode GUIs: 1. The GUI needs to be overlaid on the game image (OpenGL/DirectX). This is difficult with traditional GUIs like QT. 2. The GUI needs to be updated in sync with the game, again, it's difficult to integrate traditional GUIs event loops into the game loop, especially with stuff like double/triple buffering. 3. The GUI needs to be as fast as possible, games are severely CPU bound. A retained mode GUI is typically easier to use, convenience is not why people use immediate mode GUIs. It's worth pointing out that the immediate/retained split doesn't apply only to the GUI - there are retained mode graphical APIs - DirectX used to have one. They are only used in low-demand games, they sacrifice a lot of speed for the convenience of using a retained mode.

TazeTSchnitzel 7y ago
I think 1, 2 and maybe 3 could all be done with a traditional retained mode GUI, and indeed there are retained mode GUI systems for games. The big traditional UI frameworks just have not been made to work well with games. Which is fair, game engines are alien environments that may as well be considered their own platforms. > It's worth pointing out that the immediate/retained split doesn't apply only to the GUI Indeed. Modern game engines, and in particular almost all 3D games, use what could be described as a “retained mode” system for the rendering of the game itself — the scene graph — so it is interesting that the UI doesn't always match that.
- seanmcdirmid 7y ago
  Generic retained mode is dead, but creating a scene graph driven by something other than imperative updates (eg constraint satisfaction via a physics engine) is alive and well. The problem is that you can’t overlay the former very well in the latter, so things like OGL retained mode are never used anymore.
jayd16 7y ago
Honestly, this is far too generous. IMGUI is easy to implement so its the easiest to put in a custom game engine. IMGUI is not particularly fast in practice. Unity's certainly isn't. However, they are simple, which approximates performance for small cases and it also lets you write your own tuned implementation. Traditionally screen orientations for games were pretty simple so doing layouts in a single pass was feasible. These days, when you want to fit your game on every platform you need to do a lot of layout work. At that point you're just making an incomplete retained gui implementation.
invokestatic 7y ago
I've written real-time game UIs before so I think I have some relevant experience here. 1. It is very possible to write a retained-mode GUI in a graphics API like DirectX or OpenGL. In fact, a retained GUI would typically wipe immediate GUIs in terms of performance in this context. In immediate mode, the GUI's vertex buffers need to be completely reconstructed from scratch every single frame, which is slow, CPU bound, and cannot be (easily) parallelized. It's like reconstructing the game world every frame -- that would be ludicrous for any non-trivial game. 2. I don't think there would be that much of a difference between the two UI models, since data updates can be dispatched from the event loop. It would be faster, too, because only UI components that need updating could be redrawn. This is far faster than updating the entire UI every single frame. 3. As mentioned earlier, immediate mode GUIs are going to be a lot slower than retained mode, when implemented properly. Immediate mode GUIs put most of the work on the CPU instead of offloading most of the work to the GPU like in the retained model. I think developers that are using immediate mode GUIs are doing so because of their ease of use. I think retained mode is typically harder for a game developer to conceptualize because immediate mode is conceptually similar to a game loop. Also, I don't know of any free & open source retained mode GUIs for DirectX and OpenGL and the like. Also, DirectX at least (and probably OpenGL) encourages a retained-like model for general rendering. The only way to get decent performance is to re-use vertex buffers between frames and only update them when something changes.
- 781 7y ago
  I didn't say that it's not possible to solve these problems in a retained GUI, just that existing ones, QT, Win32, WPF have these problems. And since writing a full retained GUI is not exactly trivial, people just wrote mini-GUIs using immediate mode. When I talked about retained mode APIs, I was thinking about scene graphs, where you say "addMesh" or "addSphere" and then just call "renderFrame". I'm aware that most game engines implement their own scene graph anyway, but it's game specific, not some generic one provided by the OpenGL/DX API.
  - invokestatic 7y ago
    Sorry if I misinterpreted your post but I don't think that changes my response much. You can absolutely use Win32 Forms and WPF (and probably Qt) with a game. They can be overlaid on top of a DirectX or OpenGL window (with a transparent background) -- I've done it before! I don't think it would have any of the downsides you mentioned, either, except that it wouldn't be GPU accelerated or actually rendered inside the graphics context, which is why nobody actually does this in practice. But that ignores the dozens of GUI middleware specifically designed for games. A cursory Google search reveals that most of these are going to be retained mode. There's a reason for that. Projects like ImGUI appeal to mostly indie devs who don't have the time or resources to write their own GUI library or license some third-party middleware. And it's probably going to be just fine for their use case. But it's definitely not a perfect solution and we definitely shouldn't throw away decades worth of knowledge and experience like the article is implying.
overgard 7y ago
Every game I've worked on didn't actually use an IMGUI for anything you'd ever show a player; (unless it's a style of game that really doesn't need much UI), it's just custom gui implementations. A lot of modern games actually put the UI in with the scene graph of the game itself (IE: Unity's new GUI system)
- robmaister 7y ago
  This is definitely the way to go, especially with the more recent trend of embedding UI elements in world space, and VR/AR essentially requiring it. Internal tools tend to be IMGUI, Winforms, or WPF from what I've seen. Essentially whatever the original author is most productive in. No need to be super performant because your only users are other devs with really high end systems and rarely an actual need for a constant 30/60fps. Your engine likely already has a graph structure, event system, and input handling. The core pieces of a retained mode UI system are already there.
c-smile 7y ago
> This is difficult with traditional GUIs like QT. That's not actually that hard to do. You just need to split the UI on layers. Here is for example HTML/CSS is rendered below and on top of 3D scene: https://sciter.com/sciter-and-directx/ https://sciter.com/sciter-and-directx/

by 781 7y ago

TazeTSchnitzel 7y ago

I think 1, 2 and maybe 3 could all be done with a traditional retained mode GUI, and indeed there are retained mode GUI systems for games. The big traditional UI frameworks just have not been made to work well with games. Which is fair, game engines are alien environments that may as well be considered their own platforms. > It's worth pointing out that the immediate/retained split doesn't apply only to the GUI Indeed. Modern game engines, and in particular almost all 3D games, use what could be described as a “retained mode” system for the rendering of the game itself — the scene graph — so it is interesting that the UI doesn't always match that.

seanmcdirmid 7y ago
Generic retained mode is dead, but creating a scene graph driven by something other than imperative updates (eg constraint satisfaction via a physics engine) is alive and well. The problem is that you can’t overlay the former very well in the latter, so things like OGL retained mode are never used anymore.

jayd16 7y ago

Honestly, this is far too generous. IMGUI is easy to implement so its the easiest to put in a custom game engine. IMGUI is not particularly fast in practice. Unity's certainly isn't. However, they are simple, which approximates performance for small cases and it also lets you write your own tuned implementation. Traditionally screen orientations for games were pretty simple so doing layouts in a single pass was feasible. These days, when you want to fit your game on every platform you need to do a lot of layout work. At that point you're just making an incomplete retained gui implementation.

invokestatic 7y ago

I've written real-time game UIs before so I think I have some relevant experience here. 1. It is very possible to write a retained-mode GUI in a graphics API like DirectX or OpenGL. In fact, a retained GUI would typically wipe immediate GUIs in terms of performance in this context. In immediate mode, the GUI's vertex buffers need to be completely reconstructed from scratch every single frame, which is slow, CPU bound, and cannot be (easily) parallelized. It's like reconstructing the game world every frame -- that would be ludicrous for any non-trivial game. 2. I don't think there would be that much of a difference between the two UI models, since data updates can be dispatched from the event loop. It would be faster, too, because only UI components that need updating could be redrawn. This is far faster than updating the entire UI every single frame. 3. As mentioned earlier, immediate mode GUIs are going to be a lot slower than retained mode, when implemented properly. Immediate mode GUIs put most of the work on the CPU instead of offloading most of the work to the GPU like in the retained model. I think developers that are using immediate mode GUIs are doing so because of their ease of use. I think retained mode is typically harder for a game developer to conceptualize because immediate mode is conceptually similar to a game loop. Also, I don't know of any free & open source retained mode GUIs for DirectX and OpenGL and the like. Also, DirectX at least (and probably OpenGL) encourages a retained-like model for general rendering. The only way to get decent performance is to re-use vertex buffers between frames and only update them when something changes.

781 7y ago
I didn't say that it's not possible to solve these problems in a retained GUI, just that existing ones, QT, Win32, WPF have these problems. And since writing a full retained GUI is not exactly trivial, people just wrote mini-GUIs using immediate mode. When I talked about retained mode APIs, I was thinking about scene graphs, where you say "addMesh" or "addSphere" and then just call "renderFrame". I'm aware that most game engines implement their own scene graph anyway, but it's game specific, not some generic one provided by the OpenGL/DX API.
- invokestatic 7y ago
  Sorry if I misinterpreted your post but I don't think that changes my response much. You can absolutely use Win32 Forms and WPF (and probably Qt) with a game. They can be overlaid on top of a DirectX or OpenGL window (with a transparent background) -- I've done it before! I don't think it would have any of the downsides you mentioned, either, except that it wouldn't be GPU accelerated or actually rendered inside the graphics context, which is why nobody actually does this in practice. But that ignores the dozens of GUI middleware specifically designed for games. A cursory Google search reveals that most of these are going to be retained mode. There's a reason for that. Projects like ImGUI appeal to mostly indie devs who don't have the time or resources to write their own GUI library or license some third-party middleware. And it's probably going to be just fine for their use case. But it's definitely not a perfect solution and we definitely shouldn't throw away decades worth of knowledge and experience like the article is implying.

overgard 7y ago

Every game I've worked on didn't actually use an IMGUI for anything you'd ever show a player; (unless it's a style of game that really doesn't need much UI), it's just custom gui implementations. A lot of modern games actually put the UI in with the scene graph of the game itself (IE: Unity's new GUI system)

robmaister 7y ago
This is definitely the way to go, especially with the more recent trend of embedding UI elements in world space, and VR/AR essentially requiring it. Internal tools tend to be IMGUI, Winforms, or WPF from what I've seen. Essentially whatever the original author is most productive in. No need to be super performant because your only users are other devs with really high end systems and rarely an actual need for a constant 30/60fps. Your engine likely already has a graph structure, event system, and input handling. The core pieces of a retained mode UI system are already there.

c-smile 7y ago

> This is difficult with traditional GUIs like QT. That's not actually that hard to do. You just need to split the UI on layers. Here is for example HTML/CSS is rendered below and on top of 3D scene: https://sciter.com/sciter-and-directx/ https://sciter.com/sciter-and-directx/