12 ms·

Unanswered user makes question their MS thesis and answers self 2 years later

429 points by anderspitman 6y ago

Hnrobert42 6y ago
That’s pretty awesome. Admittedly, all I know about immediate mode and retained modes GUIs, I learned from this post, but I wonder why there’s no library that uses both. Could a library: 1. Use immediate mode to calculate the display. 2. Use retained mode as long as the amount of change is below a certain threshold. 3. Switch to immediate mode above the threshold. And so on.
- LASR 6y ago
  The difference is in the programming model. The library itself is probably quite easy to support both models. If you have some UI in retained mode, chances are that rendering it live every frame will lead to unacceptable performance. But if you have UI in immediate mode, you can probably switch to retained - essentially lazy rendering until something else happens. But you'll have to be very careful that your render calls are indeed pure functions, and cannot produce different outputs unless some input has indeed changed.
- anderspitman 6y ago
  In practice they do tend to be mixed, if not in exactly the way you describe. Immediate mode systems often retain/cache a lot of state under the hood for performance reasons, and determine exactly what needs to be redrawn on each iteration. Virtual doms (React, Vue, etc) are a great example of this.
- c-smile 6y ago
  > but I wonder why there’s no library that uses both Sciter (https://sciter.com https://sciter.com) uses both, as internally as on user's level. For example you may want to define this: var bodyElement = document.body; bodyElement.paintForeground = function(graphics) { ... draw something ... on top of <body> content ... using the graphics } This allows to benefit from both: retained mode (HTML DOM, layout cached) and immediate mode rendering - paint handlers on DOM elements.
- megameter 6y ago
  They meet in the middle when we talk about layout computation. Layout often flips between needing a globally-optimized solution of some kind(how to pack many boxes given some parameters of box size), and needing a fast, simple dynamic adjustment(same drawing algorithm with different numbers passed in). And what happens is that when the layout gets complex, the simple dynamic adjustment impacts the overall solution. Parts of the layout may also contain their own state, like window positioning. So even in immediate GUI there is retained elements, and vice versa. It's a complex tangle, all in all.
TazeTSchnitzel 6y ago
Do popular GUI frameworks that present an immediate-mode interface actually use an immediate-mode implementation under the hood?
- anderspitman 6y ago
  The most popular immediate mode system I'm aware of is React, which definitely uses retained under the hood.
  - mst 6y ago
    I think I'd argue that React is immediate mode with caching and then a retained-ish driver layer to talk to the underlying DOM. But given the constraints of the "talking to the underlying DOM" part it's going to be a bit handwavey no matter how you argue it, I suspect.
  - voodootrucker 6y ago
    As a person who has used a slew of UI frameworks and written one or two: All of this is happening at a layer considerably lower than React. This is what's happening when the browser redraws when you scroll for example. Arguably it has to do with the DOM, but certainly not how the DOM gets updated.
- mst 6y ago
  "Dear Imgui" is pure immediate mode both interface and implementation wise so far as I can tell.
  - dkersten 6y ago
    Although it is worth noting that calls to Dear ImGUI don’t directly render, but rather build up state to batch render later with your graphics API of choice.
    - mst 6y ago
      Fair. "blitmediate"
ChrisMarshallNY 6y ago
That's cool. Thanks for sharing it. I tend to use the built-in framework (Swift native API). WFM.
kristianp 6y ago
The answer compares resizing performance of their application to spotify's Electron app, which has a chrome browser doing the rendering. Not a very fair comparison. I assume their thesis doesn't do that.
- bestest 6y ago
  As far as I understand Spotify does not use Electron, it uses the Chromium Embedded Framework.
  - nefitty 6y ago
    You are correct: https://www.spotify.com/us/opensource/ https://www.spotify.com/us/opensource/
- krageon 6y ago
  It's fair because they're both GUIs. It shouldn't have to be anyone's problem that a set of developers chose to pack a whole browser inside of their user interface.
  - bmm6o 6y ago
    If the question is about comparing the performance of immediate mode and retained mode, the comparison should be between two otherwise-similar applications. There are a lot of applications that are faster than Spotify, but that says more about Spotify than about those applications.
- pulp_user 6y ago
  (OP here) I adressed that caveat in the late part of the answer. I also noted that I don't even think the performance difference is attributable to immediate vs retained mode. This was just a comparison agains existing applications. As far as I am concerned though, it is pretty irrelevant to the user if an application is slow because the developer made a mistake, or because they wanted it that way.
deleted 6y ago
[deleted]
DubiousPusher 6y ago
Very cool and as a lapsed GUI/graphics guy this makes complete sense to me. The bottleneck on guis is remastering and layout not cycling through the lists of sliders. How often you repaint or have to relayout should depend on only two things. How often is your data changing? And how soon does the user need to see that new data? Even things like moving the mouse or view are just data changes. (In fact, the biggest messes you'll get into when writing GUI and render code is when you've failed to incorporate something into your scene or data model and end up handling it with a lot of special case code.)
- bsder 6y ago
  It's interesting because this matches every GUI that has to respond in real-time. If you look at ALL the digital workstations, they're doing their own rendering. And they all punt on text scaling.
mleonhard 6y ago
How about an immediate-mode renderer that works like a Rust async executor? The executor could run all of the rendering closures on one core and benefit from caching. It could detect slow rendering closures and automatically switch them to buffered mode and run them on a "slow renderers" thread. This would keep the rest of the UI snappy. It could also tint those portions of the UI so users can see which app is the culprit of UI slowness. I hope that Rust can someday be used as the basis of an OS that does not rely on sandboxing for security, but instead uses the compiler to enforce security. Such an OS could use cooperative multitasking and use a lot less energy and less expensive hardware than our current multi-core CPU + MMU + kernel ring model.
- voodootrucker 6y ago
  I was trying to write this while learning Rust: https://github.com/avantgardnerio/rsui https://github.com/avantgardnerio/rsui I now have some production experience in Rust and have learned to shy away from Piston. I think you've inspired me to try again with SDL2 and a better approach.
- nitrogen 6y ago
  To use the compiler for security you would need a way to prove that a given piece of machine code was generated by a trustworthy compiler, and not modified after it was loaded.
  - mleonhard 6y ago
    Yes. The idea is that all software is distributed as source code. The OS compiles the software itself. To prevent software modification, we need a new CPU with two memory controllers, one for data and one for code. The code memory controller can only read, not write. A separate CPU runs the compiler and writes the code RAM.
keeganpoppen 6y ago
of course the first comment is someone "strongly" begging the user not to make a GUI library because they are "easy to get wrong" and "too big to be a fun hobby project"... i just don't get this mentality in programmers at all, but yet somehow it is rampant. i would maybe kind of understand this perverse form of insecurity / learned helplessness in the face of "complexity" for domains outside of one's training, experience, etc., but ... the whole point of programming is the sense that you can do whatever you want if you just write the code correctly. otherwise, what the hell are we doing here?
- amelius 6y ago
  Perhaps it would have been more correct and more effective to write "there is no money in rolling your own GUI library".
- deleted 6y ago
  [deleted]
- baq 6y ago
  Go ahead just don’t waste other people’s time (colleagues) and money (stakeholders) - there’s opportunity cost to everything.
- Retric 6y ago
  It’s not about helplessness, it’s understanding the difference between writing a toy _ over a week and the professional version is generally around 3+ orders of magnitude. Many people really don’t understand that gap. I have worked with people actually trying to do several of such projects like implementing RSA encryption from scratch, a charting library, and several other similar projects they assumed where going to be a quick solution. It’s not that such projects are impossible, it’s that people starting them generally have no idea what’s actually involved. That said, if you’re doing it for fun then feel free.
  - bigiain 6y ago
    Pretty sure Linus never realised his toy operating system would turn into a 30 year project/career. (I suspect RMS probably suspected the timeframe for GNU, at least for the collection of system utilities rather than the kernel, was going to be a multi decade piece of work.)
znpy 6y ago
I'm really sorry for that guy. Now that he knows for a fact that GUIs can be fast, he'll be suffering for the rest of his life when a GUI app is slow for any reason. I'm only 50% joking here.
- inetknght 6y ago
  > Now that he knows for a fact that GUIs can be fast, he'll be suffering for the rest of his life when a GUI app is slow for any reason. Not only slow GUIs but GUIs that demand tens or hundreds of MBs just to draw a few items
  - bitwize 6y ago
    GUI libraries are heavyweight for a reason: lightweight GUIs do not adequately address accessibility, or internationalization. If you were on my software project, and you weren't using native controls on macOS or Windows, one of the big two on Linux, or Electron, you'd better have a damn good reason.
    - BlueTemplar 6y ago
      Humm, what about Qt ? Or video games ?
      - bitwize 6y ago
        Qt uses native controls on Windows or Mac. Not ideally, but it's a viable solution. And it's one of Linux's big two. Video games count as a damn good reason.
        jokethrowaway 6y ago
        Qt doesn't use native controls, they're lookalike. The point of the comment asking about Qt is most likely that Qt is not heavyweight (it does a great job even on embedded.), therefore pointing out that there is a difference between Qt and the other you mentioned. I'm not a fan of OOP and I think there are more interesting framework model - but Qt is definitely the best and most efficient GUI framework I've had the pleasure to work with.
SoSoRoCoCo 6y ago
What were the problems that repelled him? I see some discussion about immediate vs retained mode, but what where the gripes that led to re-inventing something? Maybe start with X and explain what's so bad about that. Athena is pretty damn bare bones. Fast forward 40 years, what about embedded Qt? It's small and fast. What's the problem with it? I'd be curious to know his complaints and how he addressed them.
- pulp_user 6y ago
  (OP here) The main issues I had with them had nothing to do with immediate vs. retained mode, but simply with the unnecessary constraints the big GUI frameworks put on you. Usually the GUI framwork wants to own the project. I need to build an application IN Qt or IN C# with WPF. Thats just unnecessary. There is no reason why these libraries can't just provide a header with some functions that I can call, and thats it. The notion that I have to use C# for WPF seems completely ridiculous to me. Or that I don't get to use my own build system (Qt) or chose how I want my control flow to happen and so on. I want the GUI framework to be a library to my application, no the other way around. I hate all of these build shenanigans. I had a C++ application, that I built with a .bat file. I was not gonna port that to C# or have some bullshit wrapper around C#, or port the whole thing to Qt, or switch my build system. That WPF and Qt required this of me is just unnecessary, and I was super annoyed that everybody thinks that is just ok and not a big deal. Stuff like this makes modern programming a slog and kills my productivity and morale. I don't know about embedded Qt, but after seing that standard dev Qt download is 40 GB, I already knew this is not going to work because 99% of that is going to be stuff I don't need or want, and (as it always goes with these things) it is going to cause a lot of friction, because, of course, I will still have to interact with those things. So, yeah, basically I just couldn't find a small, simple, non-intrusive, no arbitrary constraints GUI library that looked halfway decent.
jokethrowaway 6y ago
Often times immediate / retained can be mixed up together (eg. https://docs.rs/conrod_core/0.71.0/conrod_core/guide/chapter_1/index.html#is-conrod-immediate-or-retained https://docs.rs/conrod_core/0.71.0/conrod_core/guide/chapter...)
emptyparadise 6y ago
Would be nice to read the full thesis. This is a pretty naive question, but I wonder if retaining the current frame (just as pixels) and only redrawing when there are actual changes is a good middle ground? Is this something already being done in immediate mode GUI libraries?
- drglove 6y ago
  Determining where there are changes would likely take up the bulk of the work to actually redraw in this scenario I would think, since you'd (to some extent or another) have to draw the new state to be able to compare it to the current state.
ma2rten 6y ago
I don't understand this. Isn't it obviously slower if you redraw every window in every frame instead of just redrawing what changed?
- drakmaniso 6y ago
  In order to know "what changed", you need at the very minimum to maintain a meta-state about your application state. In most retained-mode GUIs, this meta-state is often a copy of a significant portion of your application state, but with a very different structure. To maintain the integrity of both copies, you then need a layer of events and/or listeners on top of that, with a complex and dynamic control flow. Retained mode optimizes for display throughput, which matters a few decades ago. Immediate mode is better for latency and consistency, which are much more important now.
- swyx 6y ago
  it's "obvious" until you learn that most games (with way higher perf requirements than anything else) are immediate mode rendered. as with everything it depends - in this case, on how much usually changes, and how nested your data is (DOM trees are O(n^3) to do proper retained mode diffing, whereas full rerendering every time is O(n)) React's reconciliation bridges this - immediate mode programming model, retained mode commit model, with key-based bailout to solve the O(n^3) problem. There are reasonable disagreements as to how necessary this is, of course.
  - account42 6y ago
    > it's "obvious" until you learn that most games (with way higher perf requirements than anything else) are immediate mode rendered. Games also tend to be the only/main thing running at a time and constantly update large parts of their display. This is not true for most applications.
- josefx 6y ago
  Tracking what changed has its own complexity. If you have transparent or moving elements you have to identify the components behind them and redraw those as well. You could use buffers for each element to avoid some of it, but then you may have to update several buffers for various changes. If you have elements that are updated a thousand times per frame (progress bars) do you update the whole element every time, only update the affected region of it on every change or try to somehow collect changes that might affect different parts of it for a single update operation?
andrewmcwatters 6y ago
I don’t buy it. The creator of Dear IMGUI himself says it’s slower. These posts never talk about how their compositor (if they’re even using one) maintains performance. Are you building vertex buffers? Are you creating backing layers? These are the critical details, not whether or not the API itself “looks” like an immediate mode or retained mode API. Even the immediate mode APIs retain information! As soon as you do compositor work, you need framebuffers, and you can kiss perf goodbye as you realize you can’t just destroy panels left and right without some abstraction to efficiently use video memory. Oh, you can’t paint then scale later? Oh, so to do transparency effects you have to redraw the primitives every frame with new alphas instead of sending a different alpha to your shader? OK. Sure. What’s your invalidation strategy? You don’t have one because it’s all “immediate,” sure. That’s really interesting. If there’s no performance difference, then well, hmm, why didn’t we do this back in the win32 era? Oh well, of course, it’s because there is a performance difference. And it’s staggering. The most critically important work I’ve seen in modern GUI architecture has nothing to do with the API itself and everything to do with the compositor. You can make the API almost always look like anything you want. Under the hood, it’s frametime life or death depending on how you rasterize and display large amounts of UI and typesetting, and panel layout.
- pulp_user 6y ago
  (OP here) >Are you building vertex buffers? yes >Are you creating backing layers? I am not sure what those are. >Even the immediate mode APIs retain information! Yes, immediate mode only refers to the way the API is designed, it has nothing to do with how the framework works internally. >Oh, you can’t paint then scale later? Oh, so to do transparency effects you have to redraw the primitives every frame with new alphas instead of sending a different alpha to your shader? OK. Sure. Yes, in my implementation, I redraw every primitive every frame. >What’s your invalidation strategy? You don’t have one because it’s all “immediate,” sure. That’s really interesting. If there’s no performance difference, then well, hmm, why didn’t we do this back in the win32 era? Because the graphics hardware wasn't there. Todays GPUs are crazy fast, which enables you to just push tens of thousands of vertices to the GPU and be done in less than a millisecond. In the early days this all had to happen on the CPU, which were way too slow for this, so you had to be very careful about what to redraw.
tabtab 6y ago
We need a stateful GUI markup standard so that each language doesn't have to reinvent GUI's or GUI adaptors.
jlokier 6y ago
Immediate mode rendering is similar to what the Firefox WebRender changes are about: https://hacks.mozilla.org/2017/10/the-whole-web-at-maximum-fps-how-webrender-gets-rid-of-jank/ https://hacks.mozilla.org/2017/10/the-whole-web-at-maximum-f...
bhaak 6y ago
That's quite a feat. The best I could manage was looking for an error message when I had a problem with mp3 id3 tags handling, finding an unanswered, 4 years old question on SO which was exactly my problem and eventually answering it after solving it myself.

429 points by anderspitman 6y ago

That’s pretty awesome. Admittedly, all I know about immediate mode and retained modes GUIs, I learned from this post, but I wonder why there’s no library that uses both. Could a library: 1. Use immediate mode to calculate the display. 2. Use retained mode as long as the amount of change is below a certain threshold. 3. Switch to immediate mode above the threshold. And so on.

LASR 6y ago
The difference is in the programming model. The library itself is probably quite easy to support both models. If you have some UI in retained mode, chances are that rendering it live every frame will lead to unacceptable performance. But if you have UI in immediate mode, you can probably switch to retained - essentially lazy rendering until something else happens. But you'll have to be very careful that your render calls are indeed pure functions, and cannot produce different outputs unless some input has indeed changed.
anderspitman 6y ago
In practice they do tend to be mixed, if not in exactly the way you describe. Immediate mode systems often retain/cache a lot of state under the hood for performance reasons, and determine exactly what needs to be redrawn on each iteration. Virtual doms (React, Vue, etc) are a great example of this.
c-smile 6y ago
> but I wonder why there’s no library that uses both Sciter (https://sciter.com https://sciter.com) uses both, as internally as on user's level. For example you may want to define this: var bodyElement = document.body; bodyElement.paintForeground = function(graphics) { ... draw something ... on top of <body> content ... using the graphics } This allows to benefit from both: retained mode (HTML DOM, layout cached) and immediate mode rendering - paint handlers on DOM elements.
megameter 6y ago
They meet in the middle when we talk about layout computation. Layout often flips between needing a globally-optimized solution of some kind(how to pack many boxes given some parameters of box size), and needing a fast, simple dynamic adjustment(same drawing algorithm with different numbers passed in). And what happens is that when the layout gets complex, the simple dynamic adjustment impacts the overall solution. Parts of the layout may also contain their own state, like window positioning. So even in immediate GUI there is retained elements, and vice versa. It's a complex tangle, all in all.

TazeTSchnitzel 6y ago

Do popular GUI frameworks that present an immediate-mode interface actually use an immediate-mode implementation under the hood?

anderspitman 6y ago
The most popular immediate mode system I'm aware of is React, which definitely uses retained under the hood.
- mst 6y ago
  I think I'd argue that React is immediate mode with caching and then a retained-ish driver layer to talk to the underlying DOM. But given the constraints of the "talking to the underlying DOM" part it's going to be a bit handwavey no matter how you argue it, I suspect.
- voodootrucker 6y ago
  As a person who has used a slew of UI frameworks and written one or two: All of this is happening at a layer considerably lower than React. This is what's happening when the browser redraws when you scroll for example. Arguably it has to do with the DOM, but certainly not how the DOM gets updated.
mst 6y ago
"Dear Imgui" is pure immediate mode both interface and implementation wise so far as I can tell.
- dkersten 6y ago
  Although it is worth noting that calls to Dear ImGUI don’t directly render, but rather build up state to batch render later with your graphics API of choice.
  - mst 6y ago
    Fair. "blitmediate"

ChrisMarshallNY 6y ago

That's cool. Thanks for sharing it. I tend to use the built-in framework (Swift native API). WFM.

kristianp 6y ago

The answer compares resizing performance of their application to spotify's Electron app, which has a chrome browser doing the rendering. Not a very fair comparison. I assume their thesis doesn't do that.

bestest 6y ago
As far as I understand Spotify does not use Electron, it uses the Chromium Embedded Framework.
- nefitty 6y ago
  You are correct: https://www.spotify.com/us/opensource/ https://www.spotify.com/us/opensource/
krageon 6y ago
It's fair because they're both GUIs. It shouldn't have to be anyone's problem that a set of developers chose to pack a whole browser inside of their user interface.
- bmm6o 6y ago
  If the question is about comparing the performance of immediate mode and retained mode, the comparison should be between two otherwise-similar applications. There are a lot of applications that are faster than Spotify, but that says more about Spotify than about those applications.
pulp_user 6y ago
(OP here) I adressed that caveat in the late part of the answer. I also noted that I don't even think the performance difference is attributable to immediate vs retained mode. This was just a comparison agains existing applications. As far as I am concerned though, it is pretty irrelevant to the user if an application is slow because the developer made a mistake, or because they wanted it that way.

deleted 6y ago

[deleted]

DubiousPusher 6y ago

Very cool and as a lapsed GUI/graphics guy this makes complete sense to me. The bottleneck on guis is remastering and layout not cycling through the lists of sliders. How often you repaint or have to relayout should depend on only two things. How often is your data changing? And how soon does the user need to see that new data? Even things like moving the mouse or view are just data changes. (In fact, the biggest messes you'll get into when writing GUI and render code is when you've failed to incorporate something into your scene or data model and end up handling it with a lot of special case code.)

bsder 6y ago
It's interesting because this matches every GUI that has to respond in real-time. If you look at ALL the digital workstations, they're doing their own rendering. And they all punt on text scaling.

mleonhard 6y ago

How about an immediate-mode renderer that works like a Rust async executor? The executor could run all of the rendering closures on one core and benefit from caching. It could detect slow rendering closures and automatically switch them to buffered mode and run them on a "slow renderers" thread. This would keep the rest of the UI snappy. It could also tint those portions of the UI so users can see which app is the culprit of UI slowness. I hope that Rust can someday be used as the basis of an OS that does not rely on sandboxing for security, but instead uses the compiler to enforce security. Such an OS could use cooperative multitasking and use a lot less energy and less expensive hardware than our current multi-core CPU + MMU + kernel ring model.

voodootrucker 6y ago
I was trying to write this while learning Rust: https://github.com/avantgardnerio/rsui https://github.com/avantgardnerio/rsui I now have some production experience in Rust and have learned to shy away from Piston. I think you've inspired me to try again with SDL2 and a better approach.
nitrogen 6y ago
To use the compiler for security you would need a way to prove that a given piece of machine code was generated by a trustworthy compiler, and not modified after it was loaded.
- mleonhard 6y ago
  Yes. The idea is that all software is distributed as source code. The OS compiles the software itself. To prevent software modification, we need a new CPU with two memory controllers, one for data and one for code. The code memory controller can only read, not write. A separate CPU runs the compiler and writes the code RAM.

keeganpoppen 6y ago

of course the first comment is someone "strongly" begging the user not to make a GUI library because they are "easy to get wrong" and "too big to be a fun hobby project"... i just don't get this mentality in programmers at all, but yet somehow it is rampant. i would maybe kind of understand this perverse form of insecurity / learned helplessness in the face of "complexity" for domains outside of one's training, experience, etc., but ... the whole point of programming is the sense that you can do whatever you want if you just write the code correctly. otherwise, what the hell are we doing here?

amelius 6y ago
Perhaps it would have been more correct and more effective to write "there is no money in rolling your own GUI library".
deleted 6y ago
[deleted]
baq 6y ago
Go ahead just don’t waste other people’s time (colleagues) and money (stakeholders) - there’s opportunity cost to everything.
Retric 6y ago
It’s not about helplessness, it’s understanding the difference between writing a toy _ over a week and the professional version is generally around 3+ orders of magnitude. Many people really don’t understand that gap. I have worked with people actually trying to do several of such projects like implementing RSA encryption from scratch, a charting library, and several other similar projects they assumed where going to be a quick solution. It’s not that such projects are impossible, it’s that people starting them generally have no idea what’s actually involved. That said, if you’re doing it for fun then feel free.
- bigiain 6y ago
  Pretty sure Linus never realised his toy operating system would turn into a 30 year project/career. (I suspect RMS probably suspected the timeframe for GNU, at least for the collection of system utilities rather than the kernel, was going to be a multi decade piece of work.)

znpy 6y ago

I'm really sorry for that guy. Now that he knows for a fact that GUIs can be fast, he'll be suffering for the rest of his life when a GUI app is slow for any reason. I'm only 50% joking here.

inetknght 6y ago
> Now that he knows for a fact that GUIs can be fast, he'll be suffering for the rest of his life when a GUI app is slow for any reason. Not only slow GUIs but GUIs that demand tens or hundreds of MBs just to draw a few items
- bitwize 6y ago
  GUI libraries are heavyweight for a reason: lightweight GUIs do not adequately address accessibility, or internationalization. If you were on my software project, and you weren't using native controls on macOS or Windows, one of the big two on Linux, or Electron, you'd better have a damn good reason.
  - BlueTemplar 6y ago
    Humm, what about Qt ? Or video games ?
    - bitwize 6y ago
      Qt uses native controls on Windows or Mac. Not ideally, but it's a viable solution. And it's one of Linux's big two. Video games count as a damn good reason.
      - jokethrowaway 6y ago
        Qt doesn't use native controls, they're lookalike. The point of the comment asking about Qt is most likely that Qt is not heavyweight (it does a great job even on embedded.), therefore pointing out that there is a difference between Qt and the other you mentioned. I'm not a fan of OOP and I think there are more interesting framework model - but Qt is definitely the best and most efficient GUI framework I've had the pleasure to work with.

SoSoRoCoCo 6y ago

What were the problems that repelled him? I see some discussion about immediate vs retained mode, but what where the gripes that led to re-inventing something? Maybe start with X and explain what's so bad about that. Athena is pretty damn bare bones. Fast forward 40 years, what about embedded Qt? It's small and fast. What's the problem with it? I'd be curious to know his complaints and how he addressed them.

pulp_user 6y ago
(OP here) The main issues I had with them had nothing to do with immediate vs. retained mode, but simply with the unnecessary constraints the big GUI frameworks put on you. Usually the GUI framwork wants to own the project. I need to build an application IN Qt or IN C# with WPF. Thats just unnecessary. There is no reason why these libraries can't just provide a header with some functions that I can call, and thats it. The notion that I have to use C# for WPF seems completely ridiculous to me. Or that I don't get to use my own build system (Qt) or chose how I want my control flow to happen and so on. I want the GUI framework to be a library to my application, no the other way around. I hate all of these build shenanigans. I had a C++ application, that I built with a .bat file. I was not gonna port that to C# or have some bullshit wrapper around C#, or port the whole thing to Qt, or switch my build system. That WPF and Qt required this of me is just unnecessary, and I was super annoyed that everybody thinks that is just ok and not a big deal. Stuff like this makes modern programming a slog and kills my productivity and morale. I don't know about embedded Qt, but after seing that standard dev Qt download is 40 GB, I already knew this is not going to work because 99% of that is going to be stuff I don't need or want, and (as it always goes with these things) it is going to cause a lot of friction, because, of course, I will still have to interact with those things. So, yeah, basically I just couldn't find a small, simple, non-intrusive, no arbitrary constraints GUI library that looked halfway decent.

jokethrowaway 6y ago

Often times immediate / retained can be mixed up together (eg. https://docs.rs/conrod_core/0.71.0/conrod_core/guide/chapter_1/index.html#is-conrod-immediate-or-retained https://docs.rs/conrod_core/0.71.0/conrod_core/guide/chapter...)

emptyparadise 6y ago

Would be nice to read the full thesis. This is a pretty naive question, but I wonder if retaining the current frame (just as pixels) and only redrawing when there are actual changes is a good middle ground? Is this something already being done in immediate mode GUI libraries?

drglove 6y ago
Determining where there are changes would likely take up the bulk of the work to actually redraw in this scenario I would think, since you'd (to some extent or another) have to draw the new state to be able to compare it to the current state.

ma2rten 6y ago

I don't understand this. Isn't it obviously slower if you redraw every window in every frame instead of just redrawing what changed?

drakmaniso 6y ago
In order to know "what changed", you need at the very minimum to maintain a meta-state about your application state. In most retained-mode GUIs, this meta-state is often a copy of a significant portion of your application state, but with a very different structure. To maintain the integrity of both copies, you then need a layer of events and/or listeners on top of that, with a complex and dynamic control flow. Retained mode optimizes for display throughput, which matters a few decades ago. Immediate mode is better for latency and consistency, which are much more important now.
swyx 6y ago
it's "obvious" until you learn that most games (with way higher perf requirements than anything else) are immediate mode rendered. as with everything it depends - in this case, on how much usually changes, and how nested your data is (DOM trees are O(n^3) to do proper retained mode diffing, whereas full rerendering every time is O(n)) React's reconciliation bridges this - immediate mode programming model, retained mode commit model, with key-based bailout to solve the O(n^3) problem. There are reasonable disagreements as to how necessary this is, of course.
- account42 6y ago
  > it's "obvious" until you learn that most games (with way higher perf requirements than anything else) are immediate mode rendered. Games also tend to be the only/main thing running at a time and constantly update large parts of their display. This is not true for most applications.
josefx 6y ago
Tracking what changed has its own complexity. If you have transparent or moving elements you have to identify the components behind them and redraw those as well. You could use buffers for each element to avoid some of it, but then you may have to update several buffers for various changes. If you have elements that are updated a thousand times per frame (progress bars) do you update the whole element every time, only update the affected region of it on every change or try to somehow collect changes that might affect different parts of it for a single update operation?

andrewmcwatters 6y ago

I don’t buy it. The creator of Dear IMGUI himself says it’s slower. These posts never talk about how their compositor (if they’re even using one) maintains performance. Are you building vertex buffers? Are you creating backing layers? These are the critical details, not whether or not the API itself “looks” like an immediate mode or retained mode API. Even the immediate mode APIs retain information! As soon as you do compositor work, you need framebuffers, and you can kiss perf goodbye as you realize you can’t just destroy panels left and right without some abstraction to efficiently use video memory. Oh, you can’t paint then scale later? Oh, so to do transparency effects you have to redraw the primitives every frame with new alphas instead of sending a different alpha to your shader? OK. Sure. What’s your invalidation strategy? You don’t have one because it’s all “immediate,” sure. That’s really interesting. If there’s no performance difference, then well, hmm, why didn’t we do this back in the win32 era? Oh well, of course, it’s because there is a performance difference. And it’s staggering. The most critically important work I’ve seen in modern GUI architecture has nothing to do with the API itself and everything to do with the compositor. You can make the API almost always look like anything you want. Under the hood, it’s frametime life or death depending on how you rasterize and display large amounts of UI and typesetting, and panel layout.

pulp_user 6y ago
(OP here) >Are you building vertex buffers? yes >Are you creating backing layers? I am not sure what those are. >Even the immediate mode APIs retain information! Yes, immediate mode only refers to the way the API is designed, it has nothing to do with how the framework works internally. >Oh, you can’t paint then scale later? Oh, so to do transparency effects you have to redraw the primitives every frame with new alphas instead of sending a different alpha to your shader? OK. Sure. Yes, in my implementation, I redraw every primitive every frame. >What’s your invalidation strategy? You don’t have one because it’s all “immediate,” sure. That’s really interesting. If there’s no performance difference, then well, hmm, why didn’t we do this back in the win32 era? Because the graphics hardware wasn't there. Todays GPUs are crazy fast, which enables you to just push tens of thousands of vertices to the GPU and be done in less than a millisecond. In the early days this all had to happen on the CPU, which were way too slow for this, so you had to be very careful about what to redraw.

tabtab 6y ago

We need a stateful GUI markup standard so that each language doesn't have to reinvent GUI's or GUI adaptors.

jlokier 6y ago

Immediate mode rendering is similar to what the Firefox WebRender changes are about: https://hacks.mozilla.org/2017/10/the-whole-web-at-maximum-fps-how-webrender-gets-rid-of-jank/ https://hacks.mozilla.org/2017/10/the-whole-web-at-maximum-f...

bhaak 6y ago

That's quite a feat. The best I could manage was looking for an error message when I had a problem with mp3 id3 tags handling, finding an unanswered, 4 years old question on SO which was exactly my problem and eventually answering it after solving it myself.