6 ms·
IME, working with an immediate mode UI framework automatically gets rid of most such "architecture astronaut" problems. But I found that it's almost impossible
by flohofwoe 6y ago
IME, working with an immediate mode UI framework automatically gets rid of most such "architecture astronaut" problems.
But I found that it's almost impossible to describe to someone used to event-/callback-driven UIs why exactly that is. You really need to try it yourself on a non-trivial UI to "get it".
- amelius 6y agoYes, but immediate mode causes more CPU load (or draining of batteries) since you have to redraw everything all the time.
- panic 6y agoIt's easy to skip drawing frames when no state changes, and for frames where state does change, you can use techniques like rxi's cached software rendering to avoid redrawing the entire screen: https://rxi.github.io/cached_software_rendering.html https://rxi.github.io/cached_software_rendering.html
- amelius 6y agoHow does it handle scrolling? This is typically very fast and smooth (performed by bitblt and redrawing only the newly exposed part on every frame), but in immediate mode you'd have to redraw the entire screen on every frame.
- panic 6y agoModern renderers typically draw scrollable content into a separate texture (or collection of smaller textures that tile the scrollable region) and use the GPU to scroll rather than bitblting on the CPU. You can use the same technique to render the scrollable part of the UI in an immediate mode system.
- flohofwoe 6y agoI think this whole efficiency thing is a common misconception. Only the public API appears "immediate mode", the internal implementation doesn't need to be, and usually isn't (e.g. it keeps mutating state between frames instead of building everything from scratch again). The user-side code basically describes what the UI should look like in the current frame, those "instructions" are recorded, and this recording is reasonably cheap. The UI backend can then figure out how to "diff" the new instruction stream against the current internal state and render this with the least changes to the screen. However some immediate mode UI systems came to the conclusion that it might actually be cheaper to just render most things from scratch instead of spending lots of processing resources to figure out what needs to be updated. In conclusion: "Immediate Mode UI" doesn't say anything how the UI is actually rendered or generally how the internals are implemented, it only describes how the public API works.
- amelius 6y agoPerhaps I should give a different example: a listbox filled with 1 million items, with a scrollbar. If the public API requires you to give a new paint command on every frame (everytime the scrollbar is dragged), then regardless of whether the underlying rendering engine performs each of these paint commands, you still have to run through every item of the list (and so does the diff'ing code), making this a O(N) operation on every frame.
- flohofwoe 6y agoFrom what I've seen (in Dear ImGui), this is solved by not giving the 1 million items to the API, instead you can query what range of the list is currently visible, and only describe those to the API (in Dear ImGui this is called a ListClipper). But I guess different UI frameworks have different solution for this. Creating and updating a 1 million item list wouldn't be a cheap operation in a traditional UI system either.
- amelius 6y agoOk. Yes my point is not that you can't find efficient ways around the limitations, but my fear is that you end up with a system that is less ergonomic in the end.
- layer8 6y agoThat doesn‘t seem too different anymore to the invalidate/paint paradigm of traditional UI frameworks (e.g. win32 GDI).
- forrestthewoods 6y ago> immediate mode causes more CPU load Does it? Computers are FAST. Webdev stacks are many things, but it ain’t fast or performant. Has anyone built a moderately complex UI in a good immediate mode UI and in a good retained UI? I’d be very curious to know the actual results.
- jameshart 6y agoWhat if you have your ‘immediate mode’ produce not a direct rendering of the GUI, but a virtual object model describing what the structure of the GUI should be at that moment? Then you diff that object model against the actual object model of the GUI and apply the changes.... That’s how react works. It’s effectively an ‘immediate mode’ abstraction above a component based UI. That avoids the component connecting and wiring problems, and creates the simple determinism that makes immediate mode systems easier to reason about.
- panic 6y agoImmediate mode is great, but it can be hard to implement nice custom event handling on top of it. For example, say you have a collection of buttons. You want to be able to click on one button (the button highlights), drag onto another button (that button highlights and the first button unhighlights), and release on that second button to trigger its action. This is a real issue I ran into while building an immediate-mode UI... I found it hard to implement an interaction like this without a reified "layout" that I could query as the mouse moved around.
- boterock 6y agoI was doing something like that with Godot [1], the approach I took was setting some global "dragged_item" when the drag started, and then each drop target would check if the dragged_item was compatible and if the mouse was within its bounds. This way each drop target would do the check instead of some drag manager having to check whatever was under the cursor. [1] https://github.com/0xafbf/aether/tree/master/addons/godot_imgui https://github.com/0xafbf/aether/tree/master/addons/godot_im...
- erwincoumans 6y agoAgreed, in particular Dear Imgui is pretty nice, and high performance.
- badsectoracula 6y agoI've tried it at the past and "get it" but that doesn't mean i "agree with it" - to me immediate mode GUIs are the graphical equivalent to... Running = -1 WHILE Running PRINT "1) Add record" IF HasRecords THEN PRINT "2) Delete record" PRINT "3) Edit record" END IF PRINT "H) Help" PRINT "X) Exit" INPUT "Choice: ", I$ IF I$="1" THEN AddRecord IF (I$="2") AND HasRecords THEN DeleteRecord IF (I$="3") AND HasRecords THEN EditRecord IF I$="H" THEN ShowHelp IF I$="X" THEN Running = 0 WEND ...which, sure, it is ridiculously simple (like imguis) but it can very quickly become unwieldy the more you ask from your user interface. There is a reason why UIs moved beyond that.
- flohofwoe 6y agoI think when you look at projects like Tracy, there's no question that immediate mode UIs also work well for non-trivial use cases: https://github.com/wolfpld/tracy https://github.com/wolfpld/tracy
- jcelerier 6y ago> , there's no question that immediate mode UIs also work well I sincerely hope no one looks at the screenshot on that page and thinks "this is something that works well". This UI screams "I've been made that way because that was the easiest way in the tool I'm built with" (which is bad - good tools should not dictate the form of the resulting product)
- flohofwoe 6y agoHow would a profiling tool UI need to look differently in your opinion? And why do you think the UI looks that way because of restrictions of the UI framework? I'm quite sure if the UI would have been written with (for instance) Qt or React, it would look exactly the same, because the UI requirements would be the same (minus the different default style). The question is whether this could have been achieved with less, or "cleaner" code (which I doubt).
- Const-me 6y agoI used and implemented immediate mode GUIs, and I think it’s only good for simple stuff. Accessibility is hard. The OS needs to know visual tree to be able to conclude things like “this rectangle is a clickable button with Hello World text”. Complex layouts are hard. Desktop software have been doing fluid layouts decades before the term was coined for web apps, because different screen resolutions, and because some windows are user-resizable. Layout can be expensive (one reason is measuring pixel size of text), you want to reuse the data between frames. Animations are hard. Users expect them because smartphones introduced them back in 2007: https://streamable.com/okvhl https://streamable.com/okvhl Note the kinetic scrolling, and soft “bumps” when trying to scroll to the end. Drag and drop is very hard.
- flohofwoe 6y agoApart from accessibility, isn't this mostly caused by the "immediate mode API" versus "immediate mode implementation" confusion I described in another sub-thread? There's nothing in the idea that forbids immediate mode UI frameworks from keeping any amount of internal state between frames to keep track of changes over time (like animations or drag'n'drop actions), the difference to traditional UI frameworks is just that this persistent state tree is hidden from the outside world. Layout problems can be solved by running several passes over the UI description before rendering happens. For accessibility, the ball is mainly in the court of the operating system and browser vendors. There need to be accessibility APIs which let user interface frameworks connect to screen readers and other accessibility features (this is not a problem limited to immediate mode UIs, but custom UIs in general).
- enqk 6y agothese APIs are UI Automation on windows and NSAccessibility on macos
- Const-me 6y ago> There's nothing in the idea that forbids immediate mode UI frameworks from keeping any amount of internal state between frames to keep track of changes over time When you call ImGui::Button("Hello World") it has no way of telling if it’s the same button as on the previous frame, or a new one. You’re simply not providing that data to the framework. It’s often good enough for rendering, they don’t really care if it’s an old or new button as long as the GPU renders identical pixels. When you need state however (animations certainly do), the approach is no longer useful. A framework might use heuristics like match text of the button or relative order of things, but none of them is reliable enough. Buttons may change text, they may be dynamically shown or hidden, things may be reordered in runtime. > Layout problems can be solved by running several passes over the UI description before rendering happens. You’re correct that it’s technically solvable. It’s just becomes computationally expensive for complex GUIs. You can’t reliably attach state to visual elements, gonna have to re-measure them every frame. > There need to be accessibility APIs which let user interface frameworks connect to screen readers Windows has it for decades now. For Win32 everything is automatic and implemented by the OS. Good custom-painted GUI frameworks like WPF or UWP handle WM_GETOBJECT message and return IAccessible COM interface. That COM interface implements reflection of the complete GUI, exposing an objects hierarchy. I’m not saying it’s impossible to do with immediate rendering, just very hard without explicit visual tree that persists through the lifetime of the window.