4 ms·

And current crop of imgui libraries already maintaining states cross updates (after all, that is how you can implement things as simple as drag & drop). However

by liuliu 6y ago

And current crop of imgui libraries already maintaining states cross updates (after all, that is how you can implement things as simple as drag & drop). However, the reason you need to have a retained render object tree somewhere for accessibility is due to its interaction mode (mostly request & response) different from GUI (which renders at 60~120fps). The mismatch in interactivity thus compensated by the retained render mode. It felt almost the old days where your UI is derived strictly from business logic with fixed set of widgets is better for accessibility than our new crop of apps where you have more gestures and shortcuts. It is no wonder that imgui shines if you need more gestures and eye-candy animations.

mwcampbell 6y ago
The mismatch between the periodic from-scratch rendering (push) approach of IMGUI and the request/response (pull) model of current accessibility APIs is a tractable problem. Imagine a streaming protocol that allows the application or GUI toolkit to emit the contents of an accessibility tree, either in whole or as incremental updates, whenever it renders. An IMGUI could produce this data each time it renders, even if it doesn't have a retained tree in memory. It would be up to the consumer of the protocol to create and update a retained tree from this stream. That consumer could be a library running inside the same application, which could then implement current accessibility APIs such as UI Automation. Or, if the protocol caught on and was supported by assistive technologies (e.g. screen readers), those ATs themselves could be the consumers. Indeed, I believe that applications pushing their accessibility information to ATs, which can then create and update their own retained trees, would be a better alternative to the pull-based request/response model that you mention. IMO, the bigger problem is the one that Raph pointed out: widgets (including simple ones like static text) need to have stable identities throughout their lifetimes, even as their state, contents, and locations change. It seems to me that IMGUIs, or at least some of them, make this particularly difficult.
- liuliu 6y ago
  Yes, that would work. But that would be a departure from existing imgui paradigm where the output of imgui library is a 2d bitmap. At that point, it gets really close to the vDOM territory ... As for the stable identities, yes, it is important. But I also don't see much of a problem with explicit Key: the UI components ultimately will have map back into a data object, which should have enough contextual information to provide the said Key.
  - mwcampbell 6y ago
    You're right; for accessibility, at least for screen readers, you need something a lot like a DOM (whether real or virtual).
  - stephc_int13 6y ago
    The output of most imgui libraries is not a 2D bitmap but a structure containing vertices, indices and texture handles, that can be used to render.

by liuliu 6y ago

mwcampbell 6y ago

The mismatch between the periodic from-scratch rendering (push) approach of IMGUI and the request/response (pull) model of current accessibility APIs is a tractable problem. Imagine a streaming protocol that allows the application or GUI toolkit to emit the contents of an accessibility tree, either in whole or as incremental updates, whenever it renders. An IMGUI could produce this data each time it renders, even if it doesn't have a retained tree in memory. It would be up to the consumer of the protocol to create and update a retained tree from this stream. That consumer could be a library running inside the same application, which could then implement current accessibility APIs such as UI Automation. Or, if the protocol caught on and was supported by assistive technologies (e.g. screen readers), those ATs themselves could be the consumers. Indeed, I believe that applications pushing their accessibility information to ATs, which can then create and update their own retained trees, would be a better alternative to the pull-based request/response model that you mention. IMO, the bigger problem is the one that Raph pointed out: widgets (including simple ones like static text) need to have stable identities throughout their lifetimes, even as their state, contents, and locations change. It seems to me that IMGUIs, or at least some of them, make this particularly difficult.

liuliu 6y ago
Yes, that would work. But that would be a departure from existing imgui paradigm where the output of imgui library is a 2d bitmap. At that point, it gets really close to the vDOM territory ... As for the stable identities, yes, it is important. But I also don't see much of a problem with explicit Key: the UI components ultimately will have map back into a data object, which should have enough contextual information to provide the said Key.
- mwcampbell 6y ago
  You're right; for accessibility, at least for screen readers, you need something a lot like a DOM (whether real or virtual).
- stephc_int13 6y ago
  The output of most imgui libraries is not a 2D bitmap but a structure containing vertices, indices and texture handles, that can be used to render.