3 ms·

>How do you plan to address accessibility using imgui? I am not sure I understand this question.

by stephc_int13 6y ago

raphlinus 6y ago
> I am not sure I understand this question. I rest my case.
- boulos 6y ago
  I think it’s fair to say that accessibility is an ambiguous term. Which one did you mean? (My charitable guess was a11y in the screen reader, color blindness, font size, etc. sense)
  - raphlinus 6y ago
    You are right, but I couldn't resist the opportunity. Yes, just "accessibility" is a fairly broad term, but I do mean it in the context of assistive technology such as screen readers and navigation by keyboard and devices like directional controllers. I should note, we haven't actually done any of that work in Druid yet (we're struggling with scope as it is), but trying to think ahead. I believe that having a retained tree will help enormously with building this, and that, in the meantime, tab focus navigation is a good proxy for at least some of the problems we expect to face.
    - stephc_int13 6y ago
      It is a bit of work but navigation with keyboard/gamepad is possible with an immediate-mode GUI, as demonstrated by the excellent work of Omar Cornut (Dear Imgui) You have to set a compilation flag to enable the keyboard/gamepad navigation, if you missed it from the source code.
      - akiselev 6y ago
        Basic navigation with keyboard/gamepad is a trivial case compared to screen reader accessibility and navigation. Take a look at the Windows Narrator keyboard shortcuts [1] like "Next Heading" or "Go to next landmark/region" or "Read row header" - these require integration with the system UI framework or the accessibility APIs directly to provide extra semantic information about what's drawn on screen. Even with integration into system accessibility APIs, Narrator (and other screen readers) performs a lot of heuristics to improve the user experience. These heuristics depend on a separate tree data structure [2] and how it changes over time, so in order to implement even the most basic accessibility, IMGUI would have to retain that information instead of throwing it away every frame. Component identity is crucial to how most (all?) screen readers work today and it's a happy accident that it also allows an efficient hybrid retained-immediate mode architecture. I don't know if it has been fixed or not, but a year or two ago you could completely nuke the Windows screen reader just by assigning React components randomly generated keys so that the DOM nodes would be destroyed and recreated every redraw. [1] https://dequeuniversity.com/screenreaders/narrator-keyboard-shortcuts https://dequeuniversity.com/screenreaders/narrator-keyboard-... [2] https://docs.microsoft.com/en-us/windows/win32/winauto/inspect-objects https://docs.microsoft.com/en-us/windows/win32/winauto/inspe...
boulos 6y ago
I assume Raph meant in the screen reader sense of the term.
- stephc_int13 6y ago
  I don't see any limitations on this side, intuitively, but I might not know all the intricacies tied to this kind of implementation. If the ability to add metadata to any widget is not sufficient then I don't know.
  - boulos 6y ago
    Yeah, I’m not an a11y person (nor a UI or even 2d graphics person). What about the “zoom under my cursor” type things? That seems much harder (though not impossible) if your interface to clients is “just draw directly”. Though thinking about it, it’s actually probably what macOS does: as a post process, take your frame buffer and then magnify the stuff in a region of interest and re-render that on top. You’re winning me over :). Edit: I assume your metadata solution is “if someone asks, the text under this region of interest says < string >”.
    - stephc_int13 6y ago
      There is a common misconception about immediate-mode APIs, there are in practice very few implementations where rendering is done directly. A structure is built every frame with all the data to render later, as a batch. And of course if you need a magnify lense the obvious way would be to render it as a post process.

>How do you plan to address accessibility using imgui? I am not sure I understand this question.

raphlinus 6y ago

> I am not sure I understand this question. I rest my case.

boulos 6y ago
I think it’s fair to say that accessibility is an ambiguous term. Which one did you mean? (My charitable guess was a11y in the screen reader, color blindness, font size, etc. sense)
- raphlinus 6y ago
  You are right, but I couldn't resist the opportunity. Yes, just "accessibility" is a fairly broad term, but I do mean it in the context of assistive technology such as screen readers and navigation by keyboard and devices like directional controllers. I should note, we haven't actually done any of that work in Druid yet (we're struggling with scope as it is), but trying to think ahead. I believe that having a retained tree will help enormously with building this, and that, in the meantime, tab focus navigation is a good proxy for at least some of the problems we expect to face.
  - stephc_int13 6y ago
    It is a bit of work but navigation with keyboard/gamepad is possible with an immediate-mode GUI, as demonstrated by the excellent work of Omar Cornut (Dear Imgui) You have to set a compilation flag to enable the keyboard/gamepad navigation, if you missed it from the source code.
    - akiselev 6y ago
      Basic navigation with keyboard/gamepad is a trivial case compared to screen reader accessibility and navigation. Take a look at the Windows Narrator keyboard shortcuts [1] like "Next Heading" or "Go to next landmark/region" or "Read row header" - these require integration with the system UI framework or the accessibility APIs directly to provide extra semantic information about what's drawn on screen. Even with integration into system accessibility APIs, Narrator (and other screen readers) performs a lot of heuristics to improve the user experience. These heuristics depend on a separate tree data structure [2] and how it changes over time, so in order to implement even the most basic accessibility, IMGUI would have to retain that information instead of throwing it away every frame. Component identity is crucial to how most (all?) screen readers work today and it's a happy accident that it also allows an efficient hybrid retained-immediate mode architecture. I don't know if it has been fixed or not, but a year or two ago you could completely nuke the Windows screen reader just by assigning React components randomly generated keys so that the DOM nodes would be destroyed and recreated every redraw. [1] https://dequeuniversity.com/screenreaders/narrator-keyboard-shortcuts https://dequeuniversity.com/screenreaders/narrator-keyboard-... [2] https://docs.microsoft.com/en-us/windows/win32/winauto/inspect-objects https://docs.microsoft.com/en-us/windows/win32/winauto/inspe...

boulos 6y ago

I assume Raph meant in the screen reader sense of the term.

stephc_int13 6y ago
I don't see any limitations on this side, intuitively, but I might not know all the intricacies tied to this kind of implementation. If the ability to add metadata to any widget is not sufficient then I don't know.
- boulos 6y ago
  Yeah, I’m not an a11y person (nor a UI or even 2d graphics person). What about the “zoom under my cursor” type things? That seems much harder (though not impossible) if your interface to clients is “just draw directly”. Though thinking about it, it’s actually probably what macOS does: as a post process, take your frame buffer and then magnify the stuff in a region of interest and re-render that on top. You’re winning me over :). Edit: I assume your metadata solution is “if someone asks, the text under this region of interest says < string >”.
  - stephc_int13 6y ago
    There is a common misconception about immediate-mode APIs, there are in practice very few implementations where rendering is done directly. A structure is built every frame with all the data to render later, as a batch. And of course if you need a magnify lense the obvious way would be to render it as a post process.