6 ms·

This, hundreds times this. I can't say how many years in a row I was experimenting with various GUI toolkits, libraries and concepts of an application like rxi/

by pqb 5y ago

This, hundreds times this. I can't say how many years in a row I was experimenting with various GUI toolkits, libraries and concepts of an application like rxi/lite editor [0]. While I saw many good looking implementations but the very first what caught my attention was lack of any kind support to screen-reading software or problem with displaying right-click context menus. I even remember previous HN discussion when I pointed out few projects that could provide some screen-readers support [1]. Sadly, I would say 95% of all GUIs I know do not care about the accessibility. From the positive side, I am glad more and more hackers on HN focus on that matter. Also worth noting big players on the market really do take that point seriously, e.g. Flutter has a Semantics tag [2]. [0]: https://github.com/rxi/lite https://github.com/rxi/lite [1]: https://news.ycombinator.com/item?id=26223380 https://news.ycombinator.com/item?id=26223380 [2]: https://api.flutter.dev/flutter/widgets/Semantics-class.html https://api.flutter.dev/flutter/widgets/Semantics-class.html

amelius 5y ago
In my opinion, accessibility should be implemented at the OS level. The OS can, for example, run OCR on the entire screen and turn bitmaps into selectable text, read text out loud, etc. This kind of functionality shouldn't be replicated on a per-app basis. Accessibility of browsers is usually considered to be very good, but browsers are almost OSes, so why not lift this one more level into the OS?
- zxzax 5y ago
  That is not really going to work, at all. To have a good experience, a screen reader needs more contextual information than just the text.
- masklinn 5y ago
  > In my opinion, accessibility should be implemented at the OS level. Which it is. Apple has been praised for the accessibility of both macOS and iOS for years if not decades, accessibility concerns have been baked into Cocoa forever. And accessibility is literally a top level category of the Settings app. And I know that Windows has been improving by leaps and bounds. > The OS can, for example, run OCR on the entire screen and turn bitmaps into selectable text, read text out loud, etc That exists (look up VoiceOver Recognition). However it can not be reliable, and will never be anywhere near as good as actual semantic annotation. Image recognition has no way to understand that the physical UI layout has no relation to its logical setup, nor does it have any way to differentiate between semantic and decorative content, or to see through invisibility to know that a button is a menu versus an action.
- p_l 5y ago
  OSes do provide accessibility APIs - and usually if you use one of the more supported graphic toolkits there are somewhat easy ways to integrate directly in semantic ways, without needing to OCR and do fortune telling. Usually it means that you need a deferred mode semantic tree of the application for accessibility UI to walk through, though.
slimsag 5y ago
If we want more accessible UIs, we need to demand better standard accessibility APIs from OS vendors Microsoft and Apple. Unity has no official support for screenreaders or colorblind modes. Unreal has both. If an major game engine developer like Unity cannot be bothered to add support, can we really expect the little developer hacking away on some OpenGL side project to? Most people working with OpenGL use GLFW (or sometimes SDL), neither have any cross-platform API for supporting screen readers. Why? Because not only do Windows and MacOS have different APIs, but different screen readers, braille displays, etc. have different APIs. But okay - let's look at the web. It's the most accessible platform of them all, right? What have Google/Apple/Mozilla done to support developers adding screenreader support from WebGL and Canvas-based applications? -> not a whole lot. You need to inject text into a hidden div, which also has performance implications so you need to build a UI to toggle it on/off or detect accessibility the way Flutter Web does. We should have better accessibility, but as long as we expect developers to run through several hours/days of hoops to get even something basic working - it's just not going to happen, and that is mostly the fault of extremely poor platform APIs.
- iudqnolq 5y ago
  > Unity has no official support for screenreaders or colorblind modes. Unreal has both. If an major game engine developer like Unity cannot be bothered to add support, can we really expect the little developer hacking away on some OpenGL side project to? Games are special because the means of interaction is often a core part of the experience. An accessibility mode for a game can be a huge amount of work. Not to say that some people don't do that huge amount of work (which is great) but games lacking accessibility is no excuse to use game-like development practices to write apps that could otherwise be relatively easily made accessibile.
- mwcampbell 5y ago
  > extremely poor platform APIs As a former member of the Windows accessibility team at Microsoft, I'd appreciate your thoughts on what makes the platform accessibility APIs extremely poor. I have my own thoughts on what makes them difficult to implement, but I'd like to hear your perspective first.

by pqb 5y ago

amelius 5y ago

In my opinion, accessibility should be implemented at the OS level. The OS can, for example, run OCR on the entire screen and turn bitmaps into selectable text, read text out loud, etc. This kind of functionality shouldn't be replicated on a per-app basis. Accessibility of browsers is usually considered to be very good, but browsers are almost OSes, so why not lift this one more level into the OS?

zxzax 5y ago
That is not really going to work, at all. To have a good experience, a screen reader needs more contextual information than just the text.
masklinn 5y ago
> In my opinion, accessibility should be implemented at the OS level. Which it is. Apple has been praised for the accessibility of both macOS and iOS for years if not decades, accessibility concerns have been baked into Cocoa forever. And accessibility is literally a top level category of the Settings app. And I know that Windows has been improving by leaps and bounds. > The OS can, for example, run OCR on the entire screen and turn bitmaps into selectable text, read text out loud, etc That exists (look up VoiceOver Recognition). However it can not be reliable, and will never be anywhere near as good as actual semantic annotation. Image recognition has no way to understand that the physical UI layout has no relation to its logical setup, nor does it have any way to differentiate between semantic and decorative content, or to see through invisibility to know that a button is a menu versus an action.
p_l 5y ago
OSes do provide accessibility APIs - and usually if you use one of the more supported graphic toolkits there are somewhat easy ways to integrate directly in semantic ways, without needing to OCR and do fortune telling. Usually it means that you need a deferred mode semantic tree of the application for accessibility UI to walk through, though.

slimsag 5y ago

If we want more accessible UIs, we need to demand better standard accessibility APIs from OS vendors Microsoft and Apple. Unity has no official support for screenreaders or colorblind modes. Unreal has both. If an major game engine developer like Unity cannot be bothered to add support, can we really expect the little developer hacking away on some OpenGL side project to? Most people working with OpenGL use GLFW (or sometimes SDL), neither have any cross-platform API for supporting screen readers. Why? Because not only do Windows and MacOS have different APIs, but different screen readers, braille displays, etc. have different APIs. But okay - let's look at the web. It's the most accessible platform of them all, right? What have Google/Apple/Mozilla done to support developers adding screenreader support from WebGL and Canvas-based applications? -> not a whole lot. You need to inject text into a hidden div, which also has performance implications so you need to build a UI to toggle it on/off or detect accessibility the way Flutter Web does. We should have better accessibility, but as long as we expect developers to run through several hours/days of hoops to get even something basic working - it's just not going to happen, and that is mostly the fault of extremely poor platform APIs.

iudqnolq 5y ago
> Unity has no official support for screenreaders or colorblind modes. Unreal has both. If an major game engine developer like Unity cannot be bothered to add support, can we really expect the little developer hacking away on some OpenGL side project to? Games are special because the means of interaction is often a core part of the experience. An accessibility mode for a game can be a huge amount of work. Not to say that some people don't do that huge amount of work (which is great) but games lacking accessibility is no excuse to use game-like development practices to write apps that could otherwise be relatively easily made accessibile.
mwcampbell 5y ago
> extremely poor platform APIs As a former member of the Windows accessibility team at Microsoft, I'd appreciate your thoughts on what makes the platform accessibility APIs extremely poor. I have my own thoughts on what makes them difficult to implement, but I'd like to hear your perspective first.