6 ms·

While I do enjoy a nice and smooth gpu-accelerated ui, I never use a gpu-ui framework for my own project for one simple reason: Almost none of them properly sup

by maeln 4y ago

While I do enjoy a nice and smooth gpu-accelerated ui, I never use a gpu-ui framework for my own project for one simple reason: Almost none of them properly support accessibility. Electron (and in general the web), despite its sluggishness has a very good support for accessibility. Most "traditional" native ui toolkit also do. That would be my advice to anyone making a gpu-accelerated ui library in 2023: Try to support accessibility, and even better: make it a first class citizen.

bruce343434 4y ago
If you want to do this, where can you start? What are some patterns for making code that's not too spaghetti when you have to handle tabbing, focus, layout, speech of element contents, the actual hierarchy of the elements etc? Are there standardized OS accessibility API hooks or something?
- Gigachad 4y ago
  I’ve heard it’s hard to even work this out as all of the screen reader tools are expensive, proprietary, and there are no standards. The typical way is to just make your program, and if it gets popular, the screen reader companies will find a way to make their product work.
  - lukastyrychtr 4y ago
    That's partially true, but fortunately not completely. There are widely use open-source screen readers for Windows and, of course, there's no proprietary screen reader on Linux. And, definitely, there are standard APIs which are used to communicate the accessibility tree between an app and a screen reader. Yes, they are specific for each platform, and Windows has multiple of these, but they are standardized at least for each platform.
  - illiarian 4y ago
    > work this out as all of the screen reader tools are expensive, proprietary, and there are no standards. ARIA is a good start, and screen readers built into the OS are a good start. Moreover, major OSes have accessibility APIs that screen readers will use: - MacOS https://developer.apple.com/library/archive/documentation/Accessibility/Conceptual/AccessibilityMacOSX/index.html https://developer.apple.com/library/archive/documentation/Ac... - Windows: https://learn.microsoft.com/en-us/windows/apps/develop/accessibility https://learn.microsoft.com/en-us/windows/apps/develop/acces...
  - pflanze 4y ago
    Here are recent suggestions for Windows and Linux by a blind person: https://news.ycombinator.com/item?id=35008647 https://news.ycombinator.com/item?id=35008647
  - mwcampbell 4y ago
    This used to be the case on Windows, but hasn't been for at least 10 years, and has never been the case on Mac OS X (using the historical name for clarity) or Linux. On Windows, the open-source NVDA screen reader is widely used. Furthermore, the hacks that Windows screen reader developers historically used to "find a way to make their product work", particularly intercepting GDI API calls (either in user space or in a display driver) to build an off-screen model, are not applicable to modern UI stacks. And the other major screen reader hack, using application-specific COM object models, was mostly only applicable to Microsoft Office and Internet Explorer. So you basically have to implement platform accessibility APIs to make your UI accessible. (If you use native controls, that's more or less done for you.) Edit: BTW, I've been in the assistive technology industry a while, particularly on Windows. Feel free to ask me anything.
    - GreedCtrl 4y ago
      As someone who's doing some accessibility programming but has no background in the field, I do have a random question to throw your way if you are game. Our team is developing voice control for a website. This is mainly requested by sighted users who want to use the site hands-free. But we also have an accessibility mode for better screen reader support. We think voice control might be appreciated by screen reader users too, but we aren't sure how well it would work with a screen reader. Are there common pitfalls we should be wary of? One thing we're worried about is that the voice from the screen reader might interfere with the detection of the user's voice. I know general dictation and voice control software already exists, so my initial assumption that screen reader users would benefit might be wrong. If the existing tools are good enough, perhaps this whole question is moot.
      - mwcampbell 4y ago
        Yes, the screen reader output will probably interfere with your speech recognition. You may be able to work around that on some platforms by enabling echo cancellation when getting mic input from navigator.getUserMedia, but I don't know if that actually works on any desktop platforms. In general, you should assume that the user already has whatever assistive technologies they need, and you don't need to provide your own, just make the content and UI accessible using semantic HTML and (if needed) ARIA. Providing your own screen reader, for example, would definitely be a mistake. The same should ideally also hold for voice control, but apparently you're actually getting some demand for that.
- wnkrshm 4y ago
  Since browsers can interface, I would guess there are hooks but I would also guess that they are not standardized (between platforms).
- rikroots 4y ago
  I can't speak for GPU solutions, but I do have some experience of trying to make HTML 2d <canvas> elements as accessible as possible. You can get an overview of the issues/solutions (disclaimer: using my canvas library) here - https://scrawl-v8.rikweb.org.uk/learn/eleventh-lesson/ https://scrawl-v8.rikweb.org.uk/learn/eleventh-lesson/
littlestymaar 4y ago
The Rust GUI ecosystem looks particularly promising in that regard, because there is a foundational accessibility library called Accesskit that's being incorporated in several UI frameworks (egui being the first one to have it already but works is underway do add it in several other places)
nicoburns 4y ago
I can’t speak for this one as it’s proprietary, but You’ll be pleased to hear that pretty much all the open source Rust GUI toolkits either integrate AccessKit or have concrete plans to do so in the immediate future. There are toolkits that can’t even do basic things like render images but have accessibility support :)
- maeln 4y ago
  That is good to hear !

by maeln 4y ago

bruce343434 4y ago

If you want to do this, where can you start? What are some patterns for making code that's not too spaghetti when you have to handle tabbing, focus, layout, speech of element contents, the actual hierarchy of the elements etc? Are there standardized OS accessibility API hooks or something?

Gigachad 4y ago
I’ve heard it’s hard to even work this out as all of the screen reader tools are expensive, proprietary, and there are no standards. The typical way is to just make your program, and if it gets popular, the screen reader companies will find a way to make their product work.
- lukastyrychtr 4y ago
  That's partially true, but fortunately not completely. There are widely use open-source screen readers for Windows and, of course, there's no proprietary screen reader on Linux. And, definitely, there are standard APIs which are used to communicate the accessibility tree between an app and a screen reader. Yes, they are specific for each platform, and Windows has multiple of these, but they are standardized at least for each platform.
- illiarian 4y ago
  > work this out as all of the screen reader tools are expensive, proprietary, and there are no standards. ARIA is a good start, and screen readers built into the OS are a good start. Moreover, major OSes have accessibility APIs that screen readers will use: - MacOS https://developer.apple.com/library/archive/documentation/Accessibility/Conceptual/AccessibilityMacOSX/index.html https://developer.apple.com/library/archive/documentation/Ac... - Windows: https://learn.microsoft.com/en-us/windows/apps/develop/accessibility https://learn.microsoft.com/en-us/windows/apps/develop/acces...
- pflanze 4y ago
  Here are recent suggestions for Windows and Linux by a blind person: https://news.ycombinator.com/item?id=35008647 https://news.ycombinator.com/item?id=35008647
- mwcampbell 4y ago
  This used to be the case on Windows, but hasn't been for at least 10 years, and has never been the case on Mac OS X (using the historical name for clarity) or Linux. On Windows, the open-source NVDA screen reader is widely used. Furthermore, the hacks that Windows screen reader developers historically used to "find a way to make their product work", particularly intercepting GDI API calls (either in user space or in a display driver) to build an off-screen model, are not applicable to modern UI stacks. And the other major screen reader hack, using application-specific COM object models, was mostly only applicable to Microsoft Office and Internet Explorer. So you basically have to implement platform accessibility APIs to make your UI accessible. (If you use native controls, that's more or less done for you.) Edit: BTW, I've been in the assistive technology industry a while, particularly on Windows. Feel free to ask me anything.
  - GreedCtrl 4y ago
    As someone who's doing some accessibility programming but has no background in the field, I do have a random question to throw your way if you are game. Our team is developing voice control for a website. This is mainly requested by sighted users who want to use the site hands-free. But we also have an accessibility mode for better screen reader support. We think voice control might be appreciated by screen reader users too, but we aren't sure how well it would work with a screen reader. Are there common pitfalls we should be wary of? One thing we're worried about is that the voice from the screen reader might interfere with the detection of the user's voice. I know general dictation and voice control software already exists, so my initial assumption that screen reader users would benefit might be wrong. If the existing tools are good enough, perhaps this whole question is moot.
    - mwcampbell 4y ago
      Yes, the screen reader output will probably interfere with your speech recognition. You may be able to work around that on some platforms by enabling echo cancellation when getting mic input from navigator.getUserMedia, but I don't know if that actually works on any desktop platforms. In general, you should assume that the user already has whatever assistive technologies they need, and you don't need to provide your own, just make the content and UI accessible using semantic HTML and (if needed) ARIA. Providing your own screen reader, for example, would definitely be a mistake. The same should ideally also hold for voice control, but apparently you're actually getting some demand for that.
wnkrshm 4y ago
Since browsers can interface, I would guess there are hooks but I would also guess that they are not standardized (between platforms).
rikroots 4y ago
I can't speak for GPU solutions, but I do have some experience of trying to make HTML 2d <canvas> elements as accessible as possible. You can get an overview of the issues/solutions (disclaimer: using my canvas library) here - https://scrawl-v8.rikweb.org.uk/learn/eleventh-lesson/ https://scrawl-v8.rikweb.org.uk/learn/eleventh-lesson/

littlestymaar 4y ago

The Rust GUI ecosystem looks particularly promising in that regard, because there is a foundational accessibility library called Accesskit that's being incorporated in several UI frameworks (egui being the first one to have it already but works is underway do add it in several other places)

nicoburns 4y ago

I can’t speak for this one as it’s proprietary, but You’ll be pleased to hear that pretty much all the open source Rust GUI toolkits either integrate AccessKit or have concrete plans to do so in the immediate future. There are toolkits that can’t even do basic things like render images but have accessibility support :)

maeln 4y ago
That is good to hear !