6 ms·

Why not push the screen reader component upstream? It'd be another service add-on, but it might also be useful for folks who want to have narrative browsing, e

Why not push the screen reader component upstream? It'd be another service add-on, but it might also be useful for folks who want to have narrative browsing, e.g., the equivalent of someone reading the news sites to the listener without having to interact with the site itself.

mwcampbell 5y ago
> Why not push the screen reader component upstream? Are you suggesting that a screen reader should run on the same remote machine as the remote browser and push its audio down to the client? Or something else?
- daviddever23box 5y ago
  Yes - in the same manner as game streaming.
  - mwcampbell 5y ago
    That would be better than nothing, and if Cloudflare had done that, I don't think I'd be complaining publicly at this point. But there are still problems with this approach. The ones that come to mind: 1. Assuming the remote service only sends down streaming audio, this doesn't work for blind people that must use a refreshable Braille display, e.g. deafblind people. Perhaps one could hack a way to get their local screen reader to render specific text on the Braille display, but probably not without that screen reader speaking the same text. That leads me to... 2. A blind user is already running a screen reader, with its own text-to-speech engine, configured the way they want it. Adding a remote screen reader to the mix would mean two different TTS engines, and the user would need to have a way of configuring the remote one, e.g. to adjust its speed. For blind people, TTS settings are very personal. 3. The remote screen reader and the local one may clash on keyboard commands. And, depending on the screen reader, this is another thing that the use may have customized already; for example, some screen readers have desktop and laptop keymaps. 4. Also speaking of keyboard commands, some of them might not be implementable in a browser-based application. It's common, at least on Windows, for screen readers to use non-standard modifier keys, e.g. Insert or Caps Lock.
    - daviddever23box 5y ago
      To point 1, text/plain can be streamed; as for point 2, there may already be limited options, subject to application security audits. I hate to say this, but if there was one place I'd look for vulnerabilities within a purportedly-secure environment, screen readers would be near the top of the list.
    - mst 5y ago
      Pipedream thought: A client-server model so the text can be provided by the server and the TTS engine can still run under the user's control. Then again, I guess my pipedream is equivalent to "re-implement the chrome accessibility system except somehow magically without any user-after-free or etc. bug" :(
      - mwcampbell 5y ago
        No, what you're suggesting is actually quite feasible. Using an off-screen HTML element marked as an ARIA live region, a web page can make the user's screen reader speak arbitrary text with its normal TTS configuration. The screen reader would likely even render that text on the user's Braille display if they have one, though I'm honestly not expert enough in the Braille implementations of various screen readers to know what the limitations might be. So that solves problem #2 in my list (TTS), but may not be an adequate solution for #1 (Braille). And that still leaves the problem of keyboard commands.
        mst 5y ago
        I (shockingly enough) have no clue wrt Braille. However, I think Ins sends a normal keydown/keyup and while capslock is a bit more complicated https://stackoverflow.com/questions/39016292/keydown-event-is-not-fired-for-capslock-in-mac https://stackoverflow.com/questions/39016292/keydown-event-i... suggests it's not that hard to capture it (for front end javascript values of 'not that hard' ;). Using the 'correct' keybindings for any given screen reader would presumably require a sort of javascript 'skin' to handle that, of course, and that doesn't solve the key clash problem at all but I'm not sure what can really be done about that except maybe remap things that are in the way on the browser side. (spitballing in the hopes some of my random thoughts give you ideas ;)
marcinzm 5y ago
A screen reader is a two way device since it needs to expose ways to INTERACT with the site and not just read it. I assume there's many different settings for screen readers including voices, speed, ways of interaction with site elements (click, voice command, shortcuts, etc.), etc. It'd be like forcing you to use IE 6 to browse the modern web and then if you're not as efficient as someone on modern Chrome firing you.
- mwcampbell 5y ago
  > It'd be like forcing you to use IE 6 to browse the modern web and then if you're not as efficient as someone on modern Chrome firing you. I might have to use that analogy next time this comes up.
- daviddever23box 5y ago
  This scenario might very well require a FedGov or security audit-compliant reader application with a uniform interface.

mwcampbell 5y ago

> Why not push the screen reader component upstream? Are you suggesting that a screen reader should run on the same remote machine as the remote browser and push its audio down to the client? Or something else?

daviddever23box 5y ago
Yes - in the same manner as game streaming.
- mwcampbell 5y ago
  That would be better than nothing, and if Cloudflare had done that, I don't think I'd be complaining publicly at this point. But there are still problems with this approach. The ones that come to mind: 1. Assuming the remote service only sends down streaming audio, this doesn't work for blind people that must use a refreshable Braille display, e.g. deafblind people. Perhaps one could hack a way to get their local screen reader to render specific text on the Braille display, but probably not without that screen reader speaking the same text. That leads me to... 2. A blind user is already running a screen reader, with its own text-to-speech engine, configured the way they want it. Adding a remote screen reader to the mix would mean two different TTS engines, and the user would need to have a way of configuring the remote one, e.g. to adjust its speed. For blind people, TTS settings are very personal. 3. The remote screen reader and the local one may clash on keyboard commands. And, depending on the screen reader, this is another thing that the use may have customized already; for example, some screen readers have desktop and laptop keymaps. 4. Also speaking of keyboard commands, some of them might not be implementable in a browser-based application. It's common, at least on Windows, for screen readers to use non-standard modifier keys, e.g. Insert or Caps Lock.
  - daviddever23box 5y ago
    To point 1, text/plain can be streamed; as for point 2, there may already be limited options, subject to application security audits. I hate to say this, but if there was one place I'd look for vulnerabilities within a purportedly-secure environment, screen readers would be near the top of the list.
  - mst 5y ago
    Pipedream thought: A client-server model so the text can be provided by the server and the TTS engine can still run under the user's control. Then again, I guess my pipedream is equivalent to "re-implement the chrome accessibility system except somehow magically without any user-after-free or etc. bug" :(
    - mwcampbell 5y ago
      No, what you're suggesting is actually quite feasible. Using an off-screen HTML element marked as an ARIA live region, a web page can make the user's screen reader speak arbitrary text with its normal TTS configuration. The screen reader would likely even render that text on the user's Braille display if they have one, though I'm honestly not expert enough in the Braille implementations of various screen readers to know what the limitations might be. So that solves problem #2 in my list (TTS), but may not be an adequate solution for #1 (Braille). And that still leaves the problem of keyboard commands.
      - mst 5y ago
        I (shockingly enough) have no clue wrt Braille. However, I think Ins sends a normal keydown/keyup and while capslock is a bit more complicated https://stackoverflow.com/questions/39016292/keydown-event-is-not-fired-for-capslock-in-mac https://stackoverflow.com/questions/39016292/keydown-event-i... suggests it's not that hard to capture it (for front end javascript values of 'not that hard' ;). Using the 'correct' keybindings for any given screen reader would presumably require a sort of javascript 'skin' to handle that, of course, and that doesn't solve the key clash problem at all but I'm not sure what can really be done about that except maybe remap things that are in the way on the browser side. (spitballing in the hopes some of my random thoughts give you ideas ;)

marcinzm 5y ago

A screen reader is a two way device since it needs to expose ways to INTERACT with the site and not just read it. I assume there's many different settings for screen readers including voices, speed, ways of interaction with site elements (click, voice command, shortcuts, etc.), etc. It'd be like forcing you to use IE 6 to browse the modern web and then if you're not as efficient as someone on modern Chrome firing you.

mwcampbell 5y ago
> It'd be like forcing you to use IE 6 to browse the modern web and then if you're not as efficient as someone on modern Chrome firing you. I might have to use that analogy next time this comes up.
daviddever23box 5y ago
This scenario might very well require a FedGov or security audit-compliant reader application with a uniform interface.