6 ms·

As a partially sighted developer specializing in assistive technology, with several friends who are totally blind, I think this is an excellent project idea. T

As a partially sighted developer specializing in assistive technology, with several friends who are totally blind, I think this is an excellent project idea. The cheapest way to do it would probably be using the Orca screen reader for GNU/Linux, probably combined with the MATE desktop (forked from GNOME 2) so one doesn't have to worry about 3D acceleration in the VM, which will presumably be hosted remotely on a cloud provider somewhere. The main technical challenge that springs to mind will be capturing all keyboard events in a browser window. This is particularly important because screen readers tend to rely on esoteric keyboard commands, which repurpose keys like CapsLock and Insert as modifiers. I don't know if this can actually be done in a normal web browser. Anyway, just throwing out my quick thoughts on this. I don't currently have time to pursue it further myself.

JesseObrien 10y ago
Even better would be to get something that can run in a docker container that you can just download and press go!
- cryptarch 10y ago
  But that's a lot harder to monetize!
  - garysieling 10y ago
    There's a guy who has a book on computer vision, and he sells more expensive versions that come with test images, so it might actually be worth it. See- https://www.pyimagesearch.com/practical-python-opencv/ https://www.pyimagesearch.com/practical-python-opencv/ I've been working on a search engine for lectures (https://www.findlectures.com https://www.findlectures.com) and accessibility is an area I'd like to explore, but it's tough to know how to test something as an end user would see it.
    - blinddev 10y ago
      This is really interesting. I've often thought about a browser extension that could describe a web page. I don't mean text, as screen readers are (usually) more than capable there. I mean a visual description like: "The web site has a off white background with black text. There is a horizontal menu at the top that fills the screen 95% horizontally and 10% vertically. The horizontal menu has a navy blue background with white text. There is a logo on the left of the horizontal menu filling 25% of the screen horizontally and 5% vertically. There are five menu items to the right of the logo image in the horizontal menu." That's probably a little more verbose than it needs to be. But it could be a combination of CV, tapping into the renderer and traversing the DOM in order to best describe a page. Then if my screen reader isn't cutting it and I'm feeling lost I can just have the current view described to me.
      - mrkgnao 10y ago
        > The web site has a off white background with black text I'm genuinely curious noe: how would the descriptions of color help?
        trynewideas 10y ago
        It can provide two useful functions: - Context in places where color is used to relay information but the designer failed to provide alternative means of gaining that context without sight. For instance, if a text field is "grayed out" to indicate a certain state on the field, but that state isn't independently communicated by a screen reader. - The ability to more easily describe the page to a sighted person.
        jscholes 10y ago
        Also, some context for the blind person when designing their own pages. If they're aware that website X is considered to have a good design, they can get a description of it and know what colour schemes, positioning and placement, etc might work for them.
      - 10y ago
  - gargarplex 10y ago
    Can't you just charge $99 as a one time download license fee? Get people to buy bigger licenses if they use within the organizations? Have an anonymous whistleblower incentive fee program to combat people who would abuse the convenience factor (with heavy penalties in the EULA)? You could make the $99 renewing if you want to offer updates, and perhaps an option to buy the source code for, say, $495? Maybe you could make the software more expensive. Approximately 2.3% [1] of adults, non-institutionalized, male or female, all ages, with all education levels in the United States, are blind. If you could also provide the benefit of increasing sales, perhaps you could build a calculator (marketing tool; to estimate what percent of their customer base is blind and the money they're missing out on) and quantify that, that would help. You could especially capture some of that value if you bundle on consultancy services to perform the usability improvements. How many hours would it take to get someone to get this working in a docker container? How long would it take to put together a decent installation guide? https://nfb.org/blindness-statistics https://nfb.org/blindness-statistics
ndarilek 10y ago
I feel like you and I have had a vaguely related conversation before, but do any of the remote desktop access protocols support piping sound in addition to graphics? That might be a quick-and-dirty way to prototype something. I've been looking for a reason to play with Elixir/Phoenix, and if there's interest in this then I may try some sort of one-click Orca VM that pipes everything back to the browser. Interesting idea. Also, I feel like there was an early version/prototype of NVDA Remote that ran in the browser. I remember going to a page, turning on forms mode or whatever NVDA calls it (I've been out of the NVDA loop for a while) and I could send keys/get audio from the remote machine. I think that was before the addon was available so I'm pretty sure it was web native, but I could be misremembering. I don't think there's anything preventing transmission of the Insert key, at least. Capslock or other esoteric modifiers may be trickier.
- mwcampbell 10y ago
  RDP can carry audio.
  - camlorn38 10y ago
    It can, but not with a suitable latency for actual use. What you have to do is send the speech and ersynthesize on the far end. There's an NVDA issue for supporting RDP channels [0], but NVDA Remote made this very low priority and it's potentially blocking on another one to refactor how NVDA handles speech. 0: https://github.com/nvaccess/nvda/issues/3564 https://github.com/nvaccess/nvda/issues/3564
- bramd 10y ago
  Hmm, I don't remember any such prototype/demo. However, the NVDA remote protocol is pretty simple, JSON messages over a TCP socket. Putting those messages on a websocket should be fairly straightforward. With the web speech API landing in Firefox as well as Chrome[0] you could even synthesize the speech at the client. There would still be the problem of sending special key sequences. Not just ins/capslock, but also shortcuts that are normally capturen by the browser/OS might get tricky. Feel free to contact me if you want to develop an NVDA remote server in Elixir. I need a real project in Elixir to do more work in the language. I did some small Elixir projects and like it a lot. [0]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API#Browser_compatibility https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_...
avtar 10y ago
> I don't currently have time to pursue it further myself. I work for a non-profit where we tackle accessibility issues related to the web, documents, and tech in general. We have a few Vagrant boxes that we use for development and testing, one of them is a Fedora box (GNOME 3 though) that comes with Orca configured [1] so that it doesn't prompt you for setup options. Chrome and Firefox are installed as well. If you have Vagrant and VirtualBox installed you can make use of it like so: vagrant init inclusivedesign/fedora24 && vagrant up The box is ~2 GB. This is the repository for the box in question: * https://github.com/idi-ops/packer-fedora https://github.com/idi-ops/packer-fedora * https://atlas.hashicorp.com/inclusivedesign/boxes/fedora24 https://atlas.hashicorp.com/inclusivedesign/boxes/fedora24 We track Fedora releases and update boxes fairly regularly so there should be a Fedora 25 one with Orca once there's an official release upstream. I hope it can be of use to anyone here. If you have any questions we hang out in #fluid-work on Freenode. [1] https://github.com/gpii-ops/ansible-gpii-framework/blob/master/tasks/configure.yml https://github.com/gpii-ops/ansible-gpii-framework/blob/mast...
- blinddev 10y ago
  This is really cool, thanks for sharing. Will try it out.
- richardbrevig 10y ago
  > I work for a non-profit where we tackle accessibility issues related to the web, documents, and tech in general. What's the non-profit? Expose UX [1] is preparing an episode focused on accessibility for Global Accessibility Awareness Day: startups will get judged by UX judges based on the accessibility of their products. Would love to connect with your org. [1] http://ExposeUX.com http://ExposeUX.com
  - avtar 10y ago
    It's the Inclusive Design Research Centre at OCAD University. My email is in my profile, feel free to reach out :) http://inclusivedesign.ca/research/ocadu/ http://inclusivedesign.ca/research/ocadu/
- 10y ago
camlorn38 10y ago
Orca is a bad idea. Very few blind people use Linux because Linux desktop sucks, and Orca is always at least somewhat behind all the other screen readers. I have met a few blind people over the years who have used it for an ethical reason or because they can't afford Windows, but no one really likes it. Linux is indeed the cheapest option, but it's also almost completely ineffective in terms of making it actually work well for your users. For starters, even getting low latency out of the Linux audio stack is a major headache, and the synth situation is abysmal. You can't even touch the config files for these yourself because if you break either--even temporarily--you now can't use the computer. Then you get into how all the graphical desktops have accessibility issues to one degree or another and how you have to use a separate screen reader for anything outside them. What you want if you want a testing VM that actually has value is Windows and NVDA. NVDA is free in both senses and kind of the industry standard for sighted testing now. Jaws is still more popular, but this is slowly shifting in NVDA's favor. This would be because Jaws costs roughly $1000 per user. The advantage of NVDA is that you can be sure all users can have it, and if it works with NVDA then it's very likely to work with jaws without too much more work. But sadly you can't just test with one; in the end you have to test with all of them. Things like aria are nice if used correctly, but the aria spec doesn't say much about what screen readers have to do, and no one implements 100% of it. It's very close to the situation with needing to test on multiple browsers.

JesseObrien 10y ago

Even better would be to get something that can run in a docker container that you can just download and press go!

cryptarch 10y ago
But that's a lot harder to monetize!
- garysieling 10y ago
  There's a guy who has a book on computer vision, and he sells more expensive versions that come with test images, so it might actually be worth it. See- https://www.pyimagesearch.com/practical-python-opencv/ https://www.pyimagesearch.com/practical-python-opencv/ I've been working on a search engine for lectures (https://www.findlectures.com https://www.findlectures.com) and accessibility is an area I'd like to explore, but it's tough to know how to test something as an end user would see it.
  - blinddev 10y ago
    This is really interesting. I've often thought about a browser extension that could describe a web page. I don't mean text, as screen readers are (usually) more than capable there. I mean a visual description like: "The web site has a off white background with black text. There is a horizontal menu at the top that fills the screen 95% horizontally and 10% vertically. The horizontal menu has a navy blue background with white text. There is a logo on the left of the horizontal menu filling 25% of the screen horizontally and 5% vertically. There are five menu items to the right of the logo image in the horizontal menu." That's probably a little more verbose than it needs to be. But it could be a combination of CV, tapping into the renderer and traversing the DOM in order to best describe a page. Then if my screen reader isn't cutting it and I'm feeling lost I can just have the current view described to me.
    - mrkgnao 10y ago
      > The web site has a off white background with black text I'm genuinely curious noe: how would the descriptions of color help?
      - trynewideas 10y ago
        It can provide two useful functions: - Context in places where color is used to relay information but the designer failed to provide alternative means of gaining that context without sight. For instance, if a text field is "grayed out" to indicate a certain state on the field, but that state isn't independently communicated by a screen reader. - The ability to more easily describe the page to a sighted person.
        jscholes 10y ago
        Also, some context for the blind person when designing their own pages. If they're aware that website X is considered to have a good design, they can get a description of it and know what colour schemes, positioning and placement, etc might work for them.
    - 10y ago
- gargarplex 10y ago
  Can't you just charge $99 as a one time download license fee? Get people to buy bigger licenses if they use within the organizations? Have an anonymous whistleblower incentive fee program to combat people who would abuse the convenience factor (with heavy penalties in the EULA)? You could make the $99 renewing if you want to offer updates, and perhaps an option to buy the source code for, say, $495? Maybe you could make the software more expensive. Approximately 2.3% [1] of adults, non-institutionalized, male or female, all ages, with all education levels in the United States, are blind. If you could also provide the benefit of increasing sales, perhaps you could build a calculator (marketing tool; to estimate what percent of their customer base is blind and the money they're missing out on) and quantify that, that would help. You could especially capture some of that value if you bundle on consultancy services to perform the usability improvements. How many hours would it take to get someone to get this working in a docker container? How long would it take to put together a decent installation guide? https://nfb.org/blindness-statistics https://nfb.org/blindness-statistics

ndarilek 10y ago

I feel like you and I have had a vaguely related conversation before, but do any of the remote desktop access protocols support piping sound in addition to graphics? That might be a quick-and-dirty way to prototype something. I've been looking for a reason to play with Elixir/Phoenix, and if there's interest in this then I may try some sort of one-click Orca VM that pipes everything back to the browser. Interesting idea. Also, I feel like there was an early version/prototype of NVDA Remote that ran in the browser. I remember going to a page, turning on forms mode or whatever NVDA calls it (I've been out of the NVDA loop for a while) and I could send keys/get audio from the remote machine. I think that was before the addon was available so I'm pretty sure it was web native, but I could be misremembering. I don't think there's anything preventing transmission of the Insert key, at least. Capslock or other esoteric modifiers may be trickier.

mwcampbell 10y ago
RDP can carry audio.
- camlorn38 10y ago
  It can, but not with a suitable latency for actual use. What you have to do is send the speech and ersynthesize on the far end. There's an NVDA issue for supporting RDP channels [0], but NVDA Remote made this very low priority and it's potentially blocking on another one to refactor how NVDA handles speech. 0: https://github.com/nvaccess/nvda/issues/3564 https://github.com/nvaccess/nvda/issues/3564
bramd 10y ago
Hmm, I don't remember any such prototype/demo. However, the NVDA remote protocol is pretty simple, JSON messages over a TCP socket. Putting those messages on a websocket should be fairly straightforward. With the web speech API landing in Firefox as well as Chrome[0] you could even synthesize the speech at the client. There would still be the problem of sending special key sequences. Not just ins/capslock, but also shortcuts that are normally capturen by the browser/OS might get tricky. Feel free to contact me if you want to develop an NVDA remote server in Elixir. I need a real project in Elixir to do more work in the language. I did some small Elixir projects and like it a lot. [0]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API#Browser_compatibility https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_...

avtar 10y ago

> I don't currently have time to pursue it further myself. I work for a non-profit where we tackle accessibility issues related to the web, documents, and tech in general. We have a few Vagrant boxes that we use for development and testing, one of them is a Fedora box (GNOME 3 though) that comes with Orca configured [1] so that it doesn't prompt you for setup options. Chrome and Firefox are installed as well. If you have Vagrant and VirtualBox installed you can make use of it like so: vagrant init inclusivedesign/fedora24 && vagrant up The box is ~2 GB. This is the repository for the box in question: * https://github.com/idi-ops/packer-fedora https://github.com/idi-ops/packer-fedora * https://atlas.hashicorp.com/inclusivedesign/boxes/fedora24 https://atlas.hashicorp.com/inclusivedesign/boxes/fedora24 We track Fedora releases and update boxes fairly regularly so there should be a Fedora 25 one with Orca once there's an official release upstream. I hope it can be of use to anyone here. If you have any questions we hang out in #fluid-work on Freenode. [1] https://github.com/gpii-ops/ansible-gpii-framework/blob/master/tasks/configure.yml https://github.com/gpii-ops/ansible-gpii-framework/blob/mast...

blinddev 10y ago
This is really cool, thanks for sharing. Will try it out.
richardbrevig 10y ago
> I work for a non-profit where we tackle accessibility issues related to the web, documents, and tech in general. What's the non-profit? Expose UX [1] is preparing an episode focused on accessibility for Global Accessibility Awareness Day: startups will get judged by UX judges based on the accessibility of their products. Would love to connect with your org. [1] http://ExposeUX.com http://ExposeUX.com
- avtar 10y ago
  It's the Inclusive Design Research Centre at OCAD University. My email is in my profile, feel free to reach out :) http://inclusivedesign.ca/research/ocadu/ http://inclusivedesign.ca/research/ocadu/
10y ago

camlorn38 10y ago

Orca is a bad idea. Very few blind people use Linux because Linux desktop sucks, and Orca is always at least somewhat behind all the other screen readers. I have met a few blind people over the years who have used it for an ethical reason or because they can't afford Windows, but no one really likes it. Linux is indeed the cheapest option, but it's also almost completely ineffective in terms of making it actually work well for your users. For starters, even getting low latency out of the Linux audio stack is a major headache, and the synth situation is abysmal. You can't even touch the config files for these yourself because if you break either--even temporarily--you now can't use the computer. Then you get into how all the graphical desktops have accessibility issues to one degree or another and how you have to use a separate screen reader for anything outside them. What you want if you want a testing VM that actually has value is Windows and NVDA. NVDA is free in both senses and kind of the industry standard for sighted testing now. Jaws is still more popular, but this is slowly shifting in NVDA's favor. This would be because Jaws costs roughly $1000 per user. The advantage of NVDA is that you can be sure all users can have it, and if it works with NVDA then it's very likely to work with jaws without too much more work. But sadly you can't just test with one; in the end you have to test with all of them. Things like aria are nice if used correctly, but the aria spec doesn't say much about what screen readers have to do, and no one implements 100% of it. It's very close to the situation with needing to test on multiple browsers.