3 ms·

Screen readers, despite the name, don't do OCR, they access informations provided by the GUI toolkit (which is one of the vastly improved area in GTK 4 Afaik).

Blikkentrekker 6y ago
Then why don't actual screen readers exist when a.i.'s at this point can practically solve captcha's by enhancing the reflexion in the eyes of a highly compressed jpeg and reading the text in there? Certainly there would be significant demand for a sightless man to be able to read the dankest memes from pictures?
- danShumway 6y ago
  > Then why don't actual screen readers exist when a.i.'s at this point can practically solve captcha's Training a computer to solve a captcha is a lot easier than training a computer to understand interface conventions. There isn't an AI that can look at a jpeg screenshot of an interface and say, "there's two input elements, and it looks like they're grouped together and control the list to the left of them, and one of them is selected, which I can tell because it has some kind of subtle glow effect on it, but not the glow effect you would get if you moused over it." There's nothing that can realistically do that today, and it wouldn't be fast enough or performant enough for low-powered cell phones and laptops even if it did exist. If you're just looking at describing pictures themselves... sure, Facebook does auto-generate alt tags for images if you forget to put one in. And Youtube auto-generates captions. Those are valuable services, but they have a lot of glitches and mistakes. If you're a blind reader, you'd prefer not to have that experience when you're using a piece of software, you'd prefer something that just works reliably. It's the same reason you probably used a keyboard to type this comment instead of speech to text. Speech to text is useful in some cases, but not good enough or accurate enough that you would want to use it as your main input method.
mwcampbell 6y ago
Some screen readers can do OCR on a specific part of the screen (e.g. an unlabeled image) on request. But while OCR is useful for getting text out of an image, most implementations can't discern the structure of a UI, e.g. which part is a button, which part is an edit box, etc. Also, OCR is typically done just once on-demand, not continuously as the screen changes. However, VoiceOver for iOS has a new feature called screen recognition, which is exciting because it overcomes these limitations and provides some level of access to applications that are otherwise inaccessible. Hopefully other platforms will catch up. Even then, true screen reading will be much more CPU-intensive than what screen readers currently do. And anyway, it's not here yet, except on iOS. So I will continue to warn developers away from toolkits that are inaccessible, in hopes that some blind person somewhere will be spared the pain of being blocked from doing a task because of an inaccessible application.

Blikkentrekker 6y ago

Then why don't actual screen readers exist when a.i.'s at this point can practically solve captcha's by enhancing the reflexion in the eyes of a highly compressed jpeg and reading the text in there? Certainly there would be significant demand for a sightless man to be able to read the dankest memes from pictures?

danShumway 6y ago
> Then why don't actual screen readers exist when a.i.'s at this point can practically solve captcha's Training a computer to solve a captcha is a lot easier than training a computer to understand interface conventions. There isn't an AI that can look at a jpeg screenshot of an interface and say, "there's two input elements, and it looks like they're grouped together and control the list to the left of them, and one of them is selected, which I can tell because it has some kind of subtle glow effect on it, but not the glow effect you would get if you moused over it." There's nothing that can realistically do that today, and it wouldn't be fast enough or performant enough for low-powered cell phones and laptops even if it did exist. If you're just looking at describing pictures themselves... sure, Facebook does auto-generate alt tags for images if you forget to put one in. And Youtube auto-generates captions. Those are valuable services, but they have a lot of glitches and mistakes. If you're a blind reader, you'd prefer not to have that experience when you're using a piece of software, you'd prefer something that just works reliably. It's the same reason you probably used a keyboard to type this comment instead of speech to text. Speech to text is useful in some cases, but not good enough or accurate enough that you would want to use it as your main input method.

mwcampbell 6y ago

Some screen readers can do OCR on a specific part of the screen (e.g. an unlabeled image) on request. But while OCR is useful for getting text out of an image, most implementations can't discern the structure of a UI, e.g. which part is a button, which part is an edit box, etc. Also, OCR is typically done just once on-demand, not continuously as the screen changes. However, VoiceOver for iOS has a new feature called screen recognition, which is exciting because it overcomes these limitations and provides some level of access to applications that are otherwise inaccessible. Hopefully other platforms will catch up. Even then, true screen reading will be much more CPU-intensive than what screen readers currently do. And anyway, it's not here yet, except on iOS. So I will continue to warn developers away from toolkits that are inaccessible, in hopes that some blind person somewhere will be spared the pain of being blocked from doing a task because of an inaccessible application.