6 ms·

> I'll be using gtk-rs thank you very much. Please be aware that if you do this, your application won't be accessible with screen readers or other assistive te

> I'll be using gtk-rs thank you very much. Please be aware that if you do this, your application won't be accessible with screen readers or other assistive technologies on Windows and Mac. At least not now. Maybe I'll have time to implement GTK accessibility backends for those platforms someday.

Blikkentrekker 6y ago
Screen readers rely on some kind of toolkit a.p.i.? I would assume they would simply read the screen. Are they thus not capable of, for instance, reading a picture? They should not be called “screen readers” but “text to speech” if they not actually read the screen on a bitmap level.
- riquito 6y ago
  Screen readers, despite the name, don't do OCR, they access informations provided by the GUI toolkit (which is one of the vastly improved area in GTK 4 Afaik).
  - Blikkentrekker 6y ago
    Then why don't actual screen readers exist when a.i.'s at this point can practically solve captcha's by enhancing the reflexion in the eyes of a highly compressed jpeg and reading the text in there? Certainly there would be significant demand for a sightless man to be able to read the dankest memes from pictures?
    - danShumway 6y ago
      > Then why don't actual screen readers exist when a.i.'s at this point can practically solve captcha's Training a computer to solve a captcha is a lot easier than training a computer to understand interface conventions. There isn't an AI that can look at a jpeg screenshot of an interface and say, "there's two input elements, and it looks like they're grouped together and control the list to the left of them, and one of them is selected, which I can tell because it has some kind of subtle glow effect on it, but not the glow effect you would get if you moused over it." There's nothing that can realistically do that today, and it wouldn't be fast enough or performant enough for low-powered cell phones and laptops even if it did exist. If you're just looking at describing pictures themselves... sure, Facebook does auto-generate alt tags for images if you forget to put one in. And Youtube auto-generates captions. Those are valuable services, but they have a lot of glitches and mistakes. If you're a blind reader, you'd prefer not to have that experience when you're using a piece of software, you'd prefer something that just works reliably. It's the same reason you probably used a keyboard to type this comment instead of speech to text. Speech to text is useful in some cases, but not good enough or accurate enough that you would want to use it as your main input method.
  - mwcampbell 6y ago
    Some screen readers can do OCR on a specific part of the screen (e.g. an unlabeled image) on request. But while OCR is useful for getting text out of an image, most implementations can't discern the structure of a UI, e.g. which part is a button, which part is an edit box, etc. Also, OCR is typically done just once on-demand, not continuously as the screen changes. However, VoiceOver for iOS has a new feature called screen recognition, which is exciting because it overcomes these limitations and provides some level of access to applications that are otherwise inaccessible. Hopefully other platforms will catch up. Even then, true screen reading will be much more CPU-intensive than what screen readers currently do. And anyway, it's not here yet, except on iOS. So I will continue to warn developers away from toolkits that are inaccessible, in hopes that some blind person somewhere will be spared the pain of being blocked from doing a task because of an inaccessible application.
- deleted 6y ago
  [deleted]
- simias 6y ago
  What you describe might be more robust in some situations but vastly less for software designed to be accessible. Take "alt" attributes in HTML img tags for instance, or various metadata attached to buttons that use an icon instead of text (like a play button, or a X close button and similar things). You can see an example next to this very comment actually: the up and downvote buttons won't be accessible with OCR, but they have "title" attributes describing what they are. And consider that there's more to understanding a given user interface that raw text: radio buttons are tied to certain labels, there's hierarchy, all sorts of layout cues that would be opaque to a screen reader. I suppose that ideally you'd want both: use native accessibility data if available, fallback to OCR when there's no alternative.
- danShumway 6y ago
  Converting bitmaps to text isn't enough to make an interface usable. You need to be able to quickly convey the structure of the interface and what controls are available, and to do that well you need some kind of semantic insight into the interface. Screenreaders don't just read text, they control the interface itself using standardized keyboard shortcuts and input components within whatever graphical framework you're using, and they communicate what that interface is using a set of standardized terminology.
  - Blikkentrekker 6y ago
    I indeed am imparted that the term “screen reader” is a misnomer and not what I expected it to be. This seems to be more so an “accessibility suite” of which text-to-speech is a component rather than what I would call a “screen reader".
    - danShumway 6y ago
      Sorry, I guess? It's been an industry standard phrase for a pretty long time. Screenreaders probably have a historical reason why they're named the way they are, but the short version is that's just what everyone started using. A lot of software terminology is like that, it's weird to people who are unfamiliar with it because there's no central committee somewhere that decides what everything should be named. If you're trying to do a search online for the kind of tool you're looking for, probably the phrase you would want to search for is "OCR software", short for Optical Character Recognition, or if you're trying to tag images just straight-up "image recognition."
phkahler 6y ago
Yet another reason for them to do this. Not just a GUI for Rust, but the only accessible one. It really is a solid strategic offering to bring Rust developers to their Windows platform. But IMHO developers who do are trading tomorrow for today.
- mwcampbell 6y ago
  Actually, Qt is also accessible (more or less) on all the desktop platforms, so that's another option.
  - Shared404 6y ago
    Is there a good way to use Qt from Rust? I looked the other day, but couldn't figure it out.
    - varajelle 6y ago
      https://github.com/woboq/qmetaobject-rs https://github.com/woboq/qmetaobject-rs
    - RMPR 6y ago
      You might want to have a look at this blog post[0]. It presents an up to date version of the state of GUIs in Rust 0: https://dev.to/davidedelpapa/rust-gui-introduction-a-k-a-the-state-of-rust-gui-libraries-as-of-january-2021-40gl#comments https://dev.to/davidedelpapa/rust-gui-introduction-a-k-a-the...
ddevault 6y ago
This only matters if you care about propiretary platforms like Windows and Mac. And you should not care!
- mwcampbell 6y ago
  I still do, because people are stuck with these platforms, for reasons beyond our control, and I care more about people's access to applications that they need than about being a purist.
- rodgerd 6y ago
  "Fuck people who need assistive technologies" does not make you a good person.
  - kiawe_fire 6y ago
    More like “fuck people who use Mac OS or Windows”. I saw no malice there directed at users of assistive technology, but rather of users of non-open platforms.
    - kevingadd 6y ago
      If the assistive technology they need to use isn't available on Linux or they don't have the ability to run Linux it's not that meaningful of a difference.
      - rodgerd 6y ago
        Exactly. It's a difference without a distinction. And rather sad that free software - which in theory ought to be a perfect place for exploring assistive options - seems to lag far behind the closed shops.
        mwcampbell 6y ago
        Having discussed this with Drew, I think he was really talking about what I, personally, should choose to work on. And I think he's right that I should work on the accessibility of free software as much as I can, rather than accessibility on proprietary platforms. I don't know yet how soon I can actually start doing this.

Blikkentrekker 6y ago

Screen readers rely on some kind of toolkit a.p.i.? I would assume they would simply read the screen. Are they thus not capable of, for instance, reading a picture? They should not be called “screen readers” but “text to speech” if they not actually read the screen on a bitmap level.

riquito 6y ago
Screen readers, despite the name, don't do OCR, they access informations provided by the GUI toolkit (which is one of the vastly improved area in GTK 4 Afaik).
- Blikkentrekker 6y ago
  Then why don't actual screen readers exist when a.i.'s at this point can practically solve captcha's by enhancing the reflexion in the eyes of a highly compressed jpeg and reading the text in there? Certainly there would be significant demand for a sightless man to be able to read the dankest memes from pictures?
  - danShumway 6y ago
    > Then why don't actual screen readers exist when a.i.'s at this point can practically solve captcha's Training a computer to solve a captcha is a lot easier than training a computer to understand interface conventions. There isn't an AI that can look at a jpeg screenshot of an interface and say, "there's two input elements, and it looks like they're grouped together and control the list to the left of them, and one of them is selected, which I can tell because it has some kind of subtle glow effect on it, but not the glow effect you would get if you moused over it." There's nothing that can realistically do that today, and it wouldn't be fast enough or performant enough for low-powered cell phones and laptops even if it did exist. If you're just looking at describing pictures themselves... sure, Facebook does auto-generate alt tags for images if you forget to put one in. And Youtube auto-generates captions. Those are valuable services, but they have a lot of glitches and mistakes. If you're a blind reader, you'd prefer not to have that experience when you're using a piece of software, you'd prefer something that just works reliably. It's the same reason you probably used a keyboard to type this comment instead of speech to text. Speech to text is useful in some cases, but not good enough or accurate enough that you would want to use it as your main input method.
- mwcampbell 6y ago
  Some screen readers can do OCR on a specific part of the screen (e.g. an unlabeled image) on request. But while OCR is useful for getting text out of an image, most implementations can't discern the structure of a UI, e.g. which part is a button, which part is an edit box, etc. Also, OCR is typically done just once on-demand, not continuously as the screen changes. However, VoiceOver for iOS has a new feature called screen recognition, which is exciting because it overcomes these limitations and provides some level of access to applications that are otherwise inaccessible. Hopefully other platforms will catch up. Even then, true screen reading will be much more CPU-intensive than what screen readers currently do. And anyway, it's not here yet, except on iOS. So I will continue to warn developers away from toolkits that are inaccessible, in hopes that some blind person somewhere will be spared the pain of being blocked from doing a task because of an inaccessible application.
deleted 6y ago
[deleted]
simias 6y ago
What you describe might be more robust in some situations but vastly less for software designed to be accessible. Take "alt" attributes in HTML img tags for instance, or various metadata attached to buttons that use an icon instead of text (like a play button, or a X close button and similar things). You can see an example next to this very comment actually: the up and downvote buttons won't be accessible with OCR, but they have "title" attributes describing what they are. And consider that there's more to understanding a given user interface that raw text: radio buttons are tied to certain labels, there's hierarchy, all sorts of layout cues that would be opaque to a screen reader. I suppose that ideally you'd want both: use native accessibility data if available, fallback to OCR when there's no alternative.
danShumway 6y ago
Converting bitmaps to text isn't enough to make an interface usable. You need to be able to quickly convey the structure of the interface and what controls are available, and to do that well you need some kind of semantic insight into the interface. Screenreaders don't just read text, they control the interface itself using standardized keyboard shortcuts and input components within whatever graphical framework you're using, and they communicate what that interface is using a set of standardized terminology.
- Blikkentrekker 6y ago
  I indeed am imparted that the term “screen reader” is a misnomer and not what I expected it to be. This seems to be more so an “accessibility suite” of which text-to-speech is a component rather than what I would call a “screen reader".
  - danShumway 6y ago
    Sorry, I guess? It's been an industry standard phrase for a pretty long time. Screenreaders probably have a historical reason why they're named the way they are, but the short version is that's just what everyone started using. A lot of software terminology is like that, it's weird to people who are unfamiliar with it because there's no central committee somewhere that decides what everything should be named. If you're trying to do a search online for the kind of tool you're looking for, probably the phrase you would want to search for is "OCR software", short for Optical Character Recognition, or if you're trying to tag images just straight-up "image recognition."

phkahler 6y ago

Yet another reason for them to do this. Not just a GUI for Rust, but the only accessible one. It really is a solid strategic offering to bring Rust developers to their Windows platform. But IMHO developers who do are trading tomorrow for today.

mwcampbell 6y ago
Actually, Qt is also accessible (more or less) on all the desktop platforms, so that's another option.
- Shared404 6y ago
  Is there a good way to use Qt from Rust? I looked the other day, but couldn't figure it out.
  - varajelle 6y ago
    https://github.com/woboq/qmetaobject-rs https://github.com/woboq/qmetaobject-rs
  - RMPR 6y ago
    You might want to have a look at this blog post[0]. It presents an up to date version of the state of GUIs in Rust 0: https://dev.to/davidedelpapa/rust-gui-introduction-a-k-a-the-state-of-rust-gui-libraries-as-of-january-2021-40gl#comments https://dev.to/davidedelpapa/rust-gui-introduction-a-k-a-the...

ddevault 6y ago

This only matters if you care about propiretary platforms like Windows and Mac. And you should not care!

mwcampbell 6y ago
I still do, because people are stuck with these platforms, for reasons beyond our control, and I care more about people's access to applications that they need than about being a purist.
rodgerd 6y ago
"Fuck people who need assistive technologies" does not make you a good person.
- kiawe_fire 6y ago
  More like “fuck people who use Mac OS or Windows”. I saw no malice there directed at users of assistive technology, but rather of users of non-open platforms.
  - kevingadd 6y ago
    If the assistive technology they need to use isn't available on Linux or they don't have the ability to run Linux it's not that meaningful of a difference.
    - rodgerd 6y ago
      Exactly. It's a difference without a distinction. And rather sad that free software - which in theory ought to be a perfect place for exploring assistive options - seems to lag far behind the closed shops.
      - mwcampbell 6y ago
        Having discussed this with Drew, I think he was really talking about what I, personally, should choose to work on. And I think he's right that I should work on the accessibility of free software as much as I can, rather than accessibility on proprietary platforms. I don't know yet how soon I can actually start doing this.