3 ms·
I believe we need to bypass all of these bespoke accessibility solutions and use machine learning to visually parse a live feed of your computer screen. We shou
by suby 3y ago
I believe we need to bypass all of these bespoke accessibility solutions and use machine learning to visually parse a live feed of your computer screen. We should be able to take a given screenshot and determine where the buttons are, what the text is, and make something that is decently workable. Then we won't need to care about each app, each website, each operating system supporting accessibility standards because our accessibility tools will be using the computer in the same manner that humans do, visually decoding a live feed of images.
Easier said than done, but given what is happening in AI right now, it doesn't seem far fetched to think that we'll be able to accomplish this.
- paulryanrogers 3y agoI was hoping to do something like this for adventure game hotspots after a hand injury. Even experimented with AutoIt and its OCR, before giving up. Now there is another niche project with the goal of translation: translate.net With more capable models it's going to be fun to see what folks come up with! Still, I hope internal efforts get some attention as they don't need to rely on hacks and awkward workarounds.
- kaba0 3y agoIt doesn’t sound unachievably hard, but I cry for all the “wasted” energy it would require.