4 ms·
iOS already has such a feature, but it's obviously not 100% accurate, not real time, and not great for battery life. It's good enough if you have to click a br
by miki123211 3y ago
iOS already has such a feature, but it's obviously not 100% accurate, not real time, and not great for battery life.
It's good enough if you have to click a broken "I accept your terms and conditions" checkbox, but nowhere near good enough to daily drive your phone with. In other words, a band-aid solution for when everything else fails, mostly for situations where the app you're trying to use is mostly usable, but has an accessibility barrier preventing you from carrying out a crucial step somewhere.
There's also vocr on Mac (and equivalent solutions on Windows), which recently got some AI features, but it doesn't even recognize control types (a very basic feature of any screen reader), just text. Again, good enough to get you through most installers where all you're doing is clicking "next" ten times, probably good enough to get you through the first-run experience in a VM that doesn't yet have a screen reader installed, at one tenth the speed of a sighted person, but that's about it.
- pmarreck 3y agoAdding control type recognition seems like THE most trivial add-on feature to implement with an AI-powered solution. I mean, at that point it's just about training data... "Here are 100 different button looks. Here are 100 different radio button looks. Here are 100 different checkboxes. Here are 100 different dropdown menus." etc.