4 ms·
For websites (eg UI testing) you normally annotate things to be clicked, so I don't find this super compelling Parsing text (eg html) is a bit easier than pars
by wrsh07 3y ago
For websites (eg UI testing) you normally annotate things to be clicked, so I don't find this super compelling
Parsing text (eg html) is a bit easier than parsing images, although maybe the difference is going to zero
- hugs 3y agoGenerally, I agree, if you can get access to the underlying structure/source, then you should use that. But I've been making UI test automation tools for 20 years. There are some cases where adding annotations is not possible or not feasible at low-cost, but the feature still needs to be automated.
- zogrodea 3y agoFluent Search for Windows has a feature where typing a letter combination can click any element on the screen. I appreciate it for the same reason people like shortcuts. Fluent Search is a desktop app and this feature is meant to work regardless of which apps are on the screen (browser, Notepad, whatever else) so I think there's a legitimate point here although it might be a niche one not relevant to you. (I don't know enough about Windows internals to answer with confidence if a better solution is available like examining the OS accessibility tree, but I guess an advantage of Fluent Search's approach is that UIs from toolkits not implementing accessibility standards aren't excluded. The current approach has drawbacks like false positives too though because the ML model isn't 100% accurate.)