4 ms·
This is really cool. Have you thought of maybe accessing the screen through accessibility APIs? For Android mobile devices I have a skill I created that accesse
by dbdoskey 7mo ago
This is really cool. Have you thought of maybe accessing the screen through accessibility APIs? For Android mobile devices I have a skill I created that accesses the screen xml dump as part of feature development and it seems to work much better than screenshots / videos. Is this scalable to other OS's?
- msephton 7mo agoIt's trivial in Xcode Simulator, for Apple platform coverage.
- jberthom 6mo agoYes for now focused on web but accessibility tree dumps seems like a good alternative to screenshots for native apps. For web, agent-browser already uses compact element refs, but for mobile the a11y approach could be way more efficient. Would you be open to sharing more about how you set that up?