4 ms·
Interesting, seems to use 'pure' vision and x/y coords for clicking stuff. Most other browser automation with LLMs I've seen uses the dom/accessibility tree whi
by martinald 1y ago
Interesting, seems to use 'pure' vision and x/y coords for clicking stuff. Most other browser automation with LLMs I've seen uses the dom/accessibility tree which absolutely churns through context, but is much more 'accurate' at clicking stuff because it can use the exact text/elements in a selector.
Unfortunately it really struggled in the demos for me. It took nearly 18 attempts to click the comment link on the HN demo, each a few pixels off.
- deleted 1y ago[deleted]
- pbhjpbhj 1y ago18 attempts - emulating the human HN experience when using mobile. Well, assuming it hit other links it didn't intend to anyway. /jk