3 ms·
No. Computer use (to anthropic, as in the article) is an LLM controlling a computer via a video feed of the display, and controlling it with the mouse and keyb
by vineyardmike 8mo ago
No.
Computer use (to anthropic, as in the article) is an LLM controlling a computer via a video feed of the display, and controlling it with the mouse and keyboard.
- cowboylowrez 8mo agooh hell no haha maybe with THEIR login hahaha
- dbbk 8mo agoThat sounds weird. Why does it need a video feed? The computer can already generate an accessibility tree, same as Playwright uses it for webpages.
- 0sdi 8mo agoSo that it can utilize gui and interfaces designed for humans. Think of video editing program for example.
- dbbk 8mo agoYes. GUIs expose an accessibility tree.
- slopinthebag 8mo agoNot all of them do, and not all of the ones that do expose enough to be useful to the AI.
- bowsamic 8mo agoEven if they do (often not the case) this will be far from exhaustive, and likely won’t reflect the structure of the application very well. Vision based testing is often combined with accessibility based testing
- lsaferite 8mo agoI feel like a legion of blind computer users could attest to how bad accessibility is online. If you added AI Agents to the users of accessibility features you might even see a purposeful regression in the space.
- chasd00 8mo ago> controlling a computer via a video feed of the display, and controlling it with the mouse and keyboard. I guess that's one way to get around robots.txt. Claim that you would respect it but since the bot is not technically a crawler it doesn't apply. It's also an easier sell to not identify the bot in the user agent string because, hey, it's not a script, it's using the computer like a human would!
- jebus989 8mo agoEven simpler it just takes screenshots (or at least that's what it was doing last time I used it)