4 ms·
I think the reason is that this is the most general implementation. It doesn't need playwright or have access to the DOM or anything else, if it has a screen an
by danielbln 2y ago
I think the reason is that this is the most general implementation. It doesn't need playwright or have access to the DOM or anything else, if it has a screen and mouse/keyboard, then it will work. That's quite powerful (if slow and pricey, at the moment).
- _heimdall 2y agoUnless I'm mistaken, playwright doesn't actually use the accessibility tree directly. It does have quite a few APIs for accessing nodes based on a11y attributes, but I could have sworn those were glorified query selectors rather than directly accessing the accessibility tree. Last I checked on it, maybe a year ago, there were browser proposals for standardizing the accessibility tree APIs but they were very early discussions and seemed pretty well stuck. That would be a good reason for Anthropic using image processing here though, short of forking open source a11y tools there may not have been a simple way to use accessibility data to interact.