4 ms·
Since this is a research paper with promising ideas but non-functional code, what are people using as the best-in-class agents for computer automation? For exam
by patrickhogan1 2y ago
Since this is a research paper with promising ideas but non-functional code, what are people using as the best-in-class agents for computer automation? For example:
1. Claude for computer use
2. Various startup offerings—if you have recommendations, please list them
3. Established tools like Playwright, Selenium, and WebDriver, combined with screenshots and LLM-based guidance
What tools or approaches are actually working for building useful automation solutions?
- nmstoker 2y agoAre you sure about the non-working code point? I've yet to try it but my understanding is the repo here has got working code along with installation instructions: https://github.com/microsoft/OmniParser https://github.com/microsoft/OmniParser
- nmstoker 2y agoI confirm it works: I got the gradio demo working locally and it's pretty reasonable. Slight rough edges (to be expected) and you do need to read the README with attention but it's all par for the course. I had to install einops which wasn't in the requirements.txt and even though I had downloaded the HF models they released, it still needed to pull in another model when I first ran the demo.
- patrickhogan1 2y agoThanks for the tip, will try again.
- tomatohs 2y agoour agent is available via NPM: http://testdriver.ai http://testdriver.ai