3 ms·

Thanks for sharing your experience. I've built most parts of the agent already though I haven't tested it with any calendar apps yet. One option I'm working on

Thanks for sharing your experience. I've built most parts of the agent already though I haven't tested it with any calendar apps yet. One option I'm working on is enabling vision so the agent can see the browser state via screenshots and interact with visual elements like date pickers. It also has an orchestrator that can split a larger task into multiple sub agents running in parallel. Browser profiles are supported too. I'm using OpenAI models (gpt-4.1-mini works really well so far) and the project is written in node js. I plan to open source it once it's more polished but right now I'm especially curious about how people from different backgrounds would want to use such a tool.

ttoinou 1y ago
Couldn't it rather know the HTML source and JS state ? Like if it was plugged into Chrome DevTools. It might be more straightforward and better inputs for AI to interact with
- hamzamairaj 1y ago
  That's already there, by default the agent can see the HTML elements in the current viewport and interact with them properly. It works quite well for most cases but in some special scenarios (Like your calendar app example) we still need vision as a fallback.