3 ms·

Interesting! How far away is this tool from being a desktop simple RPA solution? Would it be able to interact with desktop apps and simulate mouse or keyboard a

• wewtyflakes 2y ago
  Right now we are constrained to using the DOM of webpages. This is because we need some reasonable way to detect what parts of a UI are interactable or not (though we do some magic to do this); this would be a bigger challenge for arbitrary desktop apps. Also, there would be new security aspects that would have to be meticulously considered if we were to allow an agent to control arbitrary desktop apps. Constraining the agent to the browser has nice alignment in this way.