> For starters, we could feed back screenshots of the generated website back to the LLM. But soon you’d want to give the LLM access to a full web browser so it

> For starters, we could feed back screenshots of the generated website back to the LLM. But soon you’d want to give the LLM access to a full web browser so it can itself poke around the app, like a human would, to see what features work and which ones don’t. We've had some success[1] with the screenshot to actions - using Gemini/Molmo and ADB on phones. And human like decisions was made by GPT 4o. It also recalibrates itself and says "oh we are still at the home screen, let's find the gmail app first" 1. https://github.com/BandarLabs/clickclickclick https://github.com/BandarLabs/clickclickclick - Letting AI control/use my phone.