4 ms·

> I've succeeded in getting it to interface with Playwright over a JSON API I wrote and prompted it to use, so it can use it for sure if you give it the command

> I've succeeded in getting it to interface with Playwright over a JSON API I wrote and prompted it to use, so it can use it for sure if you give it the commands. But you're always limited by context, so you have to use obvious tricks like constraining the viewport and walking the DOM, checking what's in the viewport, etc. I have created something similar in the last few weeks. With puppeteer instead of playwright. And yeah basically feeding gpt4 filtered views of webpages. Full blown assistant is the idea.

srslack 4y ago
>Full blown assistant Well, by that I meant personal assistant, for actions with access to email and a password manager for anything personally associated without a chance to review. That'd be a bit inconvenient in my own case anyways, because hardware key. You're braver than me, for sure. It's a pain when it runs for so long with how far it gets away from the original prompt, but light prompts with the system message do help, as well as a way for it to refresh its "commands" and goals/tasks back into the context window. It's like Memento, with the complex system of Polaroids, notes and tattoos. I don't know if it's possible to develop this stuff in the open without a headache considering all of the hysteria, but was thinking if I fixed consistent output for my browser command and got a swarm working I'd try putting it on GitHub. Mainly I want to make sure things like Google Sheets are output correctly, try to leverage as much accessibility for SPAs that try, as I can.