The LLM should not be seeing the raw DOM in its context window, but a highly simplified and compact version of it. In general LLMs perform worse both when the

The LLM should not be seeing the raw DOM in its context window, but a highly simplified and compact version of it. In general LLMs perform worse both when the context is larger and also when the context is less information dense. To achieve good performance, all input to the prompt must be made as compact and information dense as possible. I built a similar tool as well, but for automating generation of E2E browser tests. Further, you can have sub-LLMs help with compacting aspects of the context prior to handing it off to the main LLM. (Note: it's important that, by design, HTML selectors cannot be hallucinated) Modern LLMs are absolutely capable of interpreting web pages proficiently if implemented well. That being said, things like this Claude product seem to be fundamentally poorly designed from both a security and general approach perspective and I don't agree at all that prompt engineering is remotely the right way to remediate this. There are so many companies pushing out junk products where the AI is just handling the wrong part of the loop and pulls in far too much context to perform well.