3 ms·
Neat! Can you describe how the integration works and what APIs you implemented?
by cloudking 3y ago
Neat! Can you describe how the integration works and what APIs you implemented?
- aronse 3y agoAs far as APIs we only used the regular chat completion api and promoted the model to output a list of issues with a title and XPath. The rest is leveraged by our existing chrome extension to capture the screenshot, show the pins, etc.
- cloudking 3y agoAre you passing the entire context of the page code to the LLM to get back the XPath?
- aronse 3y agoYes, the whole document via document.documentElement.outerHTML. Surely there's a lot of optimizations we could look into but this was more to try if the model could localize elements via XPath by using screenshot + HTML. It's doing a decent job for the most part, I definitely have seen some XPath hallucinations though.