5 ms·
Yeah, but now ask it to write a program that uses this API and then let it debug problems which arise from the swagger spec (or the backend) having bugs. I don'
by execveat 4y ago
Yeah, but now ask it to write a program that uses this API and then let it debug problems which arise from the swagger spec (or the backend) having bugs. I don't think LLMs have any way of recognizing and dealing with bad input data. That is I don't think they can recognize when something that is supposed to work in a particular way doesn't and fixing it is completely out of your reach, but you still need to get things working (by introducing workarounds).
- sebzim4500 4y agoHave you tried it? If you copy the errors back into the chat I could imagine it working quite well. Certainly you can give it contradictory instructions and it makes a decent effort at following them.
- execveat 4y agoYes, I'm subscribed to poe.com and am playing with all public models. They all suck at debugging issues with no known answers (I'm talking about typical problems every software developer, DevOps or infosec person solves every day). You need a real ability to reason and preserve context beyond inherent context window somehow (we humans do it by keeping notes, writing emails, and filing JIRA tickets). So while this doesn't require full AGI and some form of AI might be able to do it this century, it won't be LLMs.
- ilaksh 4y agoIf you think that the average public LLM is equivalent to ChatGPT or GPT-4 then you are completely mistaken. By a factor of say 500-10000%.
- execveat 4y agopoe.com is a web interface (by Quora) to multiple LLMs. Right now it's ChatGPT, GPT-4, Claude, Claude+ as well as Sage and Dragonfly.