4 ms·
Not that it matters now but just doing something like this works 99% of the time or more with 4 and 90% with 3.5. It is VERY IMPORTANT that you respond in vali
by jonplackett 3y ago
Not that it matters now but just doing something like this works 99% of the time or more with 4 and 90% with 3.5.
It is VERY IMPORTANT that you respond in valid JSON ONLY. Nothing before or after. Make sure to escape all strings. Use this format:
{“some_variable”: [describe the variable purpose]}
- SamPatt 3y ago99% of the time is still super frustrating when it fails, if you're using it in a consumer facing app. You have to clean up the output to avoid getting an error. If it goes from 99% to 100% JSON that is a big deal for me, much simpler.
- golergka 3y agoIf you're building an app based on LLMs that expects higher than 99% correctness from it, you are bound to fail. Negative scenarios workarounds and retries are mandatory.
- jonplackett 3y agoExcept it says in the small print to expect invalid JSON occasionally, so you have to write your error handling code either way
- davepeck 3y agoYup. Is there a good/forgiving "drunken JSON parser" library that people like to use? Feels like it would be a useful (and separable) piece?
- golol 3y agoHonestly, I suspect asking GPT-4 to fix your JSON (in a new chat) is a good drunken JSON parser. We are only scraping the surface of what's possible with LLMs. If Token generation was free and instant we could come up with a giant schema of interacting model calls that generates 10 suggestions, iterates over them, ranks them and picks the best one, as silly as it sounds.
- 8organicbits 3y agoWouldn't you use traditional software to validate the JSON, then ask chatgpt to try again if it wasn't right?
- girvo 3y agoIn my experience, telling it "no thats wrong, try again" just gets it to be wrong in a new different way, or restate the same wrong answer slightly differently. I've had to explicitly guide it to correct answers or formats at times.