7 ms·
It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphaza
by jonplackett 3y ago
It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job.
GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output.
I have been using this to make a few ‘choose your own adventure’ type games and I can see there’s a TONNE of potential useful things.
- cwxm 3y agoeven with gpt 4, it hallucinates enough that it’s not reliable, forgetting to open/close brackets and quotes. This sounds like it’d be a big improvement.
- ztratar 3y agoNah, this was solved by most teams a while ago.
- bel423 3y agoI feel like I’m taking crazy pills with the amount of people saying this is game changing. Did they not even try asking gpt to format the output as json?
- worik 3y ago> I feel like I’m taking crazy pills....try asking gpt to format the output as json You are taking crazey pills. Stop gpt-? is unreliable! That is not a bug in it, it is the nature of the beast. It is not an expert at anything except natural language, and even then it is an idiot savant
- jonplackett 3y agoNot that it matters now but just doing something like this works 99% of the time or more with 4 and 90% with 3.5. It is VERY IMPORTANT that you respond in valid JSON ONLY. Nothing before or after. Make sure to escape all strings. Use this format: {“some_variable”: [describe the variable purpose]}
- SamPatt 3y ago99% of the time is still super frustrating when it fails, if you're using it in a consumer facing app. You have to clean up the output to avoid getting an error. If it goes from 99% to 100% JSON that is a big deal for me, much simpler.
- golergka 3y agoIf you're building an app based on LLMs that expects higher than 99% correctness from it, you are bound to fail. Negative scenarios workarounds and retries are mandatory.
- jonplackett 3y agoExcept it says in the small print to expect invalid JSON occasionally, so you have to write your error handling code either way
- davepeck 3y agoYup. Is there a good/forgiving "drunken JSON parser" library that people like to use? Feels like it would be a useful (and separable) piece?
- golol 3y agoHonestly, I suspect asking GPT-4 to fix your JSON (in a new chat) is a good drunken JSON parser. We are only scraping the surface of what's possible with LLMs. If Token generation was free and instant we could come up with a giant schema of interacting model calls that generates 10 suggestions, iterates over them, ranks them and picks the best one, as silly as it sounds.
- hhh 3y agoI already do this today to create domain-specific knowledge focused prompts and then have them iterate back and forth and a ‘moderator’ that chooses what goes in and what doesn’t.
- 3y ago
- whateveracct 3y agoIt forgets commas too
- bradly 3y agoI could not get GPT-4 to reliably not give some sort of text response, even if was just a simple "Sure" followed by the JSON.
- rytill 3y agoDid you try using the API and providing a very clear system message followed by several examples that were pure JSON?
- bradly 3y agoYep. I even gave it a JSON schema file to use. It just wouldn't stop added extra verbage.
- NicoJuicy 3y agoSay it's a json API and may only reply with valid json without explanation.
- bradly 3y agoLol yes of course I tried that.
- dror 3y agoI've had good luck with both: https://github.com/drorm/gish/blob/main/tasks/coding.txt https://github.com/drorm/gish/blob/main/tasks/coding.txt and https://github.com/drorm/gish/blob/main/tasks/webapp.txt https://github.com/drorm/gish/blob/main/tasks/webapp.txt With the second one, I reliably generated half a dozen apps with one command. Not to say that it won't fail sometimes.
- NicoJuicy 3y agoCombine both ? :)
- 3y ago
- reallymental 3y agoIs there any publicly available resource replicate your work? I would love to just find the right kind of "incantation" for the gpt-3.5-t or gpt-4 to output a meaningful story arc etc. Any examples of your work would be greatly helpful as well!
- SamPatt 3y agoI'm not the person you're asking, but I built a site that allows you to generate fiction if you have an OpenAI API key. You can see the prompts sent in console, and it's all open source: https://havewords.ai/ https://havewords.ai/
- devbent 3y agoI have an open source project doing exactly this at https://www.generativestorytelling.ai/ https://www.generativestorytelling.ai/ GitHub link is on the main page!
- ignite 3y ago> You just append ‘reply in json with this format’ and it does a really good job. It does an ok job. Except when it doesn't. Definitely misses a lot of the time, sometimes on prompts that succeeded on previous runs.
- deleted 3y ago[deleted]
- bel423 3y agoIt literally does it everytime perfectly. I remember I put together an entire system that would validate the JSON against a zod schema and use reflection to fix it and it literally never gets triggered because GPT3.5-turbo always does it right the first time.
- worik 3y ago> It literally does it everytime perfectly. I remember I put together an entire system that would validate the JSON against a zod schema and use reflection to fix it and it literally never gets triggered because GPT3.5-turbo always does it right the first time. Danger! There be assumptions!! gpt-? is a moving target and in rapid development. What it does Tuesday, which it did not do on Monday, it may well not do on Wednesday If there is a documented method to guarantee it, it will work that way (modulo OpenAI bugs - and now Microsoft is involved....) What we had before, what you are talking of, was observed behaviour. An assumption that what we observed in the past will continue in the future is not something to build a business on
- travisjungroth 3y agoChatGPT moves fast. The API version doesn’t seem to change except with the model and documented API changes.
- whateveracct 3y agoNo it doesn't lol. I've seen it just randomly not use a comma after one array element, for example.
- sethd 3y agoI like to define a JSON schema (https://json-schema.org/ https://json-schema.org/) and prompt GPT-4 to output JSON based on that schema. This lets me specify general requirements (not just JSON structure) inline with the schema and in a very detailed and structured manor.
- seizethecheese 3y agoIn a production system, you don’t need easy to do most of the time, you need easy without fail.
- pnpnp 3y agoOk, just playing devil's advocate here. How many FAANG companies have you seen have an outage this year? What's their budget? I think a better way to reply to the author would have been "how often does it fail"? Every system will have outages, it's just a matter of how much money you can throw at the problem to reduce them.
- jrockway 3y agoIf 99.995% correct looks bad to users, wait until they see 37%.
- muzani 3y agoIt's fine, but the article makes some good points why - less cognitive load for GPT and less tokens. I think the transistor to logic gate analogy makes sense. You can build the thing perfectly with transistors, but just use the logic gate lol.
- sheepscreek 3y agoThe solution that worked great for me - do not use JSON for GPT to agent communication. Use comma separated key=value, or something to that effect. Then have another pure code layer to parse that into structured JSON. I think it’s the JSON syntax (with curly braces) that does it in. So YAML or TOML might work just as well, but I haven’t tried that.
- bombela 3y agoIt's harder to form a tree with key value. I also tried the relational route. But it would always messup the cardinality (one person should have 0 or n friends, but a person has a single birth date).
- sheepscreek 3y agoYou could flatten it using namespaced keys. Eg. { parent1: { child1: value } } Becomes one of the following: parent1/child1=value parent1_child1=value parent1.child1=value ..you get the idea.
- rubyskills 3y agoIt's also harder to stream JSON? Maybe I'm overthinking this.
- jacobsimon 3y agoCoincidentally, I just published this JS library[1] over the weekend that helps prompt LLMs to return typed JSON data and validates it for you. Would love feedback on it if this is something people here are interested in. Haven’t played around with the new API yet but I think this is super exciting stuff! [1] https://github.com/jacobsimon/prompting https://github.com/jacobsimon/prompting
- golergka 3y agoLooks promising! Do you do retries when returned json is invalid? Personally, I used io-ts for parsing, and GPT seems to be able to correct itself easily when confronted with a well-formed error message.
- throwuwu 3y agoJust end your request with ‘’’json Or provide a few examples of user request and then agent response in json. Or both.