3 ms·
Could someone help me with context? I'm OOTL and don't understand what is going on here.
by bavarianbob 3y ago
Could someone help me with context? I'm OOTL and don't understand what is going on here.
- brucethemoose2 3y agoThis can constrain an LLM's output to an arbitrary grammar/format as it is generated, rather than asking the model to output a specific format and hoping it outputs something valid.
- Tostino 3y agoThis is important for "smaller" models, because you don't have to waste some of the potential "intelligence" (parameter space) on training it how to generate valid JSON or YAML or anything like that.
- brucethemoose2 3y agoYou still do... The model has to know JSON and YAML, its just more reliable when the generation is enforced by grammar
- Tostino 3y agoRight, but there is a big difference between ”generally knows what JSON looks like and gets it right most of the time" and "generates perfect JSON every time".