2 ms·
llama.cpp grammars will get you results that definitely conform to your grammar, but that doesn't guarantee that they'll be semantically correct - the model cou
by simonw 3y ago
llama.cpp grammars will get you results that definitely conform to your grammar, but that doesn't guarantee that they'll be semantically correct - the model could still hallucinate details incorrectly while returning valid JSON.
- _flux 3y agoIndeed it's not the solution to LLM hallucinations—as far as I know nobody knows a solution to it. But it is the solution to needing to run the model again and checking the format of the output to ensure that it conforms to you expectations.