3 ms·
It can't hallucinate, but it doesn't mean it can't make wrong decisions. Just because it adheres to a specific output format at all time, while LLMs have the ou
by flyinglizard 19d ago
It can't hallucinate, but it doesn't mean it can't make wrong decisions. Just because it adheres to a specific output format at all time, while LLMs have the output format at their mercy, then the claim of not hallucinating is made technically true.
I think that this specific part is not super interesting if your harness just recovers from invalid LLM outputs.
The latency and cost - yes, those are super interesting.
- mhitza 19d agoYou can get rigid output format from "classic" LLMs https://docs.vllm.ai/en/latest/features/structured_outputs/ https://docs.vllm.ai/en/latest/features/structured_outputs/ though model support is limited. Would like to have something like in the original post but open weights.