3 ms·
I mean, it's a structured output model that (apparently) can't hallucinate. I don't mind the name.
by toddmorey 19d ago
I mean, it's a structured output model that (apparently) can't hallucinate. I don't mind the name.
- flyinglizard 19d agoIt can't hallucinate, but it doesn't mean it can't make wrong decisions. Just because it adheres to a specific output format at all time, while LLMs have the output format at their mercy, then the claim of not hallucinating is made technically true. I think that this specific part is not super interesting if your harness just recovers from invalid LLM outputs. The latency and cost - yes, those are super interesting.
- mhitza 18d agoYou can get rigid output format from "classic" LLMs https://docs.vllm.ai/en/latest/features/structured_outputs/ https://docs.vllm.ai/en/latest/features/structured_outputs/ though model support is limited. Would like to have something like in the original post but open weights.