4 ms·
I mirror this sentiment. Even their "function calling" abstraction still hallucinates parameters and schema, and the JSON schema itself is clearly way too verbo
by sippeangelo 2y ago
I mirror this sentiment. Even their "function calling" abstraction still hallucinates parameters and schema, and the JSON schema itself is clearly way too verbose and breaks down completely if you feed it anything more complex than 5 very simple function calls. This just seems to build upon their already broken black box abstractions and isn't useful for any real world applications, but it's helpful for getting small proof-of-concept apps going, I guess...
- deleted 2y ago[deleted]
- swyx 2y ago> Even their "function calling" abstraction still hallucinates parameters and schema huh? sample code please? this should not be true since Structured Outputs came out - literally prevented from generating invalid json (more: https://www.latent.space/p/openai-api-and-o1 https://www.latent.space/p/openai-api-and-o1)
- amitness 2y agoIt's not enabled by default for their function calling API. So, hallucination is possible. You have to set 'strict' to True manually to use the same grammar-based sampling they use for structured outputs. https://platform.openai.com/docs/guides/function-calling?api-mode=responses#strict-mode https://platform.openai.com/docs/guides/function-calling?api...