10 ms·
Is there a reason we won’t have LLMs that only speak in JSON? These JSON hacks are clever and cool but feel like they’ll be obsolete in 6 weeks.
by robflaherty 3y ago
Is there a reason we won’t have LLMs that only speak in JSON? These JSON hacks are clever and cool but feel like they’ll be obsolete in 6 weeks.
- verdverm 3y agoA lot of their value is turning unstructured text or user intent into the structured data. Even if the primary input is JSON, you want to explain what to do with it in words, and then get JSON back. Why do you think they will be obsolete in 6 weeks?
- icyfox 3y agoFingers crossed OpenAI / Anthropic / etc do this! Would make working with these APIs for prediction projects that much easier. Technically speaking it's pretty to force the model into an valid JSON-schema if you have access to the inference autoregressive loop and the logit activations. You can either force known areas of the template to a prefixed template, or fill in the basic JSON wrapper and force the model to choose arbitrary keys and values. Imagine it might come down to how much of their usage is on generating text vs generating structured prediction payloads.