It's not 100% yet. Route to that: 1. Clientside, retry strategy on failed parse. Not yet implemented, we throw an exception on parse fail right now, but soon to

It's not 100% yet. Route to that: 1. Clientside, retry strategy on failed parse. Not yet implemented, we throw an exception on parse fail right now, but soon to be implemented. Not ideal because of token burn and latency, but the best quick solution. 2. For the custom model we're building, we use strict grammar definitions to bias outputs toward the needed structure (or if there is only one structurally correct token, outright skip the generation of that token and insert it directly). I've been impressed at how well gpt-4 does with the default prompt template we use. Even better if you enable the chain_of_thought flavor.