You won't get it built in but there are plenty of biased sampling projects out there that can enforce responses as JSON. It's actually going to be more reliable

You won't get it built in but there are plenty of biased sampling projects out there that can enforce responses as JSON. It's actually going to be more reliable with a local model since you can mess with logits before sampling. [1] https://github.com/ggerganov/llama.cpp#constrained-output-with-grammars https://github.com/ggerganov/llama.cpp#constrained-output-wi... [2] https://github.com/gsuuon/ad-llama https://github.com/gsuuon/ad-llama (mine)