5 ms·
I get decent JSON from it quite well with the "assistant: {" trick. I'm not sure how well trained it is to do JSON. The template on ollama has tools calls so
by the_mitsuhiko 2y ago
I get decent JSON from it quite well with the "assistant: {" trick. I'm not sure how well trained it is to do JSON. The template on ollama has tools calls so I assume they made sure JSON works: https://ollama.com/library/mistral-small:24b/blobs/6db27cd4e277 https://ollama.com/library/mistral-small:24b/blobs/6db27cd4e...
- a_wild_dandan 2y agoAnd for anyone looking to dig deeper, check out "grammar-based sampling."
- azinman2 2y agoWhat’s the “assistant: {" trick? You just end your prompt with that?
- simonw 2y agoMistral supports prefixes: https://docs.mistral.ai/guides/prefix/ https://docs.mistral.ai/guides/prefix/
- azinman2 2y agoThat’s cool. However it only shows a few odd. I’d imagine the model needs to explicitly support this (be trained with it). None are about json… do you use that trick with json?
- simonw 2y agoI use it to get JSON pretty often. See also: https://twitter.com/simonw/status/1885091289554968975 https://twitter.com/simonw/status/1885091289554968975