10 ms·
Shocked this wasn't already a feature. Bummed they only seem to have JSON Schema and not something more flexible like BNF grammar's, like llama.cpp has for a lo
by causal 11mo ago
Shocked this wasn't already a feature. Bummed they only seem to have JSON Schema and not something more flexible like BNF grammar's, like llama.cpp has for a long time: https://github.com/ggml-org/llama.cpp/blob/master/grammars/README.md https://github.com/ggml-org/llama.cpp/blob/master/grammars/R...
- cma 11mo agoIf they every gave really finegrained constraints you could constrain to subsets of tokens and extract the logits a lot cheaper than by random sampling limited to a few top choices and distill claude at a much deeper level. I wonder if that plays into some of the restrictions.
- causal 11mo agoThat makes sense, and if that's the reason it's another vote for open models