2 ms·

you could also always specify the logit bias parameter in openai apis

• pshc 3y ago
  That's true, and one can bias logits in llama.cpp and friends too, but those are global biases that affect the entire output rather than being specified per-token. Uploading a grammar or a wasm binary to the inference engine does seem more expressive.