Yes the inference loop provides a list of “logits” every loop which are possible predictions for the next token. LLM parameters like temperature and top_p confi

Yes the inference loop provides a list of “logits” every loop which are possible predictions for the next token. LLM parameters like temperature and top_p configure the default logic that selects one of those logits but that logic can be replaced with anything you want. It just has to select the next token and feed it back into inference. Llama.cpp and its downstream users like ollama have long supported using BNF grammars to constrain the output that way.