Because the model is also encoding and generating something the user will not usually see. Demarcation tokens such as <|endofsentence|>, <|im_start|> etc. The

Because the model is also encoding and generating something the user will not usually see. Demarcation tokens such as <|endofsentence|>, <|im_start|> etc. The probability of generating the end of sentence token which tells the model when to stop increases in probability as it responds to you. If it's just a completion engine, it will generate silly things like a continuation of your input message. For all we know you're not asking an interactive question but requiring the completion of a list of questions. Who knows? As far as the model is concerned if it's not trained to reply it will just be a dumb completion engine. When we do other training like Question, Answer pairs - the model learns that questions usually have an answer and the answer usually has a high probability of outputting the end of sentence token when the full answer has been written. This is of course dependant on the fact that I'm allowed to generate that many tokens, I might get cut off prematurely if the developer set my token limit too low. I might also miss the mark and continue on if the other parameters made my outputs a bit too noisy but that's ok as either I'll get cut off or I'll reach a point before the token limit where I've said enough of what I wanted to say.<|endofsentence|>