2 ms·
> It's reasonable to call minimising the training loss function the "objective" of the LLM This article isn't doing that, though. What it is calling the model
by chrisjj 3y ago
> It's reasonable to call minimising the training loss function the "objective" of the LLM
This article isn't doing that, though. What it is calling the model objective is "producing clear and coherent text".
> Whether "producing clear and coherent text" is a good characterisation of a "predict the next token" loss function is another question though.
I agree. But lets us be clear that prediction too is a misnomer. Simply generating a word coherent with prior words is not prediction.