> The loss function is the feedback Please expand -- as I understand it the LLMs use cross entropy on next token prediction as their loss function and I fail t

> The loss function is the feedback Please expand -- as I understand it the LLMs use cross entropy on next token prediction as their loss function and I fail to see how this gives any feedback about translation, even given parallel texts. Predicting the next token in a French language text and predicting the next token in an English language text are not obviously more interesting than dealing with non-parallel texts in both languages. It's totally possible that these parallel texts are critical in a sense to the translation capabilities but this is not obvious. It would be incredibly difficult to establish empirically because reducing training data reduces the effectiveness at all tasks. Translation, like almost all of the capabilities of LLMs, is an emergent behavior that we don't fully understand yet.