7 ms·
When the attempt is though to have the LLM output an "idea", not just a "next token", the selection over the logits vector should break that original idea... If
by mdp2021 2y ago
When the attempt is though to have the LLM output an "idea", not just a "next token", the selection over the logits vector should break that original idea... If the idea is complete, there should be no need to use sampling over the logits.
The sampling, in this framework, should not happen near the output level ("what will the next spoke word be").
- minimaxir 2y agoLLMs are trained to maximize the probability of correct guesses for the next token, not "ideas". You cannot define an idea as a training loss objective.
- mdp2021 2y agoThat is an architectural problem. If you want the post to be rephrased: it is paradoxical to have changes made near the output level, "changing words before it says them", given that the expected is to work with ideas. (And even then, selection would not be at the output level - it would be during the definition of the structure.) So, articles like this submission - while interesting from many points of view - make the elephant in the room more evident. > You cannot define an idea as a training loss objective What tells you so? If you see a technical limit, note e.g. that sentences and paragraphs can have their own position in an embedding space.
- orbital-decay 2y agoInterpretability studies offer several orthogonal ways to look at this, it's like Newtonian vs Lagrangian mechanics. Autoregressive token prediction, pattern matching, idea conceptualization, pathfinding in the extremely multidimensional space...