4 ms·
They're not a reproduction in any sense. A marginal text token isnt computed from the weights of an LLM in anything like any sense of any activity of any mammal
by mjburgess 20d ago
They're not a reproduction in any sense. A marginal text token isnt computed from the weights of an LLM in anything like any sense of any activity of any mammal.
LLMs are immitation machines: they take impressions of prior text. Today, these include reasoning traces and they include reinforcement so the user-facing completions are correlated with these reasoning traces. The computation here, of "taking an impression" of a data distribution is similar to some impression-taking processes in animals (eg., there's no doubt a similar mechanism in the sensory-motor system acquiring initial impressions of external objects) -- but the computation says nothing about any process of intelligence.
I dont have the time atm to write the needed amount on this to make it clear. But the whole history of life from emergence of valence, bilateral symeterry, to model-free reinforcement and model-based reinforcement, sensory-motor coordination and the imagination -- and so on --- all these give a great amount of detail as to what the capacities of intelligence are which has generated this text for LLMs to copy. And they are nothing like this computation of immitation