7 ms·
> LLMs predict words. Any semantic validity is a side effect of enough training data reinforcing the close correlation of those tokens. The mechanisms which ar
by baq 3y ago
> LLMs predict words. Any semantic validity is a side effect of enough training data reinforcing the close correlation of those tokens.
The mechanisms which are built during training in the big blob of bits we call weights are anything but transparent. How they predict the next word is the big thing here. Saying they ‘just’ predict the next word is ignoring basically everything that actually matters.
- famouswaffles 3y agoExactly. Feed a sequence of proteins to a transformer and biological structure and function will emerge in the inner layers. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 https://www.pnas.org/doi/full/10.1073/pnas.2016239118 Why? Because it needs to learn that to make correct predictions. All it takes to incentive a transformer to learn something is data that would require learning it to predict. It's fairly obvious (and not just because of this) that LLMs model emotion somehow. They need to.
- Last5Digits 3y agoThis is what makes these discussions so infuriating. Saying that LLMs "just predict the next word" is about as insightful as saying that computers "just do a bunch of logical operations" - neither point constraints the possible capabilities of the systems they refer to in any meaningful way.
- mjburgess 3y agoSure it does. It perfectly deliniates it. LLMs are not: sensitive to causal structure, dynamically adapting to environmental changes, growing, developing sensory-motor capacities, they are not with us in our environment, they are not: expressing desires, preferences, intentions, beliefs, motivations, etc. And so on. To say, "they just predict the next word" is literally to say that all apparent functions of an LLM are engineering tricks, circumstantially useful -- to be found by (largely software) engineers in building apps. The reason any reply is given to any prompt is that this reply is maximally probabilistically consistent with a historical corpus of text. This excludes the possibility the reply is say, an expression of a history of aesthetic experiences which form an individual's taste. Or, likewise, anything. This is a scientific claim about what LLMs are, not an engineering claim about what over-hyped apps might be abled to do with them.
- JohnAaronNelson 3y agoWhat if humans’ responses are merely probabilistically consistent with a history of sensory experiences? Would this change the significance of human emotions vs apparent emergent emotional responses from LLMs?
- mjburgess 3y agoemotions regulate motivation, desire, action, behaviour etc. to be angry is for your sensory-motor system to be primed for aggression; it's for your cognitive systems to be narrowed and focused on analysing high-threat parts of your environment; it is for your memory-formulation to be modulated towards threat recollection etc. Sure, if an LLM's prompt "be angry" causes it to adopt a threat stance to its environment, to regulate it's theory-of-mind to engage with possible hostile entities, and so on --- then yes, when LLMs are there, I shall concede the point However, how terrible it would be to start with an analysis of emotions in terms of the capacities of LLMs -- right? Since if you did that you'd basically be hobbling your own ability to give an accurate account of emotions (etc.). And no doubt, far worse, end up thinking of yourself as a far narrower, less complex, less interesting, dumber thing than you really are. Indeed, I wonder if we might consider there being something kinda intellectually offensive in this supposition. Here's my silly trinket, now, everything is just like that! End all science, we're done boys -- it's just P(Y|X)
- kelseyfrog 3y agoWhy should I discount a theory just to protect my ego? We've read countless stories about science only progressing when those with big egos die. It would only seem logical that eventually it will come for my own.
- mjburgess 3y agoYou're protecting your ego more when you compare people to LLMs, since that is your existing prejudice. The sort of fashionable pseudo-scientific scientism in the belief that animals are alike digital electrical machines is a kind of egoism. It says, "the engineer of these machines (me!) knows all!" The real hit to your ego is to suppose you are vastly more complex than you understand -- and this is why these engineers crop up and demand that there is nothing more to know than what they have already learned this is an illusion of humility: these engineers take the implied nihilism of this view (that of the emptyness of animal life) as evidence that it is the humble one. But, as ever nihilism, ends up being the most profound kind of arrogance, here its an ego-defence against the threat of their own ignorance. And the threat is real: all you have ever learned about how to sequence transformations of natural numbers (all the algorithms of computer science) are of no use at all in the study of intelligence. What an injury to the ego!
- intended 3y agoIf you look at the comment, it’s not just “LLMs predict the next token.” It is that people have forgotten that it’s just “predict the next token.” Right now it’s like people saw a 486 processor and started thinking it was a brain.
- Last5Digits 3y agoYour comment literally reads: "LLMs predict words. Any semantic validity is a side effect of enough training data reinforcing the close correlation of those tokens." How am I supposed to interpret this any other way? If your claim is that LLMs currently do not possess the same generalization ability as humans, then no one here would disagree with you. But you went way further by claiming that only close correlations were being considered and that semantic validity was simply accidental. Semantic validity is the norm for GPT3/4, finding failures of generalization three or four steps of inference removed from its training domain is not sufficient to make a grand claim like yours. In fact, you wrote multiple comments with claims like that LLMs are "super advanced lorem ipsum." and "Word predictors not world state predictors". Each of these claims has been dis-proven multiple times, unless you want to set the bar for world modeling at perfect generalization over all domains of computation. A bar that no system, including humans, would pass. To stay with the computer analogy: Imagine that same 486 processor not being able to solve a very complex SAT problem before the end of the universe and then denying the Turing-completeness of said processor based on that failure. (In conjunction with memory)
- intended 3y agoI think LLMs are a good start. I am certain they lack a world model, the kind you and me use. This is, to me, a fact. I think that eventually we will bridge these gaps. I also work on implementations that smash into the limits I am describing. I am not the only one. I have scrupulously avoided calling it hallucinations, but these are the litmus test where the claims fail. The failures are not a case of not knowing specific nouns, they are a generalization failure that a world model would prevent. I have linked a paper in my comments that shows emergent properties are an issue of metrics, and that model capability increases are linear. If your model decides that a rose by any other name doesn’t smell just as sweet, then your model is fundamentally not seeing roses. That is the gap you see in production settings. The model sees tokens we see “hallucinations”. I dont see that this takes away from what LLMs achieve, it takes away from claims being made that are not validated by empirics. Look, you can argue with me or you can try it out. Push the system, see how far it can go.