3 ms·

>> LLMs just predict tokens. Out of curiosity, how do you think human intelligence works? I am no expert and I don't claim to know how the human brain works bu

>> LLMs just predict tokens. Out of curiosity, how do you think human intelligence works? I am no expert and I don't claim to know how the human brain works but my layman's mental model of thinking would be basically that: biological neural networks that "do" the thinking are token prediction machines, except "tokens" are not words that appear on the screen but thoughts that appear in consciousness... while the underlying machinery is in both cases a network of units that fire (or not) based on the connections between them. Sure, human experience is a lot more than just intelligence (I have no mental model of how consciousness or qualia might work) but it is surprising to me that so many people keep repeating arguments similar to yours - that LLMs do not really think, they are "just" doing [X] (where X usually describes how I imagine human intelligence works) - if there is more to human intelligence, what do you think it is?

spicyusername 3y ago
if there is more to human intelligence, what do you think it is Thousands of dedicated scientists have spent their entire careers trying to answer this question and we still don't have something remotely approximating an answer. If the answer to "what is consciousness" was as simple as some groups of linear algebra equations, I think someone would have figure that out by now. A gut feeling that one thing is kind of analogous to another is just that. A carpenter and a woodpecker both bang on wood, but knowing anything at all about one doesn't really tell you anything about the other. I suspect laypeople would be less likely to try to make the connection between LLM graphs and physical neurons, if they wouldn't have called those groups of equations "neural networks". They share the name, but not much else.
Mawr 3y ago
How it works is well beyond my ability to answer, but I can attempt a guess on what the result looks like. Humans build complex mental models. As far as I can tell, LLMs don't. An LLM playing chess is only able to do so because it essentially encoded what to do for any given board state by having seen a lot of chess games. That's why it's fairly competent at the beginning of a game when the number of possible states is low, but falls apart in later stages where memorization is useless. That's why LLMs make illegal chess moves, generate code that doesn't parse and hallucinate. They have no mental models, no understanding. A human playing chess would instead slowly build up a mental model of the game as they played and play off that model. ... maybe, I dunno.
- honzabe 3y ago
  I think that a lot depends on how you define concepts like "understanding" and "mental model". Imagine that I am building a translation app: one way to do it would be simply a dictionary database where the Czech word "pták" is related to the English word "bird"- we certainly agree that this app does not understand what a bird is at all. But when I say I do understand what a bird is, what does that mean? The way I imagine it, my brain is wired in such a way that when the word "bird" occurs, a certain pattern of neurons that encode stuff like "living creature", "flying", "feathers", "eggs", and "beak" also fire (each of those is itself a pattern of other associations). In this sense, I think LLMs can "understand": connections between words "bird" and "eggs" or "flying" have weights closer to 1 while connections to words like "iron" or "multiplication" have weights closer to 0. A mental model of a bird can be represented as a set of vectors in multi-dimensional space. Maybe I am missing something but I see no fundamental reason why we would not be able to encode complex mental models like chess or liberalism or flat earth theory as sets of vectors (and maybe even determine the set of operations that can move someone from flat earth model to globe model in a similar manner as those examples with 'King – Man + Woman = Queen'). Again, I am not saying I know how the human brain works. All I am saying is that I think we can plausibly explain a lot of it in terms of just networks of units that have some weights attached to them - it is a very powerful concept. That is why I am asking when people seem to be assuming that there is more to "legitimate intelligence". Also - and I say this with a high level of insecurity because I am no expert - I do not think that LLMs actually play chess the way you describe it. If I even understand correctly what you mean - are you saying that LLMs are only able to play moves they have "seen" in their training sets? If that is the case, I would disagree - I think there is evidence that LLMs have some ability to generalize, which means the ability to work with abstract patterns (which I would be tempted to call mental models).

spicyusername 3y ago

if there is more to human intelligence, what do you think it is Thousands of dedicated scientists have spent their entire careers trying to answer this question and we still don't have something remotely approximating an answer. If the answer to "what is consciousness" was as simple as some groups of linear algebra equations, I think someone would have figure that out by now. A gut feeling that one thing is kind of analogous to another is just that. A carpenter and a woodpecker both bang on wood, but knowing anything at all about one doesn't really tell you anything about the other. I suspect laypeople would be less likely to try to make the connection between LLM graphs and physical neurons, if they wouldn't have called those groups of equations "neural networks". They share the name, but not much else.

Mawr 3y ago

How it works is well beyond my ability to answer, but I can attempt a guess on what the result looks like. Humans build complex mental models. As far as I can tell, LLMs don't. An LLM playing chess is only able to do so because it essentially encoded what to do for any given board state by having seen a lot of chess games. That's why it's fairly competent at the beginning of a game when the number of possible states is low, but falls apart in later stages where memorization is useless. That's why LLMs make illegal chess moves, generate code that doesn't parse and hallucinate. They have no mental models, no understanding. A human playing chess would instead slowly build up a mental model of the game as they played and play off that model. ... maybe, I dunno.

honzabe 3y ago
I think that a lot depends on how you define concepts like "understanding" and "mental model". Imagine that I am building a translation app: one way to do it would be simply a dictionary database where the Czech word "pták" is related to the English word "bird"- we certainly agree that this app does not understand what a bird is at all. But when I say I do understand what a bird is, what does that mean? The way I imagine it, my brain is wired in such a way that when the word "bird" occurs, a certain pattern of neurons that encode stuff like "living creature", "flying", "feathers", "eggs", and "beak" also fire (each of those is itself a pattern of other associations). In this sense, I think LLMs can "understand": connections between words "bird" and "eggs" or "flying" have weights closer to 1 while connections to words like "iron" or "multiplication" have weights closer to 0. A mental model of a bird can be represented as a set of vectors in multi-dimensional space. Maybe I am missing something but I see no fundamental reason why we would not be able to encode complex mental models like chess or liberalism or flat earth theory as sets of vectors (and maybe even determine the set of operations that can move someone from flat earth model to globe model in a similar manner as those examples with 'King – Man + Woman = Queen'). Again, I am not saying I know how the human brain works. All I am saying is that I think we can plausibly explain a lot of it in terms of just networks of units that have some weights attached to them - it is a very powerful concept. That is why I am asking when people seem to be assuming that there is more to "legitimate intelligence". Also - and I say this with a high level of insecurity because I am no expert - I do not think that LLMs actually play chess the way you describe it. If I even understand correctly what you mean - are you saying that LLMs are only able to play moves they have "seen" in their training sets? If that is the case, I would disagree - I think there is evidence that LLMs have some ability to generalize, which means the ability to work with abstract patterns (which I would be tempted to call mental models).