4 ms·

>Same as humans Hard disagree. LLMs are very accurately described by the "stochastic parrot" analogy that gets thrown around a lot. They do not "think" like

>Same as humans Hard disagree. LLMs are very accurately described by the "stochastic parrot" analogy that gets thrown around a lot. They do not "think" like humans at all, even if we use the word "think" because it's convenient.

Der_Einzige 2y ago
The more tokens spent on getting a result, the more likely the result is to be accurate. “If it walks like a duck, quacks like a duck” etc
- deleted 2y ago
  [deleted]
- bredren 2y ago
  I suspect gp means thinking in a way that is different from being able to reason. The ability to use any manner of inference or logic to arrive at correct answers does not constitute thought, even if the question was hard. But in general I’m in agreement about the duck. Existing LLMs can already “think” in a non-philosophical sense far better and faster than many adults walking around. Most folks do not Consider the Lobster anyway and would likely not pay much attention to the details of its difference from a brain.
quantadev 2y ago
I think the phrase "stochastic parrot" is misleading and has fooled millions of people into thinking that LLMs can't do genuine reasoning about situations they've never seen before, nor been trained on, which is wrong because LLMs definitely are doing genuine intelligent reasoning. What model training is doing is building up a semantic space of vectors from which astronomically large numbers of true facts and ideas can be derived during inference. I mean like a number of facts larger than the number of molecules in the known universe. A googolplex more facts than the sum of all of humanity has ever "thought".
- bobsomers 2y ago
  > I think the phrase "stochastic parrot" is misleading and has fooled millions of people into thinking that LLMs can't do genuine reasoning about situations they've never seen before, nor been trained on, which is wrong because LLMs definitely are doing genuine intelligent reasoning. Can you describe specifically which part of an LLM architecture does the "reasoning" and how it works? Because every architecture I'm familiar with is literally just a fancy way of predicting the likelihood of the next token given the stream of previous tokens and the known distribution of token based on the training data. This is not reasoning. This is simple statistical prediction, making the "stochastic parrot" analogy actually quite accurate. > What model training is doing is building up a semantic space of vectors from which astronomically large numbers of true facts and ideas can be derived during inference. I mean like a number of facts larger than the number of molecules in the known universe. A googolplex more facts than the sum of all of humanity has ever "thought". Sort of. It's like an extremely lossy compression process which has absolutely no guarantee that "truth" was maintained in the process. Also I'm extremely dubious of your claim that it can accurately encode "a number of facts larger than the molecules in the known universe" given that it's trivially easy to get the best LLMs to give you an incorrect answer to a question any human would easily get right.
  - quantadev 2y ago
    The "reasoning" is an emergent property that no one understands yet. Yes we do all the training only to try to predict the next word (i.e. train to do word prediction), yet with enough training data, then at some scale (GPT 3.5ish) the embedding vectors in semantic space begin to build a geometric scaffolding in into the weights, for lack of a better way to phrase it. If you know about facts like (Vector(man) minus Vector(woman) equals Vector(king) minus Vector(queen)), that's an indication that this "scaffolding" is taking shape. It means the "concept of gender" has a "direction" in the roughly 4,000 dimensional vector "space". This vector behaves geometrically, so that vectors behave in vector space as if it was a geometric space of sorts (there are directions and distances), even though there's no true space coordinates, just logical "directions". Mankind doesn't quite yet understand the "Geometry of Logic". LLMs prove we don't. I think it's a new math field to be invented. As far as the actual number of "facts" contained in an LLM, I think you have to consider something that's a function of the number of bits in an entire model, and ask how many "states" can that store, as a rough approximation from an entropy standpoint. But these aren't pure facts. They're reasoning. I guess you can call reasoning something like "fuzzy facts", so there's a bit of uncertainty to each one of them. LLMs don't store facts, they store fuzzy reasoning. But I call it "factual" when an LLM fixes a bug in my code, or correctly states some piece of knowledge.
MobiusHorizons 2y ago
I tend to agree that LLMs are not thinking in the way that we usually mean it when referring to human thinking. However I think it is dangerous to assert on the capabilities of a system based on the structure seemingly imposed by its API. Take for example the instruction set of a CPU. One could argue that a cpu only has N registers or can process only one instruction at a time because the instruction set only contains N registers, and processes instruction linearly. But on any modern application processor, the register renamer allows many more physical registers to be allocated than can be names in the ISA, and instructions are dispatched in parallel to mitigate memory latency and increase throughput. What I mean is that an LLM is not a stochastic parrot because of its API, but rather because it does not outdo a stochastic parrot when tested. This could change though. LLMs could think in full sentences and spoon feed them to us one token at a time, rewording as necessary to provide a few possibilities. They could memorize the letters in each token and count letters correctly despite the limitations imposed by the API.

Der_Einzige 2y ago

The more tokens spent on getting a result, the more likely the result is to be accurate. “If it walks like a duck, quacks like a duck” etc

deleted 2y ago
[deleted]
bredren 2y ago
I suspect gp means thinking in a way that is different from being able to reason. The ability to use any manner of inference or logic to arrive at correct answers does not constitute thought, even if the question was hard. But in general I’m in agreement about the duck. Existing LLMs can already “think” in a non-philosophical sense far better and faster than many adults walking around. Most folks do not Consider the Lobster anyway and would likely not pay much attention to the details of its difference from a brain.

quantadev 2y ago

I think the phrase "stochastic parrot" is misleading and has fooled millions of people into thinking that LLMs can't do genuine reasoning about situations they've never seen before, nor been trained on, which is wrong because LLMs definitely are doing genuine intelligent reasoning. What model training is doing is building up a semantic space of vectors from which astronomically large numbers of true facts and ideas can be derived during inference. I mean like a number of facts larger than the number of molecules in the known universe. A googolplex more facts than the sum of all of humanity has ever "thought".

bobsomers 2y ago
> I think the phrase "stochastic parrot" is misleading and has fooled millions of people into thinking that LLMs can't do genuine reasoning about situations they've never seen before, nor been trained on, which is wrong because LLMs definitely are doing genuine intelligent reasoning. Can you describe specifically which part of an LLM architecture does the "reasoning" and how it works? Because every architecture I'm familiar with is literally just a fancy way of predicting the likelihood of the next token given the stream of previous tokens and the known distribution of token based on the training data. This is not reasoning. This is simple statistical prediction, making the "stochastic parrot" analogy actually quite accurate. > What model training is doing is building up a semantic space of vectors from which astronomically large numbers of true facts and ideas can be derived during inference. I mean like a number of facts larger than the number of molecules in the known universe. A googolplex more facts than the sum of all of humanity has ever "thought". Sort of. It's like an extremely lossy compression process which has absolutely no guarantee that "truth" was maintained in the process. Also I'm extremely dubious of your claim that it can accurately encode "a number of facts larger than the molecules in the known universe" given that it's trivially easy to get the best LLMs to give you an incorrect answer to a question any human would easily get right.
- quantadev 2y ago
  The "reasoning" is an emergent property that no one understands yet. Yes we do all the training only to try to predict the next word (i.e. train to do word prediction), yet with enough training data, then at some scale (GPT 3.5ish) the embedding vectors in semantic space begin to build a geometric scaffolding in into the weights, for lack of a better way to phrase it. If you know about facts like (Vector(man) minus Vector(woman) equals Vector(king) minus Vector(queen)), that's an indication that this "scaffolding" is taking shape. It means the "concept of gender" has a "direction" in the roughly 4,000 dimensional vector "space". This vector behaves geometrically, so that vectors behave in vector space as if it was a geometric space of sorts (there are directions and distances), even though there's no true space coordinates, just logical "directions". Mankind doesn't quite yet understand the "Geometry of Logic". LLMs prove we don't. I think it's a new math field to be invented. As far as the actual number of "facts" contained in an LLM, I think you have to consider something that's a function of the number of bits in an entire model, and ask how many "states" can that store, as a rough approximation from an entropy standpoint. But these aren't pure facts. They're reasoning. I guess you can call reasoning something like "fuzzy facts", so there's a bit of uncertainty to each one of them. LLMs don't store facts, they store fuzzy reasoning. But I call it "factual" when an LLM fixes a bug in my code, or correctly states some piece of knowledge.

MobiusHorizons 2y ago

I tend to agree that LLMs are not thinking in the way that we usually mean it when referring to human thinking. However I think it is dangerous to assert on the capabilities of a system based on the structure seemingly imposed by its API. Take for example the instruction set of a CPU. One could argue that a cpu only has N registers or can process only one instruction at a time because the instruction set only contains N registers, and processes instruction linearly. But on any modern application processor, the register renamer allows many more physical registers to be allocated than can be names in the ISA, and instructions are dispatched in parallel to mitigate memory latency and increase throughput. What I mean is that an LLM is not a stochastic parrot because of its API, but rather because it does not outdo a stochastic parrot when tested. This could change though. LLMs could think in full sentences and spoon feed them to us one token at a time, rewording as necessary to provide a few possibilities. They could memorize the letters in each token and count letters correctly despite the limitations imposed by the API.