4 ms·

I view this as compelling evidence that current models are more than "stochastic parrots," because as the OP shows, they are learning to model the world in ways

by cs702 2y ago

I view this as compelling evidence that current models are more than "stochastic parrots," because as the OP shows, they are learning to model the world in ways that are similar (up to a linear transformation) to those exhibited by the human brain. The OP's findings, in short: * A linear transformation of a speech encoder's embeddings closely aligns them with patterns of neural activity in the brain's speech areas in response to the same speech sample. * A linear transformation of a language decoder's embeddings closely aligns them with patterns of neural activity in the brain's language areas in response to the same language sample.

hacker_homie 2y ago
I view this as the language processing is similar but we’re not talking about thinking, just processing language. I see this as maybe it’s not a statistical parrot, but it’s still only some kind of parrot, Maybe a sleep deprived one.
- deleted 2y ago
  [deleted]
  - deleted 2y ago
    [deleted]
  - Terr_ 2y ago
    > the vast number of internal models of the world that are generated strictly from text input Do you mean in the sense than an LLM can emit documents following grammar rules or pirate-talk? For humans, text doesn't generate our models so much as as it triggers them.
- famouswaffles 2y ago
  There isn't really any clear delineation between 'thinking' and 'language processing' in the brain.
  - aithrowawaycomm 2y ago
    That's 100% false, dogs and pigeons can obviously think, and it is childish to suppose that their thoughts are a sequence of woofs or coos. Trying to make an AI that thinks like a human without being able to think like a chimpanzee gives you reasoning LLMs that can spit out proofs in algebraic topology, yet still struggle with out-of-distribution counting problems which frogs and fish can solve.
  - wruza 2y ago
    How human brains process thoughts is non-uniform across population. There’s imagery, written language, sound, speech, tactile, etc. Not everything that you think about is readily expressible in your language. There are definitely people with and without “internal screen”, and probably few more types with/without X, where X is a set of things we’ve never talked about, either assuming everyone has it or not realizing that it’s a non-mandatory part of how you think.
    - famouswaffles 2y ago
      That's not really what I'm saying. What I'm saying is how does the brain look when you do both ? Is there a clear difference ? There's no 'thinking mode' followed by a 'language processing mode'. Language processing is thinking as far the brain is concerned and there's no evidence that these are 2 cleanly separated processes whether you 'think' in words or not.
      - foobarqux 2y ago
        That's not true, the Moro experiments show they use different capacities as do similar experiments on people who have certain severe cognitive deficiencies that don't impact language processing (e.g. the subject "Chris")
        famouswaffles 2y ago
        My argument is that “thinking” and “language processing” are not two sequential or clearly separated modes in the brain but deeply intertwined. Language is a lot more than parsing syntax, whatever your thoughts are on the matter, even LLMs are clearly doing more than that. Are there any experiments where subjects had severe cognitive deficiencies and language in its full breadth or maybe i should say communication ? came out unscathed ? The chris experiments don't seem to go into much detail in that front.
hakaneskici 2y ago
> current models are more than "stochastic parrots" I believe the same, and also I'm willing to accept that the human brain can intentionally operate in an stochastic parrot mode. Some people have the ability to fluently speak non-stop, completely impromptu. I wonder if it's similar to an LLM pipeline, where there'a constant stream of thoughts being generated based on very recent context, which are then passed through various output filters.
- iamleppert 2y ago
  That certainly would explain how Trump can rattle on for hours and hours, despite a notable lack of high quality training data.
foobarqux 2y ago
The correlations are 0.25-0.5, which is quite poor (Gaussian distribution plots with those correlations look like noise). That's before analyzing the methodology and assumptions.
- ogogmad 2y ago
  Isn't correlation a wrong way of measuring statistical dependence anyway? I thought that entropy-based method like Relative Entropy did this better.
  - foobarqux 2y ago
    Yeah, Taleb has some pretty convincing arguments that correlation is generally bad/misleading and entropy is better.
- bravura 2y ago
  Correlation of 0.25-0.5 being poor is very problem dependent. For example, in difficult perceptual tasks ("can you taste which of these three biscuits is different" [one biscuit is made with slightly less sugar]), a correlation of 0.3 is commonplace and considered an appropriate amount of annotator agreement to make decisions.
  - foobarqux 2y ago
    Yes for certain things like statistical trading (assuming some kind of "nice" Gaussian-like distribution) where you have lots of trades and just need to be more right than wrong it's probably useful. Not here though, where you are trying to prove a (near) equivalence.
devmor 2y ago
I do not understand what you find convincing about this that changes your mind. We have a closed system that we designed to operate in a way that is similar to our limited understanding of how a portion of the brain works, based on how we would model that part of the brain if it had to traverse an nth-dimensional array. We have loosely observed it working in a way that could roughly be defined as similar to our limited understanding of how a portion of the brain works given that limitation that we know is not true of the human brain, with a fairly low confidence level. Even if you put an extreme level of faith into those very subpar conclusions and take them to be rigid... That does not make it actually similar to the human brain, or any kind of brain at all.
robmccoll 2y ago
I view this as further evidence that we less different from "stochastic parrots" than we would like to believe.
- corysama 2y ago
  <user corysama squawks indignantly and flies away...>
flir 2y ago
Galaxies, hurricanes, whirlpools. Black Spleenwort, Barnsley Fern. Power laws every bloody where. (My utterly uninformed knee-jerk reaction here, but even if I was a true believer I don't think I'd reach for "compelling".)
- whymeogod 2y ago
  natural consequence when results are the sum of random processes. if you add random rolls, you get a gaussian, thanks central limit theorem. if you sum them, you get a lognormal distribution, which approximates a power law up to a cutoff
hatthew 2y ago
Yeah, I have always firmly maintained that there is less fundamental difference between LLMs and human brains than most people seems to assume. Going a bit further, I'll speculate that the actions made by a human brain are simply a function of the "input" from our ~5 senses combined with our memory (obviously there are complications such as spinal reflexes, but I don't think those affect my main point). Neural nets are universal function approximators, so can't a sufficiently large neural net approximate a full human brain? In that case, is there any merit to saying that a human "understands" something in a way that a neural net doesn't? There's obviously a huge gap between the two right now, but I don't see any fundamental difference besides "consciousness" which is not well defined to begin with.
- AlotOfReading 2y ago
  The UAT is a pretty weak result in practice. A lot of systems have the same property, and most of them are pretty poor approximators in practice. It may very well be that no reasonable amount of computing power allows approximating the "function of consciousness". Plus, if you're a certain kind of dualist the entire idea of a compact, smooth "consciousness" function may be something you reject philosophically.
  - hatthew 2y ago
    I agree there are issues with the UAT, but I feel like my conclusion is still valid: a neural net, given the memories and senses that a humans has, is capable of approximating a human's response accurately enough to be indistinguishable from another human, at least to another human. I philosophically reject the notion that consciousness is an important factor here. The question of whether or not you have a consciousness doesn't affect what I take away from this conversation, and similarly the question of whether an AI has a consciousness doesn't affect what I take away from my actions with it. If the (non-)existence of others' consciousnesses doesn't materially affect my life—and we assume that it's a fundamentally unanswerable question—why should I care other than curiosity?
    - suddenlybananas 2y ago
visarga 2y ago
> I view this as compelling evidence that current models are more than "stochastic parrots," More evidence against "stochastic parrots" - zero shot translation, where LLMs can translate between unseen pairs of languages - repeated sampling of responses from the same prompt - which shows diversity of expression with convergence of semantics - reasoning models - solving problems But my main critique is that they are better seen as pianos, not parrots. Pianos don't make music, but we do. And we play the LLMs on the keyboard like regular pianos.

by cs702 2y ago

hacker_homie 2y ago

I view this as the language processing is similar but we’re not talking about thinking, just processing language. I see this as maybe it’s not a statistical parrot, but it’s still only some kind of parrot, Maybe a sleep deprived one.

deleted 2y ago
[deleted]
- deleted 2y ago
  [deleted]
- Terr_ 2y ago
  > the vast number of internal models of the world that are generated strictly from text input Do you mean in the sense than an LLM can emit documents following grammar rules or pirate-talk? For humans, text doesn't generate our models so much as as it triggers them.
famouswaffles 2y ago
There isn't really any clear delineation between 'thinking' and 'language processing' in the brain.
- aithrowawaycomm 2y ago
  That's 100% false, dogs and pigeons can obviously think, and it is childish to suppose that their thoughts are a sequence of woofs or coos. Trying to make an AI that thinks like a human without being able to think like a chimpanzee gives you reasoning LLMs that can spit out proofs in algebraic topology, yet still struggle with out-of-distribution counting problems which frogs and fish can solve.
- wruza 2y ago
  How human brains process thoughts is non-uniform across population. There’s imagery, written language, sound, speech, tactile, etc. Not everything that you think about is readily expressible in your language. There are definitely people with and without “internal screen”, and probably few more types with/without X, where X is a set of things we’ve never talked about, either assuming everyone has it or not realizing that it’s a non-mandatory part of how you think.
  - famouswaffles 2y ago
    That's not really what I'm saying. What I'm saying is how does the brain look when you do both ? Is there a clear difference ? There's no 'thinking mode' followed by a 'language processing mode'. Language processing is thinking as far the brain is concerned and there's no evidence that these are 2 cleanly separated processes whether you 'think' in words or not.
    - foobarqux 2y ago
      That's not true, the Moro experiments show they use different capacities as do similar experiments on people who have certain severe cognitive deficiencies that don't impact language processing (e.g. the subject "Chris")
      - famouswaffles 2y ago
        My argument is that “thinking” and “language processing” are not two sequential or clearly separated modes in the brain but deeply intertwined. Language is a lot more than parsing syntax, whatever your thoughts are on the matter, even LLMs are clearly doing more than that. Are there any experiments where subjects had severe cognitive deficiencies and language in its full breadth or maybe i should say communication ? came out unscathed ? The chris experiments don't seem to go into much detail in that front.

hakaneskici 2y ago

> current models are more than "stochastic parrots" I believe the same, and also I'm willing to accept that the human brain can intentionally operate in an stochastic parrot mode. Some people have the ability to fluently speak non-stop, completely impromptu. I wonder if it's similar to an LLM pipeline, where there'a constant stream of thoughts being generated based on very recent context, which are then passed through various output filters.

iamleppert 2y ago
That certainly would explain how Trump can rattle on for hours and hours, despite a notable lack of high quality training data.

foobarqux 2y ago

The correlations are 0.25-0.5, which is quite poor (Gaussian distribution plots with those correlations look like noise). That's before analyzing the methodology and assumptions.

ogogmad 2y ago
Isn't correlation a wrong way of measuring statistical dependence anyway? I thought that entropy-based method like Relative Entropy did this better.
- foobarqux 2y ago
  Yeah, Taleb has some pretty convincing arguments that correlation is generally bad/misleading and entropy is better.
bravura 2y ago
Correlation of 0.25-0.5 being poor is very problem dependent. For example, in difficult perceptual tasks ("can you taste which of these three biscuits is different" [one biscuit is made with slightly less sugar]), a correlation of 0.3 is commonplace and considered an appropriate amount of annotator agreement to make decisions.
- foobarqux 2y ago
  Yes for certain things like statistical trading (assuming some kind of "nice" Gaussian-like distribution) where you have lots of trades and just need to be more right than wrong it's probably useful. Not here though, where you are trying to prove a (near) equivalence.

devmor 2y ago

I do not understand what you find convincing about this that changes your mind. We have a closed system that we designed to operate in a way that is similar to our limited understanding of how a portion of the brain works, based on how we would model that part of the brain if it had to traverse an nth-dimensional array. We have loosely observed it working in a way that could roughly be defined as similar to our limited understanding of how a portion of the brain works given that limitation that we know is not true of the human brain, with a fairly low confidence level. Even if you put an extreme level of faith into those very subpar conclusions and take them to be rigid... That does not make it actually similar to the human brain, or any kind of brain at all.

robmccoll 2y ago

I view this as further evidence that we less different from "stochastic parrots" than we would like to believe.

corysama 2y ago
<user corysama squawks indignantly and flies away...>

flir 2y ago

Galaxies, hurricanes, whirlpools. Black Spleenwort, Barnsley Fern. Power laws every bloody where. (My utterly uninformed knee-jerk reaction here, but even if I was a true believer I don't think I'd reach for "compelling".)

whymeogod 2y ago
natural consequence when results are the sum of random processes. if you add random rolls, you get a gaussian, thanks central limit theorem. if you sum them, you get a lognormal distribution, which approximates a power law up to a cutoff

hatthew 2y ago

Yeah, I have always firmly maintained that there is less fundamental difference between LLMs and human brains than most people seems to assume. Going a bit further, I'll speculate that the actions made by a human brain are simply a function of the "input" from our ~5 senses combined with our memory (obviously there are complications such as spinal reflexes, but I don't think those affect my main point). Neural nets are universal function approximators, so can't a sufficiently large neural net approximate a full human brain? In that case, is there any merit to saying that a human "understands" something in a way that a neural net doesn't? There's obviously a huge gap between the two right now, but I don't see any fundamental difference besides "consciousness" which is not well defined to begin with.

AlotOfReading 2y ago
The UAT is a pretty weak result in practice. A lot of systems have the same property, and most of them are pretty poor approximators in practice. It may very well be that no reasonable amount of computing power allows approximating the "function of consciousness". Plus, if you're a certain kind of dualist the entire idea of a compact, smooth "consciousness" function may be something you reject philosophically.
- hatthew 2y ago
  I agree there are issues with the UAT, but I feel like my conclusion is still valid: a neural net, given the memories and senses that a humans has, is capable of approximating a human's response accurately enough to be indistinguishable from another human, at least to another human. I philosophically reject the notion that consciousness is an important factor here. The question of whether or not you have a consciousness doesn't affect what I take away from this conversation, and similarly the question of whether an AI has a consciousness doesn't affect what I take away from my actions with it. If the (non-)existence of others' consciousnesses doesn't materially affect my life—and we assume that it's a fundamentally unanswerable question—why should I care other than curiosity?
  - suddenlybananas 2y ago

visarga 2y ago

> I view this as compelling evidence that current models are more than "stochastic parrots," More evidence against "stochastic parrots" - zero shot translation, where LLMs can translate between unseen pairs of languages - repeated sampling of responses from the same prompt - which shows diversity of expression with convergence of semantics - reasoning models - solving problems But my main critique is that they are better seen as pianos, not parrots. Pianos don't make music, but we do. And we play the LLMs on the keyboard like regular pianos.