3 ms·

> Since they are trained on information, the sequences chosen often seem informational, even though they are just patterns. The point is is that dismissing it

> Since they are trained on information, the sequences chosen often seem informational, even though they are just patterns. The point is is that dismissing it because it's "just patterns" ignores the fact that nobody has proven that human cognition isn't also "just patterns". If it is, then that undermines the whole justification for the dismissal. So the dismissal assumes the conclusion until that evidence is presented.

slowmovintarget 4y ago
All the arguments for LLMs actually having "understanding" are of the form "I want to believe" or "you can't prove they're not." Chomsky's point that we have no basis for judging LLMs as cognition machines because we don't know what cognition is, is correct. Building these machines may produce useful tools, but it also doesn't advance studies of cognition, either. We do know one characteristic of cognition as it occurs in nature. There seem to be natural affinities to distinguish the real and true from the hallucination. Perhaps that comes from persistence and physicality. How do we give that to ChatGPT?
- naasking 4y ago
  > Chomsky's point that we have no basis for judging LLMs as cognition machines because we don't know what cognition is, is correct. That's not Chomsky's point at all. He's saying they are emphatically not understanding or intelligent in any way, and that's not justified. > Building these machines may produce useful tools, but it also doesn't advance studies of cognition, either. I couldn't disagree more. We have here apparently intelligent systems completely unlike naturally occurring ones. IMO, you'd have to be crazy to think that comparing and contrasting the properties of these systems won't provide any valuable data about the nature of intelligence, perception and cognition as a whole. In fact, we're already seeing papers discussing how LLMs basically refute Chomsky's whole model of language [1]. [1] https://lingbuzz.net/lingbuzz/007180 https://lingbuzz.net/lingbuzz/007180
  - naasking 4y ago
    To clarify, by "completely unlike naturally occurring ones", I mean their origin and training set is completely unlike them, not that their operation is mechanistically different. We don't really know that yet.
  - calf 4y ago
    I read the first 9 pages, it claims to refute it but I don't agree with the premise. The author admits the status quo is like when doctors discovered lemons cured scurvy, but nobody knew why. So Noam's position is basically that doctors will never discover Vitamin C unless they develop chemistry. Both positions are true, they're just talking past each other.
    - naasking 4y ago
      Then you missed the actual refutation, which starts on page 14, "The refutation of key principles". LLMs undermine many of the principles that Chomsky's position depends on. > So Noam's position is basically that doctors will never discover Vitamin C unless they develop chemistry. I disagree with this analogy. LLMs and machine learning is playing with chemistry, and Noam's position is more like, "You'll never understand any chemistry unless you first figure out subatomic physics". That's just obviously false, we can understand and do quite a bit of chemistry without a full understanding of quantum field theory. At best, such knowledge impacts our understanding at the fringes of chemistry.
      - calf 4y ago
        You're missing the admission of the author who wrote the paper. If you didn't get that then you're just overselling the refutation, by not understanding the stated premise more critically. Literally the author used the analogy so I have no idea what your point is if you disagree with the author's analogy. It tells me you are not thinking through the piece analytically. It's not an uninteresting paper, but by the author's own admission, he doesn't give an astute reader good reason to proceed to the next section, since that was the analogy that he tried to draw.

slowmovintarget 4y ago

All the arguments for LLMs actually having "understanding" are of the form "I want to believe" or "you can't prove they're not." Chomsky's point that we have no basis for judging LLMs as cognition machines because we don't know what cognition is, is correct. Building these machines may produce useful tools, but it also doesn't advance studies of cognition, either. We do know one characteristic of cognition as it occurs in nature. There seem to be natural affinities to distinguish the real and true from the hallucination. Perhaps that comes from persistence and physicality. How do we give that to ChatGPT?

naasking 4y ago
> Chomsky's point that we have no basis for judging LLMs as cognition machines because we don't know what cognition is, is correct. That's not Chomsky's point at all. He's saying they are emphatically not understanding or intelligent in any way, and that's not justified. > Building these machines may produce useful tools, but it also doesn't advance studies of cognition, either. I couldn't disagree more. We have here apparently intelligent systems completely unlike naturally occurring ones. IMO, you'd have to be crazy to think that comparing and contrasting the properties of these systems won't provide any valuable data about the nature of intelligence, perception and cognition as a whole. In fact, we're already seeing papers discussing how LLMs basically refute Chomsky's whole model of language [1]. [1] https://lingbuzz.net/lingbuzz/007180 https://lingbuzz.net/lingbuzz/007180
- naasking 4y ago
  To clarify, by "completely unlike naturally occurring ones", I mean their origin and training set is completely unlike them, not that their operation is mechanistically different. We don't really know that yet.
- calf 4y ago
  I read the first 9 pages, it claims to refute it but I don't agree with the premise. The author admits the status quo is like when doctors discovered lemons cured scurvy, but nobody knew why. So Noam's position is basically that doctors will never discover Vitamin C unless they develop chemistry. Both positions are true, they're just talking past each other.
  - naasking 4y ago
    Then you missed the actual refutation, which starts on page 14, "The refutation of key principles". LLMs undermine many of the principles that Chomsky's position depends on. > So Noam's position is basically that doctors will never discover Vitamin C unless they develop chemistry. I disagree with this analogy. LLMs and machine learning is playing with chemistry, and Noam's position is more like, "You'll never understand any chemistry unless you first figure out subatomic physics". That's just obviously false, we can understand and do quite a bit of chemistry without a full understanding of quantum field theory. At best, such knowledge impacts our understanding at the fringes of chemistry.
    - calf 4y ago
      You're missing the admission of the author who wrote the paper. If you didn't get that then you're just overselling the refutation, by not understanding the stated premise more critically. Literally the author used the analogy so I have no idea what your point is if you disagree with the author's analogy. It tells me you are not thinking through the piece analytically. It's not an uninteresting paper, but by the author's own admission, he doesn't give an astute reader good reason to proceed to the next section, since that was the analogy that he tried to draw.