4 ms·

Chomsky gets it wrong here. A fairer analogy can be made by first supposing that submarines do not exist yet, and people are intensely interested if one could

Chomsky gets it wrong here. A fairer analogy can be made by first supposing that submarines do not exist yet, and people are intensely interested if one could actually be built. Add in the situation that many people are claiming that such a swimming machine is inherently impossible because any machine would lack the swim-enabling "stuff" that fish have. Then one day submarines are built. It passes all tests of swimming ability that naysayers said were impossible for a machine to pass. And now the naysayers say that they weren't actually all that interested in the question of swimming machines in the first place. The actually interesting question is how do fish do it.

notatoad 3y ago
i think "Chomsky gets it right by accident" is maybe better than "Chomsky gets it wrong". "do LLMs think" and "do submarines swim" is kind of the same question - it's a fixation on terminology rather than results. submarines accomplish something useful, and you can either quibble about what terminology to apply to that, or you can use them for the things they're good for and not be super bothered about what it's called. it seems to me that humans have a vested interest in defining thinking as an innately human thing, so it's impossible to build a truly thinking machine, because if you do we'll simply find a new definition of thinking that excludes your machine.
foobarqux 3y ago
> And now the naysayers say that they weren't actually all that interested in the question of swimming machines in the first place. The actually interesting question is how do fish do it. Yes, if you are studying how fish swim it's completely irrelevant that you can build a machine to swim. Chomsky has been making this point for decades. He has never said you could not statistically approximate a natural process, in fact he has explicitly said that you probably could, it's just that it's irrelevant for scientific understanding.
- drdaeman 3y ago
  Just curious. And how far/detailed is modern scientific understanding of human cognition? I have an impression that, generally speaking, unlike how we know how an LLM operates, we barely have any clues on how humans form thoughts - or am I wrong here? Just some overall ideas that it's "probably somewhere out there, where fMRI shows the activity" but nothing concrete?
  - foobarqux 3y ago
    Chomsky himself says it's extremely primitive ("pre-Galilean" he says). "Where thoughts come from" is probably too vague too even consider as a scientific question but very few things are known relative to the rest of science.
- markisus 3y ago
  Even if you only care about fish, it is possible that the pursuit of a swimming machine will yield insights into how fish do it. In terms of scientific understanding in general, I'm not sure why you would say that building a statistical model of a natural process is be irrelevant. On one level, much of science is about building models of natural processes and then probing those models for insights. On another level, attempts to build artificial intelligence are acts of science. We are studying the limits of statistical algorithms in their ability to pick out patterns from raw data. This study has produced data that has already surprised those who had assumed that a competent language model would need a universal grammar component, as mentioned in the article. If a solid theory is developed, I assume it would yield insights for studies of human cognition.
  - foobarqux 3y ago
    > Even if you only care about fish, it is possible that the pursuit of a swimming machine will yield insights into how fish do it. It's possible but it doesn't happen by accident, you have to show that they are related, and no one does (because the people who work on LLMs aren't actually interested in this question, despite the fact that argue it to death). > In terms of scientific understanding in general, I'm not sure why you would say that building a statistical model of a natural process is be irrelevant. On one level, much of science is about building models of natural processes and then probing those models for insights. You have to have a reason to believe that it would yield insights. There isn't one. > attempts to build artificial intelligence are acts of science. They are not, they are math and engineering. > If a solid theory is developed, I assume it would yield insights for studies of human cognition. You have to actually have a rational process to produce insights, not just mix a bunch of numbers in a vat and "assume" that some insight is going to come from it.
    - markisus 3y ago
      > It's possible but it doesn't happen by accident, you have to show that they are related, and no one does (because the people who work on LLMs aren't actually interested in this question, despite the fact that argue it to death). The relatedness of human language cognition and machine language performance is already evident, in that studies of machine language performance can already falsify some theories of human language cognition. The pattern goes like this. 1. Language competency requires component X. 2. Humans have language competency. 3. Therefore, humans have component X. When we see a machine, without component X, using language as a human does, we know the theory has gone wrong at the first step. The article mentions one such instantiation of this pattern. The new data has narrowed the space of plausible theories of human language cognition. The machine learning engineers play the role of experimentalists. They collect data but may not have a theory ready to explain what results. > You have to have a reason to believe that it would yield insights. There isn't one. The reason is that models in general yield insights. They have yielded insights before, and so we may expect them to yield insights in the future. The explanation for this is either philosophical or psychological. Whatever the reason, humans who look into a specific system are very good at generalizing to a whole class of systems. F=ma is a model, and certainly wrong. But by probing it, we can begin to discover general principles like energy conservation. In the same way, a large language model is a model of a system that acquires language. By examining its specific behavior, we may come to understand more general principles of language acquisition. > You have to actually have a rational process to produce insights, not just mix a bunch of numbers in a vat and "assume" that some insight is going to come from it. Where there is already an understood rational process, you no longer need the insight. Sometimes the big leaps of scientific understanding come from looking at the data and then coming to Eureka.
- jameshart 3y ago
  > [Chomsky] has never said you could not statistically approximate a natural process, in fact he has explicitly said that you probably could Okay, but the article has Rodney Brooks asserting that > I personally remember very heated arguments on the MIT campus between Chomsky followers and computer scientists working on statistical models in the early 2000’s. The arguments came up in computer science faculty hiring meetings. The Chomskians claimed that because of the need for universal grammar there would never be anything useful that came out of statistical approaches to language. ChatGPT has proved them wrong. Worth noting that in the early 2000s, Rodney Brooks was director of the MIT Computer Science and Artificial Intelligence Laboratory, so... you know, he might have some context on that debate. Although I'll concede he attributes this to Chomsky followers, not to Chomsky.
yetanother12345 3y ago
I've not read Chomsky first hand, but the points referred in the article as "Key points of Chomsky’s Universal Grammar" seem to be a non-traditional description of a LLM, IMHO (some amount of creative interpretation required): - Innate Language Faculty: A “language acquisition device” - Universal Grammar Principles - Poverty of the Stimulus - Language Acquisition as Rule-Based (I'm only suggesting that these points are broadly analogous, and in a very broad sense. The Innate Faculty being the LLM itself, the Universal Grammar Principles being the stochastic weights, the Stimulus being the reinforcement mechanism, and the Rule Based Aquisition being a specific methodology of how to apply stochastic methods to language flow, etc.) From the article I understand that Chomsky is specialized in human language, and it seems to suggest that C does not believe that language exists outside of the human domain. I'm not sure if this is really the opinion of C, or if the article just makes it seem so. Perhaps this language-learning model is really more universal than it is human-specific. We'll see when/if "Doctor Doolittle" models start to emerge eventually
- foobarqux 3y ago
  > it seems to suggest that C does not believe that language exists outside of the human domain That is in fact what he explicitly says. He thinks it's nonsense to look at an abstract concept of "digestion" for example that is divorced from natural phenomenon.

notatoad 3y ago

i think "Chomsky gets it right by accident" is maybe better than "Chomsky gets it wrong". "do LLMs think" and "do submarines swim" is kind of the same question - it's a fixation on terminology rather than results. submarines accomplish something useful, and you can either quibble about what terminology to apply to that, or you can use them for the things they're good for and not be super bothered about what it's called. it seems to me that humans have a vested interest in defining thinking as an innately human thing, so it's impossible to build a truly thinking machine, because if you do we'll simply find a new definition of thinking that excludes your machine.

foobarqux 3y ago

> And now the naysayers say that they weren't actually all that interested in the question of swimming machines in the first place. The actually interesting question is how do fish do it. Yes, if you are studying how fish swim it's completely irrelevant that you can build a machine to swim. Chomsky has been making this point for decades. He has never said you could not statistically approximate a natural process, in fact he has explicitly said that you probably could, it's just that it's irrelevant for scientific understanding.

drdaeman 3y ago
Just curious. And how far/detailed is modern scientific understanding of human cognition? I have an impression that, generally speaking, unlike how we know how an LLM operates, we barely have any clues on how humans form thoughts - or am I wrong here? Just some overall ideas that it's "probably somewhere out there, where fMRI shows the activity" but nothing concrete?
- foobarqux 3y ago
  Chomsky himself says it's extremely primitive ("pre-Galilean" he says). "Where thoughts come from" is probably too vague too even consider as a scientific question but very few things are known relative to the rest of science.
markisus 3y ago
Even if you only care about fish, it is possible that the pursuit of a swimming machine will yield insights into how fish do it. In terms of scientific understanding in general, I'm not sure why you would say that building a statistical model of a natural process is be irrelevant. On one level, much of science is about building models of natural processes and then probing those models for insights. On another level, attempts to build artificial intelligence are acts of science. We are studying the limits of statistical algorithms in their ability to pick out patterns from raw data. This study has produced data that has already surprised those who had assumed that a competent language model would need a universal grammar component, as mentioned in the article. If a solid theory is developed, I assume it would yield insights for studies of human cognition.
- foobarqux 3y ago
  > Even if you only care about fish, it is possible that the pursuit of a swimming machine will yield insights into how fish do it. It's possible but it doesn't happen by accident, you have to show that they are related, and no one does (because the people who work on LLMs aren't actually interested in this question, despite the fact that argue it to death). > In terms of scientific understanding in general, I'm not sure why you would say that building a statistical model of a natural process is be irrelevant. On one level, much of science is about building models of natural processes and then probing those models for insights. You have to have a reason to believe that it would yield insights. There isn't one. > attempts to build artificial intelligence are acts of science. They are not, they are math and engineering. > If a solid theory is developed, I assume it would yield insights for studies of human cognition. You have to actually have a rational process to produce insights, not just mix a bunch of numbers in a vat and "assume" that some insight is going to come from it.
  - markisus 3y ago
    > It's possible but it doesn't happen by accident, you have to show that they are related, and no one does (because the people who work on LLMs aren't actually interested in this question, despite the fact that argue it to death). The relatedness of human language cognition and machine language performance is already evident, in that studies of machine language performance can already falsify some theories of human language cognition. The pattern goes like this. 1. Language competency requires component X. 2. Humans have language competency. 3. Therefore, humans have component X. When we see a machine, without component X, using language as a human does, we know the theory has gone wrong at the first step. The article mentions one such instantiation of this pattern. The new data has narrowed the space of plausible theories of human language cognition. The machine learning engineers play the role of experimentalists. They collect data but may not have a theory ready to explain what results. > You have to have a reason to believe that it would yield insights. There isn't one. The reason is that models in general yield insights. They have yielded insights before, and so we may expect them to yield insights in the future. The explanation for this is either philosophical or psychological. Whatever the reason, humans who look into a specific system are very good at generalizing to a whole class of systems. F=ma is a model, and certainly wrong. But by probing it, we can begin to discover general principles like energy conservation. In the same way, a large language model is a model of a system that acquires language. By examining its specific behavior, we may come to understand more general principles of language acquisition. > You have to actually have a rational process to produce insights, not just mix a bunch of numbers in a vat and "assume" that some insight is going to come from it. Where there is already an understood rational process, you no longer need the insight. Sometimes the big leaps of scientific understanding come from looking at the data and then coming to Eureka.
jameshart 3y ago
> [Chomsky] has never said you could not statistically approximate a natural process, in fact he has explicitly said that you probably could Okay, but the article has Rodney Brooks asserting that > I personally remember very heated arguments on the MIT campus between Chomsky followers and computer scientists working on statistical models in the early 2000’s. The arguments came up in computer science faculty hiring meetings. The Chomskians claimed that because of the need for universal grammar there would never be anything useful that came out of statistical approaches to language. ChatGPT has proved them wrong. Worth noting that in the early 2000s, Rodney Brooks was director of the MIT Computer Science and Artificial Intelligence Laboratory, so... you know, he might have some context on that debate. Although I'll concede he attributes this to Chomsky followers, not to Chomsky.

yetanother12345 3y ago

I've not read Chomsky first hand, but the points referred in the article as "Key points of Chomsky’s Universal Grammar" seem to be a non-traditional description of a LLM, IMHO (some amount of creative interpretation required): - Innate Language Faculty: A “language acquisition device” - Universal Grammar Principles - Poverty of the Stimulus - Language Acquisition as Rule-Based (I'm only suggesting that these points are broadly analogous, and in a very broad sense. The Innate Faculty being the LLM itself, the Universal Grammar Principles being the stochastic weights, the Stimulus being the reinforcement mechanism, and the Rule Based Aquisition being a specific methodology of how to apply stochastic methods to language flow, etc.) From the article I understand that Chomsky is specialized in human language, and it seems to suggest that C does not believe that language exists outside of the human domain. I'm not sure if this is really the opinion of C, or if the article just makes it seem so. Perhaps this language-learning model is really more universal than it is human-specific. We'll see when/if "Doctor Doolittle" models start to emerge eventually

foobarqux 3y ago
> it seems to suggest that C does not believe that language exists outside of the human domain That is in fact what he explicitly says. He thinks it's nonsense to look at an abstract concept of "digestion" for example that is divorced from natural phenomenon.