3 ms·

> obviously they are going to do much better on languages better represented in the training set. Which, again, indicates that the current crop of LLM’s are ju

> obviously they are going to do much better on languages better represented in the training set. Which, again, indicates that the current crop of LLM’s are just fuzzy-recollection/fuzzy-interpolation machines, and have none of the actual thought/reasoning capabilities some people are claiming they have.

int_19h 3y ago
If that were the case, it would be impossible to describe rules to them and have them follow those rules.
ravelfan 3y ago
I think Chomsky's idea that LLM's are a form of automated plagiarism is the best description if we think of plagiarism in a neutral sense and not a highly negative sense. If we think in terms of automated plagiarism, no one can complain that the model doesn't produce original ideas. Automated plagiarism could have many uses but it is obviously stupid to expect an original idea from plagiarism. We wouldn't complain the model can't reproduce things it hasn't seen before if we think in terms of plagiarism.
- HarHarVeryFunny 3y ago
  Chomsky has been wrong for decades. He doesn't want to believe that LLMs are really learning language because that conflicts with his assertion that it takes something much more specialized to do so. Basically Chompsky has been proved wrong by LLMs, but rather than believing what is in front of his eyes he is saying "that can't be right, because it'd mean I am wrong".
  - FridgeSeal 3y ago
    “Chomsky is wrong because I say he is, and because he doesn’t like the LLM I like” Mmmhhmmm yes very convincing argument there. I’d be willing to bet that Chomsky’s arguments don’t rely on such spurious arguments. Automated plagiarism reads like a fantastic description of current LLM’s to me.
    - HarHarVeryFunny 3y ago
      Do you disagree that LLMs appear to have learnt language? Whether they plagiarize or not is irrelevant as long as they are not quoting verbatim. If they are coming up with their own sequence of words, making sense, and remaining grammatical, then how can you argue that they have not learnt language? Since they do at least (whatever other shortcomings they may have) appear to have learnt language, just by "listening to" a bunch of examples, and since they are based on a transformer, not some specialized Chompskian "language organ", then the thesis that a "language organ" is needed is disproved.
      - foobarqux 3y ago
        Even assuming LLMs have learnt language (whatever that means) it is completely irrelevant to what Chomsky is studying which is how the human language ability works. As an analogy training a neural network to successfully predict the weather doesn't falsify physics-based models of the weather.
        HarHarVeryFunny 3y ago
        Sure, but for our cortex not to be able to learn language without becoming specialized for it, while a transformer can, implies that a transformer is a more powerful prediction engine than our cortex, which seems unlikely. If you look at the cortex as a whole, what seems most striking is how uniform it is - same basic architecture of six layers of neurons with a specific pattern of inter-layer connectivity. It seems nature has come up with a universal prediction architecture. Maybe Wernicke's areas is fine-tuned for language, but to characterize it as a specialized "language organ" seems a bit of a stretch. Let's note too that language is only a million years old, while the cortex itself is 100's of millions, yet has this mostly uniform architecture that evidentially works just as well for vision as hearing, etc, etc. So, sure, the ability of the ridiculously simple transformer architecture to learn language (and many other prediction tasks you throw at it), doesn't PROVE that the brain didn't have to evolve a highly specialized way to do it, but it certainly seems highly suggestive of it. Since we now have an existence proof that a very simple architecture, not specialized for language, can learn language, it seems the onus is now on Chompsky to put some meat on the bones of his claim (without evidence) that the general cortical architecture is incapable of this without a high degree of specialization.
HarHarVeryFunny 3y ago
If that were the case, then you wouldn't get the transfer learning from models being trained on programming languages, to reasoning applied in a non-programming context, which has been universally observed for all these models. They are learning abstract "thought patterns", not just interpolation. Learning a programming language (or any complex skill) just by reading a book and looking at examples, without actually practicing it yourself, is basically impossible, even for ourselves, let alone an LLM. It is amazing how well the LLMs can do at programming in cases where they were trained on masses of well-commented examples, but without also being able to learn "on the job" themselves, you need to tamper expectations.

int_19h 3y ago

If that were the case, it would be impossible to describe rules to them and have them follow those rules.

ravelfan 3y ago

I think Chomsky's idea that LLM's are a form of automated plagiarism is the best description if we think of plagiarism in a neutral sense and not a highly negative sense. If we think in terms of automated plagiarism, no one can complain that the model doesn't produce original ideas. Automated plagiarism could have many uses but it is obviously stupid to expect an original idea from plagiarism. We wouldn't complain the model can't reproduce things it hasn't seen before if we think in terms of plagiarism.

HarHarVeryFunny 3y ago
Chomsky has been wrong for decades. He doesn't want to believe that LLMs are really learning language because that conflicts with his assertion that it takes something much more specialized to do so. Basically Chompsky has been proved wrong by LLMs, but rather than believing what is in front of his eyes he is saying "that can't be right, because it'd mean I am wrong".
- FridgeSeal 3y ago
  “Chomsky is wrong because I say he is, and because he doesn’t like the LLM I like” Mmmhhmmm yes very convincing argument there. I’d be willing to bet that Chomsky’s arguments don’t rely on such spurious arguments. Automated plagiarism reads like a fantastic description of current LLM’s to me.
  - HarHarVeryFunny 3y ago
    Do you disagree that LLMs appear to have learnt language? Whether they plagiarize or not is irrelevant as long as they are not quoting verbatim. If they are coming up with their own sequence of words, making sense, and remaining grammatical, then how can you argue that they have not learnt language? Since they do at least (whatever other shortcomings they may have) appear to have learnt language, just by "listening to" a bunch of examples, and since they are based on a transformer, not some specialized Chompskian "language organ", then the thesis that a "language organ" is needed is disproved.
    - foobarqux 3y ago
      Even assuming LLMs have learnt language (whatever that means) it is completely irrelevant to what Chomsky is studying which is how the human language ability works. As an analogy training a neural network to successfully predict the weather doesn't falsify physics-based models of the weather.
      - HarHarVeryFunny 3y ago
        Sure, but for our cortex not to be able to learn language without becoming specialized for it, while a transformer can, implies that a transformer is a more powerful prediction engine than our cortex, which seems unlikely. If you look at the cortex as a whole, what seems most striking is how uniform it is - same basic architecture of six layers of neurons with a specific pattern of inter-layer connectivity. It seems nature has come up with a universal prediction architecture. Maybe Wernicke's areas is fine-tuned for language, but to characterize it as a specialized "language organ" seems a bit of a stretch. Let's note too that language is only a million years old, while the cortex itself is 100's of millions, yet has this mostly uniform architecture that evidentially works just as well for vision as hearing, etc, etc. So, sure, the ability of the ridiculously simple transformer architecture to learn language (and many other prediction tasks you throw at it), doesn't PROVE that the brain didn't have to evolve a highly specialized way to do it, but it certainly seems highly suggestive of it. Since we now have an existence proof that a very simple architecture, not specialized for language, can learn language, it seems the onus is now on Chompsky to put some meat on the bones of his claim (without evidence) that the general cortical architecture is incapable of this without a high degree of specialization.

HarHarVeryFunny 3y ago

If that were the case, then you wouldn't get the transfer learning from models being trained on programming languages, to reasoning applied in a non-programming context, which has been universally observed for all these models. They are learning abstract "thought patterns", not just interpolation. Learning a programming language (or any complex skill) just by reading a book and looking at examples, without actually practicing it yourself, is basically impossible, even for ourselves, let alone an LLM. It is amazing how well the LLMs can do at programming in cases where they were trained on masses of well-commented examples, but without also being able to learn "on the job" themselves, you need to tamper expectations.