3 ms·

> You can pre-train an LLM on English or an impossible language and they do equally well It's impressive that LLMs can learn languages that humans cannot. In

> You can pre-train an LLM on English or an impossible language and they do equally well It's impressive that LLMs can learn languages that humans cannot. In what frame is this a negative? Separately, "impossible language" is a pretty clear misnomer. If an LLM can learn it, it's possible.

foobarqux 1y ago
The latter. Moro showed that you can construct simple language rules, in particular linear rules, like the third word of every sentence modifies the noun, that humans have a hard time learning (specifically they use different parts of their brain in MRI scans and take longer to process than control languages) and are different from conventional human language structure (which hierarchical structure dependent, i.e. roughly that words are interpreted according to their position in a parse tree not their linear order). That's what "impossible language" means in this context, not something like computationally impossible or random.
- paulsutter 1y ago
  Ok then .. what makes that a negative? You're describing a human limitation and a strength of LLMs
  - foobarqux 1y ago
    It's not a negative, it's just not what humans do, which is Chomsky's (a person studying what humans do) point. As I said in another comment this whole dispute would be put to bed if people understood that they don't care about what humans do (and that Chomsky does).
    - paulsutter 1y ago
      Suggestion for you then, in your first response you would have been clearer to say "The reason Chomsky seems like such a retard here, is because he clings to irrelevant nonsense" It's completely unremarkable that humans are unable to learn certain languages, and soon it will be unremarkable when humans have no cognitive edge over machines. Response: Science? "Ancient Linguistics" would more accurately describe Chomsky's field of study and its utility
      - foobarqux 1y ago
        > Suggestion for you then, in your first response you would have been clearer to say "The reason Chomsky seems like such a retard here, is because he clings to irrelevant nonsense" If science is irrelevant to you it's you who should have recognized this before spouting off.

foobarqux 1y ago

The latter. Moro showed that you can construct simple language rules, in particular linear rules, like the third word of every sentence modifies the noun, that humans have a hard time learning (specifically they use different parts of their brain in MRI scans and take longer to process than control languages) and are different from conventional human language structure (which hierarchical structure dependent, i.e. roughly that words are interpreted according to their position in a parse tree not their linear order). That's what "impossible language" means in this context, not something like computationally impossible or random.

paulsutter 1y ago
Ok then .. what makes that a negative? You're describing a human limitation and a strength of LLMs
- foobarqux 1y ago
  It's not a negative, it's just not what humans do, which is Chomsky's (a person studying what humans do) point. As I said in another comment this whole dispute would be put to bed if people understood that they don't care about what humans do (and that Chomsky does).
  - paulsutter 1y ago
    Suggestion for you then, in your first response you would have been clearer to say "The reason Chomsky seems like such a retard here, is because he clings to irrelevant nonsense" It's completely unremarkable that humans are unable to learn certain languages, and soon it will be unremarkable when humans have no cognitive edge over machines. Response: Science? "Ancient Linguistics" would more accurately describe Chomsky's field of study and its utility
    - foobarqux 1y ago
      > Suggestion for you then, in your first response you would have been clearer to say "The reason Chomsky seems like such a retard here, is because he clings to irrelevant nonsense" If science is irrelevant to you it's you who should have recognized this before spouting off.