4 ms·

Chomsky's article really seems to show the results of decades of living in an echo chamber. >On the contrary, the human mind is a surprisingly efficient and ev

by twarge 4y ago

Chomsky's article really seems to show the results of decades of living in an echo chamber. >On the contrary, the human mind is a surprisingly efficient and even elegant system that operates with small amounts of information; True, but then > it seeks not to infer brute correlations among data points but to create explanations. This is such an odd dichotomy, and feels really wrong to me. Explanations are to me just further correlations. > a young child acquiring a language is developing — unconsciously, automatically and speedily from minuscule data — a grammar, a stupendously sophisticated system of logical principles and parameters. Chomsky apparently has children, but clearly may have forgotten what it's like for them to learn language! It's years of trying to communicate, constantly, failing, and learning. Now, go ask ChatGPT to Write an ode to <obscure topic> in iambic pentameter and I bet that it will be as astonishing, creative and fluent as a native speaker.

Ologn 4y ago
> Chomsky apparently has children, but clearly may have forgotten what it's like for them to learn language! It's years of trying to communicate, constantly failing, and learning. You are correct that children learning language are constantly failing in some manner. There has been a lot of study of the language acquisition of children. However, the way in which children fail and do not fail is specific. Some things children will fail at a number of times until they learn. On the other hand some rules of grammar, logical principles and parameters children seem to pick up immediately, and in Chomsky's view are even innate. So you can divide theoretically possible failures into two categories - ones theoretically possible that are not made, and ones that are theoretically possible and are made. The first category is what he is talking about. Also - there are different languages with different rules, and people learning a language have to learn the rules of English which are different from Spanish which are different than Japanese. Although in Chomsky's view, these languages are all so similar (compared to say the recursively enumerable language a Turing machine uses), they are virtually identical from a theoretical point of view. Also - children make grammar errors such as first saying "I go to the store" (correct) and then "I goed to the store" (incorrect). These errors were not made by data they acquired! The errors children make are showing they are beginning to understand grammar rules, it's a sign they are going off a series of rules and not just regurgitating data they have acquired.
- twarge 4y ago
  I'm not here to argue about Chomsky's basic claims (which I do not find useful), but I am here specifically to argue against the idea that children easily learn language with somehow way less data than ChatGPT. Those who claim otherwise simply have not spent a couple years with a baby.
  - ambicapter 4y ago
    ChatGPT is trained on some healthy fraction of the internet. You're arguing that the amount of information a baby consumes to learn language is comparable to the size of the internet?
    - sebzim4500 4y ago
      A baby almost certainly consumes far more data before it speaks than has been fed into GPT-3, yes.
  - morelisp 4y ago
    GPT3's input is something like 45TB of data. It's a significant fraction of all text humans ever produced, to the point researches are concerned we may run out of text within the next few years. Normal healthy language exposure for children is around 100KB/day, we can round up to 40MB/year. No, I have not spent ~1000000 years around a baby.
- genman 4y ago
  Chomsky is a fool and this article demonstrates it clearly. First we don't really know how the mind works and therefore we really have no idea if the language models are path toward AGI or not. He doesn't have an idea how a child obtains a language. You really need a lot of data to acquire the grammar, no, minuscule data is not enough. I have observed it personally with my multilingual children, their mother tongue, to what they have the least exposure and what is the most complex, is not still not fluent for them and yes, they make obvious grammar mistakes that demonstrates that they have not fully internalized it. His claims are simply false, easily refuted by empirical data. I would call it fallacy of English. English is stupid simple language. Most languages are not.

by twarge 4y ago

Ologn 4y ago

> Chomsky apparently has children, but clearly may have forgotten what it's like for them to learn language! It's years of trying to communicate, constantly failing, and learning. You are correct that children learning language are constantly failing in some manner. There has been a lot of study of the language acquisition of children. However, the way in which children fail and do not fail is specific. Some things children will fail at a number of times until they learn. On the other hand some rules of grammar, logical principles and parameters children seem to pick up immediately, and in Chomsky's view are even innate. So you can divide theoretically possible failures into two categories - ones theoretically possible that are not made, and ones that are theoretically possible and are made. The first category is what he is talking about. Also - there are different languages with different rules, and people learning a language have to learn the rules of English which are different from Spanish which are different than Japanese. Although in Chomsky's view, these languages are all so similar (compared to say the recursively enumerable language a Turing machine uses), they are virtually identical from a theoretical point of view. Also - children make grammar errors such as first saying "I go to the store" (correct) and then "I goed to the store" (incorrect). These errors were not made by data they acquired! The errors children make are showing they are beginning to understand grammar rules, it's a sign they are going off a series of rules and not just regurgitating data they have acquired.

twarge 4y ago
I'm not here to argue about Chomsky's basic claims (which I do not find useful), but I am here specifically to argue against the idea that children easily learn language with somehow way less data than ChatGPT. Those who claim otherwise simply have not spent a couple years with a baby.
- ambicapter 4y ago
  ChatGPT is trained on some healthy fraction of the internet. You're arguing that the amount of information a baby consumes to learn language is comparable to the size of the internet?
  - sebzim4500 4y ago
    A baby almost certainly consumes far more data before it speaks than has been fed into GPT-3, yes.
- morelisp 4y ago
  GPT3's input is something like 45TB of data. It's a significant fraction of all text humans ever produced, to the point researches are concerned we may run out of text within the next few years. Normal healthy language exposure for children is around 100KB/day, we can round up to 40MB/year. No, I have not spent ~1000000 years around a baby.
genman 4y ago
Chomsky is a fool and this article demonstrates it clearly. First we don't really know how the mind works and therefore we really have no idea if the language models are path toward AGI or not. He doesn't have an idea how a child obtains a language. You really need a lot of data to acquire the grammar, no, minuscule data is not enough. I have observed it personally with my multilingual children, their mother tongue, to what they have the least exposure and what is the most complex, is not still not fluent for them and yes, they make obvious grammar mistakes that demonstrates that they have not fully internalized it. His claims are simply false, easily refuted by empirical data. I would call it fallacy of English. English is stupid simple language. Most languages are not.