4 ms·

The weirdest thing to me is that a person with a high personal, financial involvement in the subject went and took that quote from an old man (which to your and

by fock 4y ago

The weirdest thing to me is that a person with a high personal, financial involvement in the subject went and took that quote from an old man (which to your and my understanding only states that these things are not linguistic "science", but they solve problems alright), then created a strawman from thin air (points A-E) to then say "oh, all my arguments are void, statistical models are great, don't you dare criticizing me, you old fool". And then he went and took "Science", aka the epitome of publish or perish academia and tried to argue that all this looks the same as the thing he does. Oh well, who would have guessed... The looks on this are weird, even more so as GPT nowadays works wonders, but still doesn't help explaining why and how language evolved (which seems to be the idea of linguistics, no?).

hackandthink 4y ago
Norvig has strong arguments, but this in bad faith: "Chomsky has a philosophy based on the idea that we should focus on the deep whys and that mere explanations of reality don't matter. In this, Chomsky is in complete agreement with O'Reilly." (O'Really stands for mythmaking, religion or philosophy) Chomsky is no mysticist - he is an old fashioned scientist looking for a parsimonious theory. Maybe there is no simple theory for language with great explanatory power but there should be some people looking for it.
- foobarqux 4y ago
  > Norvig has strong arguments What was one of those arguments? I didn't see any.
  - hackandthink 4y ago
    "We all saw the limitations of the old tools, and the benefits of the new" Probabilistic models work incredibly well, much better than transformational-generative grammars.
    - foobarqux 4y ago
      > Probabilistic models work incredibly well, much better than transformational-generative grammars. You've missed everything Chomsky said even though it is repeated in the article: Probabilistic models can be useful tools but they tell you nothing about the human language faculty (i.e. they are not science).
      - visarga 4y ago
        This kind of top-down approach misses the real hero - it's not the model, it's the data. 500GB of text can transform a randomly initialised neural net into a chatting, problem solving AI. And language turns babies into modern, functional adults. It doesn't matter how the model is implemented, they all learn more or less, the real hero is text. Let's talk about it more. It would have been interesting if Chomsky's approach could have predicted at what size of text we see the emergence of AI that passes the Turing test. Or even if it predicted that there is an emergent process in there.
        foobarqux 4y ago
        No, again it's missing the point: None of this explains how the human language faculty works.
        morelisp 4y ago
        A child needs about 100KB of "text" a day to learn a language. If anything the data requirements of LLMs are proof positive they can't bear any relation to the human language faculty.
        pixl97 4y ago
        I mean we could be far overfeeding our data models too. Of course data models don't take years in real time to train either.
        morelisp 4y ago
- YeGoblynQueenne 4y ago
  >> Chomsky is no mysticist - he is an old fashioned scientist looking for a parsimonious theory. When did that become old fashioned?
  - hackandthink 4y ago
    (mostly a joke) In the good old days a scientist was a hard thinker and published each year a paper and it was hard to gather data. (Einstein thinking about general relativity publishing after how man years?) Nowadays a scientist scraps data from the web and hacks p-values. In the future LLMs will feed each other their generated papers.
LudwigNagasena 4y ago
Have you read the original question by Pinker and the response by Chomsky (http://languagelog.ldc.upenn.edu/myl/PinkerChomskyMIT.html http://languagelog.ldc.upenn.edu/myl/PinkerChomskyMIT.html)? It doesn't look like a strawman, though it's a bit hard to get what he was gesturing at as the answer was impromptu.
dang 4y ago
I'm not sure who you're slurring worse here but can you please make your substantive points without personal attack and generally not post in the flamewar style? We're trying to avoid that here. We detached this subthread from https://news.ycombinator.com/item?id=34857541 https://news.ycombinator.com/item?id=34857541.

by fock 4y ago

hackandthink 4y ago

Norvig has strong arguments, but this in bad faith: "Chomsky has a philosophy based on the idea that we should focus on the deep whys and that mere explanations of reality don't matter. In this, Chomsky is in complete agreement with O'Reilly." (O'Really stands for mythmaking, religion or philosophy) Chomsky is no mysticist - he is an old fashioned scientist looking for a parsimonious theory. Maybe there is no simple theory for language with great explanatory power but there should be some people looking for it.

foobarqux 4y ago
> Norvig has strong arguments What was one of those arguments? I didn't see any.
- hackandthink 4y ago
  "We all saw the limitations of the old tools, and the benefits of the new" Probabilistic models work incredibly well, much better than transformational-generative grammars.
  - foobarqux 4y ago
    > Probabilistic models work incredibly well, much better than transformational-generative grammars. You've missed everything Chomsky said even though it is repeated in the article: Probabilistic models can be useful tools but they tell you nothing about the human language faculty (i.e. they are not science).
    - visarga 4y ago
      This kind of top-down approach misses the real hero - it's not the model, it's the data. 500GB of text can transform a randomly initialised neural net into a chatting, problem solving AI. And language turns babies into modern, functional adults. It doesn't matter how the model is implemented, they all learn more or less, the real hero is text. Let's talk about it more. It would have been interesting if Chomsky's approach could have predicted at what size of text we see the emergence of AI that passes the Turing test. Or even if it predicted that there is an emergent process in there.
      - foobarqux 4y ago
        No, again it's missing the point: None of this explains how the human language faculty works.
      - morelisp 4y ago
        A child needs about 100KB of "text" a day to learn a language. If anything the data requirements of LLMs are proof positive they can't bear any relation to the human language faculty.
        pixl97 4y ago
        I mean we could be far overfeeding our data models too. Of course data models don't take years in real time to train either.
        morelisp 4y ago
YeGoblynQueenne 4y ago
>> Chomsky is no mysticist - he is an old fashioned scientist looking for a parsimonious theory. When did that become old fashioned?
- hackandthink 4y ago
  (mostly a joke) In the good old days a scientist was a hard thinker and published each year a paper and it was hard to gather data. (Einstein thinking about general relativity publishing after how man years?) Nowadays a scientist scraps data from the web and hacks p-values. In the future LLMs will feed each other their generated papers.

LudwigNagasena 4y ago

Have you read the original question by Pinker and the response by Chomsky (http://languagelog.ldc.upenn.edu/myl/PinkerChomskyMIT.html http://languagelog.ldc.upenn.edu/myl/PinkerChomskyMIT.html)? It doesn't look like a strawman, though it's a bit hard to get what he was gesturing at as the answer was impromptu.

dang 4y ago

I'm not sure who you're slurring worse here but can you please make your substantive points without personal attack and generally not post in the flamewar style? We're trying to avoid that here. We detached this subthread from https://news.ycombinator.com/item?id=34857541 https://news.ycombinator.com/item?id=34857541.