7 ms·

Having studied Chomsky a fair bit in grad school, and also studied cognitive linguistics a fair bit in grad school, I think the idea that Chomsky's models will

by phuff 14y ago

Having studied Chomsky a fair bit in grad school, and also studied cognitive linguistics a fair bit in grad school, I think the idea that Chomsky's models will ever win anything is just wrong. Chomsky's central problem is that his modeling is not based on anything biological at all. His models don't correspond to reality. Some of them were based on some assumptions about how the brain works that were untestable in the 50s and 60s when all of his linguistic models were developed, and have since become testable and are not particularly evident in the way we currently understand the brain to work. Given this fact, I think your current best bet is Norvig as a modern approach to AI or anything linguistic-y. But this is only because it is slightly more grounded in reality rather than being something that Chomsky (who is a very smart guy) came up with on his own without the benefit of actual biological models of the brain. In the end, I think there will be (eventually, a long time from now) an actual model of how the brain processes language based on actual observations of working brains that throws away much of what Chomsky has proposed but probably uses some of it and that doesn't use huge Google-esque lookup tables but is highly influenced by statistics. But until we get to that point, statistics are probably your best bet since at least they're grounded in reality (unlike much of Chomsky's work).

sp332 14y ago
Children learn language much faster than it seems possible for a "blank" neural network to learn. It seems that there is some "circuitry" hard-wired into the human brain that helps learning language. So the question is: can a computer learn language as well as a human, without simply hard-coding language into it?
- debacle 14y ago
  Children are definitely not a 'blank' neural network. They spend ~6 months staring off into nowhere, looking, listening, and slowly developing the skills to respond to their environment.
  - sp332 14y ago
    By "blank" of course I mean that they begin blank and immediately start learning from their environment. You say they develop skills "slowly" but it's still much faster than you would expect, unless children have some innate skill at language built in instead of being "blank." Edit: sorry if this is vague, this is what I'm talking about https://en.wikipedia.org/wiki/Psychological_nativism https://en.wikipedia.org/wiki/Psychological_nativism
    - knowtheory 14y ago
      You are being way too vague. What is setting your expectations of "slowly"? What rate would you expect children to learn language at? Even if that were to be the case, your argument is essentially a god of the gaps argument. Not P therefor Q is not sound reasoning. The whole notion of the Universal Grammar and innate language faculties which instantiate subsets of the Universal Grammar is weird.
      - gnaritas 14y ago
        No, he's right, check out Pinkers the language instinct for the full treatment on the issue. Children learn at a rate impossible from just what they hear and imitate. We are pre-wired for language.
        Retric 14y ago
        30,000+ hours using more computing power than the fastest computer on the planet to get the a 3 year olds grasp of language is hardly 'fast'. PS: Language acquisition starts before birth, and continues 24/7 after that until we 'learn' it.
        gnaritas 14y ago
        "Pinker explains that a universal grammar represents specific structures in the human brain that recognize the general rules of other humans' speech, such as whether the local language places adjectives before or after nouns, and begin a specialized and very rapid learning process not explainable as reasoning from first principles or pure logic. This learning machinery exists only during a specific critical period of childhood and is then disassembled for thrift, freeing resources in an energy-hungry brain." Having read the book, his arguments are far more convincing than your assertions.
- radarsat1 14y ago
  I'm not sure what you mean by "faster," (what are you comparing to exactly?), but I think something that speeds up human learning considerably as compared to computers is feedback. Children don't just blankly sit there taking in information and then "fitting it" to a model-- they perform actions and observe the consequences; it is empirical. The embodied action-perception loop is fundamental to how real-world learning works. A closer computer model is reinforcement learning, for example, which does exactly this, it wraps a neural network in an action-perception loop an uses online training to learn the reward function. The problem is of course that the reward function can be very hard to design except for fairly simple tasks.
- Jun8 14y ago
  This is a popular summary of Chomsky's thesis that was put down decades ago, when cognitive psychology was at its infancy. Now we know a lot more on how babies learn the world and language (do a Google search on "infant statistical learning") and most evidence points to the fact that they employ algorithms that are mainly statistical in nature for learning. "Children learn language much faster than it seems possible for a "blank" neural network to learn." This is a very strong statement that has no mathematical or computational proof AFAIK. It had no proof when Chomksy first put down that thesis either, it was an axiom of his. BTW, belief in a specialized language faculty was not universal, even in the past, esp. some philosophers of language disagreed with this view.
  - sp332 14y ago
    It follows logically from these 3 premises https://en.wikipedia.org/wiki/Poverty_of_the_stimulus#Summary https://en.wikipedia.org/wiki/Poverty_of_the_stimulus#Summar... Those premises could be wrong but it's not just an axiom.
    - Retric 14y ago
      #2 is simply wrong. Children are corrected when their grammar is off so they do get to see incorrect sentence structure. It's questionable if children could learn language from only watching TV, but that's not the standard learning environment. PS: I am far from the first person to point that out. At this point they are treating it as an axiom because they continue to believe it despite the disproof of their premises.
      - ddd571 14y ago
        Yet, they consistently ignore this input. This has also been pointed out dozens and dozens of times.
      - deleted 14y ago
        [deleted]
- deleted 14y ago
  [deleted]

by phuff 14y ago

sp332 14y ago

Children learn language much faster than it seems possible for a "blank" neural network to learn. It seems that there is some "circuitry" hard-wired into the human brain that helps learning language. So the question is: can a computer learn language as well as a human, without simply hard-coding language into it?

debacle 14y ago
Children are definitely not a 'blank' neural network. They spend ~6 months staring off into nowhere, looking, listening, and slowly developing the skills to respond to their environment.
- sp332 14y ago
  By "blank" of course I mean that they begin blank and immediately start learning from their environment. You say they develop skills "slowly" but it's still much faster than you would expect, unless children have some innate skill at language built in instead of being "blank." Edit: sorry if this is vague, this is what I'm talking about https://en.wikipedia.org/wiki/Psychological_nativism https://en.wikipedia.org/wiki/Psychological_nativism
  - knowtheory 14y ago
    You are being way too vague. What is setting your expectations of "slowly"? What rate would you expect children to learn language at? Even if that were to be the case, your argument is essentially a god of the gaps argument. Not P therefor Q is not sound reasoning. The whole notion of the Universal Grammar and innate language faculties which instantiate subsets of the Universal Grammar is weird.
    - gnaritas 14y ago
      No, he's right, check out Pinkers the language instinct for the full treatment on the issue. Children learn at a rate impossible from just what they hear and imitate. We are pre-wired for language.
      - Retric 14y ago
        30,000+ hours using more computing power than the fastest computer on the planet to get the a 3 year olds grasp of language is hardly 'fast'. PS: Language acquisition starts before birth, and continues 24/7 after that until we 'learn' it.
        gnaritas 14y ago
        "Pinker explains that a universal grammar represents specific structures in the human brain that recognize the general rules of other humans' speech, such as whether the local language places adjectives before or after nouns, and begin a specialized and very rapid learning process not explainable as reasoning from first principles or pure logic. This learning machinery exists only during a specific critical period of childhood and is then disassembled for thrift, freeing resources in an energy-hungry brain." Having read the book, his arguments are far more convincing than your assertions.
radarsat1 14y ago
I'm not sure what you mean by "faster," (what are you comparing to exactly?), but I think something that speeds up human learning considerably as compared to computers is feedback. Children don't just blankly sit there taking in information and then "fitting it" to a model-- they perform actions and observe the consequences; it is empirical. The embodied action-perception loop is fundamental to how real-world learning works. A closer computer model is reinforcement learning, for example, which does exactly this, it wraps a neural network in an action-perception loop an uses online training to learn the reward function. The problem is of course that the reward function can be very hard to design except for fairly simple tasks.
Jun8 14y ago
This is a popular summary of Chomsky's thesis that was put down decades ago, when cognitive psychology was at its infancy. Now we know a lot more on how babies learn the world and language (do a Google search on "infant statistical learning") and most evidence points to the fact that they employ algorithms that are mainly statistical in nature for learning. "Children learn language much faster than it seems possible for a "blank" neural network to learn." This is a very strong statement that has no mathematical or computational proof AFAIK. It had no proof when Chomksy first put down that thesis either, it was an axiom of his. BTW, belief in a specialized language faculty was not universal, even in the past, esp. some philosophers of language disagreed with this view.
- sp332 14y ago
  It follows logically from these 3 premises https://en.wikipedia.org/wiki/Poverty_of_the_stimulus#Summary https://en.wikipedia.org/wiki/Poverty_of_the_stimulus#Summar... Those premises could be wrong but it's not just an axiom.
  - Retric 14y ago
    #2 is simply wrong. Children are corrected when their grammar is off so they do get to see incorrect sentence structure. It's questionable if children could learn language from only watching TV, but that's not the standard learning environment. PS: I am far from the first person to point that out. At this point they are treating it as an axiom because they continue to believe it despite the disproof of their premises.
    - ddd571 14y ago
      Yet, they consistently ignore this input. This has also been pointed out dozens and dozens of times.
    - deleted 14y ago
      [deleted]
deleted 14y ago
[deleted]