5 ms·

Human children have the great benefit of interactively learning from their parents and other humans raising them. Could we expect a child to learn to speak if t

Human children have the great benefit of interactively learning from their parents and other humans raising them. Could we expect a child to learn to speak if they only heard recordings of existing speech with otherwise no human interaction/feedback - correcting them or offering customized and contextual new bits of information? It would be interesting to add feedback path for human corrective input. i.e. because it's direct interaction, feed it back but somehow weight it a little more than just another corpus input.

jpatokal 11y ago
The TL;DR appears to be "no", interaction is necessary: https://en.wikipedia.org/wiki/Language_acquisition#General_approaches https://en.wikipedia.org/wiki/Language_acquisition#General_a... Although for obvious reasons this is very hard to study experimentally: https://en.wikipedia.org/wiki/Language_deprivation_experiments https://en.wikipedia.org/wiki/Language_deprivation_experimen...
- eli_gottlieb 11y ago
  Proper language usage probably involves causal modelling, in which case intervention experiments are one of the only known ways to learn correctly.
solve 11y ago
I'm convinced that voice pitch, syllable meter, hand gestures, facial gestures, and other forms of non-word emphasis are also crucial to figuring out basic phrase chunking and word types, with minimal interaction, early on. Seems it would be far harder to infer the basic initial structure from just plain text.
tzs 11y ago
> Could we expect a child to learn to speak if they only heard recordings of existing speech with otherwise no human interaction/feedback - correcting them or offering customized and contextual new bits of information? I once asked a similar question on some online forum [1] where many linguists hung out. My question was if an English-only speaking household left a general interest Spanish language TV station on most of the time when they weren't actively using the TV to watch something, so that their child received a very large exposure to Spanish language programming (news, sports, soap operas, sitcoms, movies, etc) from birth onward, would the child naturally learn Spanish? I don't recall for sure what the linguists who responded said, but I think they all said the child would not learn Spanish from this. [1] I have no recollection of where this was.
- geomark 11y ago
  I've seen something like this in action with young kids who are given a tablet and stumble upon cartoons they like on YouTube, but in a different language. After they watch a few cartoons, YouTube's recommender system keeps offering them more cartoons in that language. And it isn't long before they start spouting words and phrases in that language.
- dzhiurgis 11y ago
  In Baltic states, majority of TV content either comes in native German or is US shows translated to Russian. I've picked up quite a bit of Russian by watching Discovery channel this way.
- spdionis 11y ago
  When I was little (5-7 years old) I had quite a few anime videos and magazines in italian sent to me by my parents who were abroad. Where I lived no one knew a single word of italian. I often watched and rewatched those videos and read those magazines without any other external input. I can tell you that doing that I easily learned the language. When I was 8 years old I also left for Italy and in two weeks of time I already started speaking fluently, albeit with a few mistakes. If the child will actually watch the Spanish TV he will learn the language. EDIT: Even now I often learn new japanese words (and remember them) just by watching animes. The difference is that now I have english subtitles but back then I had no subtitles, only the images to help me understand the meaning.
  - digikata 11y ago
    But this is a little different, in the AI we want the ability to form syntactically correct sentences, but also some intelligence behind the sentences too. You as a human had another foundation of intelligence to lean on, your native language, and an understanding of the world outside of learning the Italian language. If you had no other human interaction, would you have learned any language? That's the more the situation of these AI algorithms.
    - spdionis 11y ago
dwiel 11y ago
There are some recent examples where people have trained a collection of large nets which are then used to teach a smaller net. The smaller net can learn more quickly and finally achieves better performance than the large collection. The methods involve providing more detailed feedback at each example. With most training data used now, we give a 0 or 1, does this example belong to this class. In the teacher networks, they were able to teach with more subtly: this is definitely not a car, it is very lizard like and a little snake like.
- kimbly 11y ago
  Do you have a reference?
  - sinwave 11y ago
    Geoffrey Hinton gave results to this effect in a talk about "Dark Knowledge" [1]. Haven't seen any of these results published, though. I think he mentions something in the talk about NIPS rejecting the paper. [1] - https://www.youtube.com/watch?v=EK61htlw8hY https://www.youtube.com/watch?v=EK61htlw8hY
frozenport 11y ago
Umm. When you train you train against a cost function...
- digikata 11y ago
  Well not exactly, if we had a cost function as intelligent as a human, the cost function to train the AI, would be an AI. Or maybe I'm completely off base here...

jpatokal 11y ago

The TL;DR appears to be "no", interaction is necessary: https://en.wikipedia.org/wiki/Language_acquisition#General_approaches https://en.wikipedia.org/wiki/Language_acquisition#General_a... Although for obvious reasons this is very hard to study experimentally: https://en.wikipedia.org/wiki/Language_deprivation_experiments https://en.wikipedia.org/wiki/Language_deprivation_experimen...

eli_gottlieb 11y ago
Proper language usage probably involves causal modelling, in which case intervention experiments are one of the only known ways to learn correctly.

solve 11y ago

I'm convinced that voice pitch, syllable meter, hand gestures, facial gestures, and other forms of non-word emphasis are also crucial to figuring out basic phrase chunking and word types, with minimal interaction, early on. Seems it would be far harder to infer the basic initial structure from just plain text.

tzs 11y ago

> Could we expect a child to learn to speak if they only heard recordings of existing speech with otherwise no human interaction/feedback - correcting them or offering customized and contextual new bits of information? I once asked a similar question on some online forum [1] where many linguists hung out. My question was if an English-only speaking household left a general interest Spanish language TV station on most of the time when they weren't actively using the TV to watch something, so that their child received a very large exposure to Spanish language programming (news, sports, soap operas, sitcoms, movies, etc) from birth onward, would the child naturally learn Spanish? I don't recall for sure what the linguists who responded said, but I think they all said the child would not learn Spanish from this. [1] I have no recollection of where this was.

geomark 11y ago
I've seen something like this in action with young kids who are given a tablet and stumble upon cartoons they like on YouTube, but in a different language. After they watch a few cartoons, YouTube's recommender system keeps offering them more cartoons in that language. And it isn't long before they start spouting words and phrases in that language.
dzhiurgis 11y ago
In Baltic states, majority of TV content either comes in native German or is US shows translated to Russian. I've picked up quite a bit of Russian by watching Discovery channel this way.
spdionis 11y ago
When I was little (5-7 years old) I had quite a few anime videos and magazines in italian sent to me by my parents who were abroad. Where I lived no one knew a single word of italian. I often watched and rewatched those videos and read those magazines without any other external input. I can tell you that doing that I easily learned the language. When I was 8 years old I also left for Italy and in two weeks of time I already started speaking fluently, albeit with a few mistakes. If the child will actually watch the Spanish TV he will learn the language. EDIT: Even now I often learn new japanese words (and remember them) just by watching animes. The difference is that now I have english subtitles but back then I had no subtitles, only the images to help me understand the meaning.
- digikata 11y ago
  But this is a little different, in the AI we want the ability to form syntactically correct sentences, but also some intelligence behind the sentences too. You as a human had another foundation of intelligence to lean on, your native language, and an understanding of the world outside of learning the Italian language. If you had no other human interaction, would you have learned any language? That's the more the situation of these AI algorithms.
  - spdionis 11y ago

dwiel 11y ago

There are some recent examples where people have trained a collection of large nets which are then used to teach a smaller net. The smaller net can learn more quickly and finally achieves better performance than the large collection. The methods involve providing more detailed feedback at each example. With most training data used now, we give a 0 or 1, does this example belong to this class. In the teacher networks, they were able to teach with more subtly: this is definitely not a car, it is very lizard like and a little snake like.

kimbly 11y ago
Do you have a reference?
- sinwave 11y ago
  Geoffrey Hinton gave results to this effect in a talk about "Dark Knowledge" [1]. Haven't seen any of these results published, though. I think he mentions something in the talk about NIPS rejecting the paper. [1] - https://www.youtube.com/watch?v=EK61htlw8hY https://www.youtube.com/watch?v=EK61htlw8hY

frozenport 11y ago

Umm. When you train you train against a cost function...

digikata 11y ago
Well not exactly, if we had a cost function as intelligent as a human, the cost function to train the AI, would be an AI. Or maybe I'm completely off base here...