4 ms·

I see this argument a lot. I have essentially zero ml experience, but isn't it fair to say that DNA indeed carries a similarly large amount of data? Also, it's

I see this argument a lot. I have essentially zero ml experience, but isn't it fair to say that DNA indeed carries a similarly large amount of data? Also, it's not like any human is just born fully capable of inventing an entire language. It seemingly takes entire societies to pull this off. Granted, our ml isn't even close to human intelligence yet. But why is the assumption made that humans are "trained" with very little data? We've collected millions of years of data to accomplish what we have.

YeGoblynQueenne 6y ago
Humans seem to come into the world with very good quality biases that allow us to learn any human language but I don't think anyone has ever found DNA encoding information about, say, the English language. An I misunderstanding your comment? If so, I apologise- I'm not sure where DNA comes in. >> Also, it's not like any human is just born fully capable of inventing an entire language. It seemingly takes entire societies to pull this off. This is a complicated matter and as I'm not a linguist I don't know what theories there even exist. However, I think a human child is perfectly capable of inventing an entire language. Small groups of children are certainly capable of doing that in the early stages of their lives. For example, see Creole languages [1] which are the languages invented by the children of people who only have a pidgin as a common language. A pidgin is not a fully developed language and pidgins are used e.g. for communication between migrant communities with different origins and who do not have a common language, while a Creole is a fully developed natural language that is invented by the migrants' children, who have their parents' share pidgin as a maternal language. The children start with their parents' pidgin and develop it into a fully formed natural language with all the bells and whistles. It's quite a thing to read about really, have a look at the wikipedia article I link above it's very interesting. Also, as far as I understand it there's a lot that we don't know about how the first human languages were created. From some short readings, the current understanding is that at some point, humans must have become capable of producing natural language whereas before they weren't. The way things usually work the turning point for this was most likely some random mutation that affected a very few individuals, possibly no more than two. So the first natural language was probably invented not by an entire society but by a couple of people, probably a pair of siblings. The idea is that the ability to use natural language gave those very few individuals an important advantage that ensured its propagation through generations until our time. In any case that all goes to invention of a new language. Learning an existing language is quite another matter. Children who grow up to learn their maternal language don't have access to "millions of years of data". They certainly don't have access to a few billion examples of utterances in their maternal language, like the datasets that language models are trained on. That is to say, kids learn whatever language is their maternal language very, very quickly as they grow up and from very few examples. And they're capable of learning any of the few thousand natural languages, beyond English. For me at least this lends credence to an idea of an innate bias to learn a certain type of language, a type broad enough to cover every human language. To put a name on this idea, that's through and through Noam Chomsky's idea of an innate "Universal Grammar". See wikipedia about criticisms of UG and linguistic nativism in general [2]. ____________ [1] https://en.wikipedia.org/wiki/Creole_language https://en.wikipedia.org/wiki/Creole_language [2] https://en.wikipedia.org/wiki/Universal_grammar#Criticisms https://en.wikipedia.org/wiki/Universal_grammar#Criticisms

YeGoblynQueenne 6y ago

Humans seem to come into the world with very good quality biases that allow us to learn any human language but I don't think anyone has ever found DNA encoding information about, say, the English language. An I misunderstanding your comment? If so, I apologise- I'm not sure where DNA comes in. >> Also, it's not like any human is just born fully capable of inventing an entire language. It seemingly takes entire societies to pull this off. This is a complicated matter and as I'm not a linguist I don't know what theories there even exist. However, I think a human child is perfectly capable of inventing an entire language. Small groups of children are certainly capable of doing that in the early stages of their lives. For example, see Creole languages [1] which are the languages invented by the children of people who only have a pidgin as a common language. A pidgin is not a fully developed language and pidgins are used e.g. for communication between migrant communities with different origins and who do not have a common language, while a Creole is a fully developed natural language that is invented by the migrants' children, who have their parents' share pidgin as a maternal language. The children start with their parents' pidgin and develop it into a fully formed natural language with all the bells and whistles. It's quite a thing to read about really, have a look at the wikipedia article I link above it's very interesting. Also, as far as I understand it there's a lot that we don't know about how the first human languages were created. From some short readings, the current understanding is that at some point, humans must have become capable of producing natural language whereas before they weren't. The way things usually work the turning point for this was most likely some random mutation that affected a very few individuals, possibly no more than two. So the first natural language was probably invented not by an entire society but by a couple of people, probably a pair of siblings. The idea is that the ability to use natural language gave those very few individuals an important advantage that ensured its propagation through generations until our time. In any case that all goes to invention of a new language. Learning an existing language is quite another matter. Children who grow up to learn their maternal language don't have access to "millions of years of data". They certainly don't have access to a few billion examples of utterances in their maternal language, like the datasets that language models are trained on. That is to say, kids learn whatever language is their maternal language very, very quickly as they grow up and from very few examples. And they're capable of learning any of the few thousand natural languages, beyond English. For me at least this lends credence to an idea of an innate bias to learn a certain type of language, a type broad enough to cover every human language. To put a name on this idea, that's through and through Noam Chomsky's idea of an innate "Universal Grammar". See wikipedia about criticisms of UG and linguistic nativism in general [2]. ____________ [1] https://en.wikipedia.org/wiki/Creole_language https://en.wikipedia.org/wiki/Creole_language [2] https://en.wikipedia.org/wiki/Universal_grammar#Criticisms https://en.wikipedia.org/wiki/Universal_grammar#Criticisms