5 ms·

Human intelligence itself is shaped by our interaction with outputs. Our learning and understanding of the world are profoundly influenced by the language, beha

Human intelligence itself is shaped by our interaction with outputs. Our learning and understanding of the world are profoundly influenced by the language, behaviors, and cultural artifacts we observe. Think about the process of a child learning a language. The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of others around them. Over time, they develop a sophisticated understanding of language, not by direct instruction of underlying rules, but through pattern recognition and contextual inference from these outputs. Then that language itself, learned from outputs, becomes the cognitive apparatus that enables the child to imagine, to reason symbolically and abstractly. Humans bootstrap intelligence on top of language, which itself is learned by mimicking outputs. Moreover, the analogy to weather prediction or stock market analysis is somewhat misleading. Yes, these models benefit from input data (like air currents for weather, company fundamentals and CEO statements to the media for stocks). But these systems are fundamentally different from intelligence. Intelligence, whether artificial or human, is about the ability to learn, adapt, and generate novel responses in a broad range of scenarios, not just about predicting specific outcomes based on specific inputs.

wavemode 3y ago
> The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of others around them. I would argue that that learning is always contextualized by visual and spatial information about the real world (which is what our language is meant to describe). And (equally importantly) the child gets real-world feedback on their decisions - some decisions achieve their desired goal and some don't. Some statements cause certain responses from people, some actions have certain consequences. > Moreover, the analogy to weather prediction or stock market analysis is somewhat misleading. Yes, these models benefit from input data (like air currents for weather, company fundamentals and CEO statements to the media for stocks). But these systems are fundamentally different from intelligence. > Intelligence, whether artificial or human, is about the ability to learn, adapt, and generate novel responses in a broad range of scenarios, not just about predicting specific outcomes based on specific inputs. "Intelligence" is kind of a nebulous term. If all it means is the ability to learn, adapt and generate novel responses, then sure, I think we could call almost any neural network intelligent. But I would argue that we usually do have some expectation that an intelligent system can produce "specific outcomes based on specific inputs". We want to be able to train a worker and have them follow that training so they do their job correctly.
- sdenton4 3y ago
  Visual and spatial feedback aren't terribly difficult, though - we already have video-game training gyms, from Atari to GTA. If multimodality and feedback are the main barriers to full AGI, I expect we'll be there soon.
Jensson 3y ago
> Over time, they develop a sophisticated understanding of language, not by direct instruction of underlying rules, but through pattern recognition and contextual inference from these outputs. Kids learn through supervised learning. Children don't develop strong language skills without parents or other people to correct them when they use language incorrectly. We don't use supervised learning on LLMs. There is no way we can train an LLM by using human supervisors to rate every output. It works for humans since humans learn with so few examples, supervising a kids language learning doesn't take much effort, a single person can easily manage it while doing it for an LLM would consume the whole worlds workforce for many years.
- famouswaffles 3y ago
  >Kids learn through supervised learning. Children don't develop strong language skills without parents or other people to correct them when they use language incorrectly. Untrue. Many cultures don't much speak to their children and they turn out just fine. It's fairly evident Language learning is primarily unsupervised. https://www.scientificamerican.com/article/parents-in-a-remote-amazon-village-barely-talk-to-their-babies-mdash-and-the-kids-are-fine/ https://www.scientificamerican.com/article/parents-in-a-remo...
  - Jensson 3y ago
    That doesn't say that parents don't correct kids, just that they don't speak to their infants. The initial words are learnt that way, but I don't think you master language without anyone to correct you when you make mistakes.
    - GolDDranks 3y ago
      This is simply not true. I am very well read in language acquisition research, and no evidence supports that children would need corrections to get language right.
    - famouswaffles 3y ago
      If they are correcting their kids, they are doing so far less than many parents and it evidently doesn't matter. I'm honestly curious as to why you think it's so important. Fringe Amazon groups aside, the number of parents extremely laissez-faire with correction is astounding and they raise kids that use language perfectly well. Even with parents that aren't laissez-faire, they're correcting a handful of times a day tops which can't even begin to account for full language proficiency. Children need correction yes but in a supervised way? There's no indication of that.
advael 3y ago
> Then that language itself, learned from outputs, becomes the cognitive apparatus that enables the child to imagine, to reason symbolically and abstractly. Humans bootstrap intelligence on top of language, which itself is learned by mimicking outputs. This isn't an uncommon claim, but it's certainly far from settled science. Complex language is an impressive capability of human cognition. But I think the claim that cognition itself is a byproduct of language proves too much. Obviously the products of cognition that are easiest for us to understand are linguistic, because language is the primary standardized means by which we communicate. To me, this is just more legibility bias, though. There are plenty of cognitive processes that people find impossible to explain in words, but still nonetheless exist
ayakang31415 3y ago
> Human intelligence itself is shaped by our interaction with outputs. Our learning and understanding of the world are profoundly influenced by the language, behaviors, and cultural artifacts we observe. I always thought that language definitely shapes our understanding of the world, but at much more fundamental level, I believe language falls apart to teach us anything. For example, there are words in dictionary that should not be defined using other words (but dictionary still do, this sort of circular reasoning is something I always have problem with), but we just know them through our interactions with others and environments. If you try to teach a kid a concept of temperature "hot", you would not be able to do so unless the kid touches something really hot like boiling kettle to "feel" it and when mom comes along and say something like "do not touch it it is very hot". There are things we just cannot understand through language.
- timschmidt 3y ago
  People wiser than me have said for an age: the map is not the territory. Neither is the word the thing.
  - bugglebeetle 3y ago
    A humorous outcome of the AI craze may be Wittgenstein achieving the status of Gödel with regard to the “incompleteness” of language.
- red75prime 3y ago
  OK. How can we operationalize the definition of "understanding" you are talking about? That is which tests will allow us to know who understands "hot" and who does not?
- ordu 3y ago
  > For example, there are words in dictionary that should not be defined using other words (but dictionary still do, this sort of circular reasoning is something I always have problem with) You shouldn't. When you learn a foreign language dictionaries can be a great help and it doesn't matter if they use circular definitions. Well it can matter in a situation like Stanisław Lem described with his "sepulkas"[1], but even then the definitions were a red warning. Ijon Tichy just failed to understand it. > you would not be able to do so unless the kid touches something really hot like boiling kettle to "feel" it I learned English by reading books mostly. In most cases I had no easy access to a dictionary and inferred meanings of words by a context. In some cases I failed to infer meaning but nevertheless in each case I managed to get some idea about a word. I remember some surprises when I found a real meaning of a word and it was not exactly what I thought. Or even some ideas that felt new and inspiring for me before I connected them to ideas I learned long before that in my native language. I'm like an English LLM myself, because I never used English in a real world context, only to read texts and to write comments in English. To this very moment I cannot talk about some topics in Russian, because I do not know Russian words to talk about them. All this experience led me to doubt an idea that you can understand language only by connecting it to reality. I believe you can. Your language will be disconnected from reality and it can be a disaster probably if you try to apply it to reality, but you can talk a lot, participate in philosophical debates and it doesn't matter your understanding is different. It is like redness of red: do you see red as I do and does it matter? > There are things we just cannot understand through language. We cannot link our senses with language without burns. But it doesn't mean we cannot understand. For example, you can watch other people touching really hot boiling kettles. If you never touched anything hot you will not understand their pain, but you'll know they are in a pain and you'll know of a special nature of that pain. You can learn all the important intricacies of interaction with hot objects by just watching. Or by reading. Your understanding will be limited still, but in a lot of cases it doesn't matter. Though of course we coming very near to a debate what is understanding. Is it a human-centric definition, that boils down to "only a human can understand", or it is more relaxed and rely on a pragmatic idea, like does your understanding enables you to make right choices. If LLM doesn't have a body that can feel pain, it doesn't matter if LLM cannot know how pain feels. [1] https://en.wikipedia.org/wiki/Sepulka https://en.wikipedia.org/wiki/Sepulka

wavemode 3y ago

> The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of others around them. I would argue that that learning is always contextualized by visual and spatial information about the real world (which is what our language is meant to describe). And (equally importantly) the child gets real-world feedback on their decisions - some decisions achieve their desired goal and some don't. Some statements cause certain responses from people, some actions have certain consequences. > Moreover, the analogy to weather prediction or stock market analysis is somewhat misleading. Yes, these models benefit from input data (like air currents for weather, company fundamentals and CEO statements to the media for stocks). But these systems are fundamentally different from intelligence. > Intelligence, whether artificial or human, is about the ability to learn, adapt, and generate novel responses in a broad range of scenarios, not just about predicting specific outcomes based on specific inputs. "Intelligence" is kind of a nebulous term. If all it means is the ability to learn, adapt and generate novel responses, then sure, I think we could call almost any neural network intelligent. But I would argue that we usually do have some expectation that an intelligent system can produce "specific outcomes based on specific inputs". We want to be able to train a worker and have them follow that training so they do their job correctly.

sdenton4 3y ago
Visual and spatial feedback aren't terribly difficult, though - we already have video-game training gyms, from Atari to GTA. If multimodality and feedback are the main barriers to full AGI, I expect we'll be there soon.

Jensson 3y ago

> Over time, they develop a sophisticated understanding of language, not by direct instruction of underlying rules, but through pattern recognition and contextual inference from these outputs. Kids learn through supervised learning. Children don't develop strong language skills without parents or other people to correct them when they use language incorrectly. We don't use supervised learning on LLMs. There is no way we can train an LLM by using human supervisors to rate every output. It works for humans since humans learn with so few examples, supervising a kids language learning doesn't take much effort, a single person can easily manage it while doing it for an LLM would consume the whole worlds workforce for many years.

famouswaffles 3y ago
>Kids learn through supervised learning. Children don't develop strong language skills without parents or other people to correct them when they use language incorrectly. Untrue. Many cultures don't much speak to their children and they turn out just fine. It's fairly evident Language learning is primarily unsupervised. https://www.scientificamerican.com/article/parents-in-a-remote-amazon-village-barely-talk-to-their-babies-mdash-and-the-kids-are-fine/ https://www.scientificamerican.com/article/parents-in-a-remo...
- Jensson 3y ago
  That doesn't say that parents don't correct kids, just that they don't speak to their infants. The initial words are learnt that way, but I don't think you master language without anyone to correct you when you make mistakes.
  - GolDDranks 3y ago
    This is simply not true. I am very well read in language acquisition research, and no evidence supports that children would need corrections to get language right.
  - famouswaffles 3y ago
    If they are correcting their kids, they are doing so far less than many parents and it evidently doesn't matter. I'm honestly curious as to why you think it's so important. Fringe Amazon groups aside, the number of parents extremely laissez-faire with correction is astounding and they raise kids that use language perfectly well. Even with parents that aren't laissez-faire, they're correcting a handful of times a day tops which can't even begin to account for full language proficiency. Children need correction yes but in a supervised way? There's no indication of that.

advael 3y ago

> Then that language itself, learned from outputs, becomes the cognitive apparatus that enables the child to imagine, to reason symbolically and abstractly. Humans bootstrap intelligence on top of language, which itself is learned by mimicking outputs. This isn't an uncommon claim, but it's certainly far from settled science. Complex language is an impressive capability of human cognition. But I think the claim that cognition itself is a byproduct of language proves too much. Obviously the products of cognition that are easiest for us to understand are linguistic, because language is the primary standardized means by which we communicate. To me, this is just more legibility bias, though. There are plenty of cognitive processes that people find impossible to explain in words, but still nonetheless exist

ayakang31415 3y ago

> Human intelligence itself is shaped by our interaction with outputs. Our learning and understanding of the world are profoundly influenced by the language, behaviors, and cultural artifacts we observe. I always thought that language definitely shapes our understanding of the world, but at much more fundamental level, I believe language falls apart to teach us anything. For example, there are words in dictionary that should not be defined using other words (but dictionary still do, this sort of circular reasoning is something I always have problem with), but we just know them through our interactions with others and environments. If you try to teach a kid a concept of temperature "hot", you would not be able to do so unless the kid touches something really hot like boiling kettle to "feel" it and when mom comes along and say something like "do not touch it it is very hot". There are things we just cannot understand through language.

timschmidt 3y ago
People wiser than me have said for an age: the map is not the territory. Neither is the word the thing.
- bugglebeetle 3y ago
  A humorous outcome of the AI craze may be Wittgenstein achieving the status of Gödel with regard to the “incompleteness” of language.
red75prime 3y ago
OK. How can we operationalize the definition of "understanding" you are talking about? That is which tests will allow us to know who understands "hot" and who does not?
ordu 3y ago
> For example, there are words in dictionary that should not be defined using other words (but dictionary still do, this sort of circular reasoning is something I always have problem with) You shouldn't. When you learn a foreign language dictionaries can be a great help and it doesn't matter if they use circular definitions. Well it can matter in a situation like Stanisław Lem described with his "sepulkas"[1], but even then the definitions were a red warning. Ijon Tichy just failed to understand it. > you would not be able to do so unless the kid touches something really hot like boiling kettle to "feel" it I learned English by reading books mostly. In most cases I had no easy access to a dictionary and inferred meanings of words by a context. In some cases I failed to infer meaning but nevertheless in each case I managed to get some idea about a word. I remember some surprises when I found a real meaning of a word and it was not exactly what I thought. Or even some ideas that felt new and inspiring for me before I connected them to ideas I learned long before that in my native language. I'm like an English LLM myself, because I never used English in a real world context, only to read texts and to write comments in English. To this very moment I cannot talk about some topics in Russian, because I do not know Russian words to talk about them. All this experience led me to doubt an idea that you can understand language only by connecting it to reality. I believe you can. Your language will be disconnected from reality and it can be a disaster probably if you try to apply it to reality, but you can talk a lot, participate in philosophical debates and it doesn't matter your understanding is different. It is like redness of red: do you see red as I do and does it matter? > There are things we just cannot understand through language. We cannot link our senses with language without burns. But it doesn't mean we cannot understand. For example, you can watch other people touching really hot boiling kettles. If you never touched anything hot you will not understand their pain, but you'll know they are in a pain and you'll know of a special nature of that pain. You can learn all the important intricacies of interaction with hot objects by just watching. Or by reading. Your understanding will be limited still, but in a lot of cases it doesn't matter. Though of course we coming very near to a debate what is understanding. Is it a human-centric definition, that boils down to "only a human can understand", or it is more relaxed and rely on a pragmatic idea, like does your understanding enables you to make right choices. If LLM doesn't have a body that can feel pain, it doesn't matter if LLM cannot know how pain feels. [1] https://en.wikipedia.org/wiki/Sepulka https://en.wikipedia.org/wiki/Sepulka