6 ms·

To Understand Language Is to Understand Generalization

91 points by ericjang 5y ago

iamgopal 5y ago
Does reverse is true ? To understand generalisation is to understand language ?
enderm 5y ago
I like the design of your website! What do you mean when you say words are disentangled, standalone concepts? I see words as being very much related to each other. I assume I may be misinterpreting what you mean by "disentangled, standalone concepts”. Barbara Tversky's research seems to contradict linguistic relativism. I definitely don’t think language is the foundation of cognition.
- ericjang 5y ago
  Thanks! Words are considered a "discrete unit of meaning", i.e. 3/4 of a word doesn't really mean much. So words like "red" and "grass" are "standalone" in the sense that the mean something by themselves. I agree that words are very much related to each other, in the sense that you can combine them. I was trying to draw a connection that the "disentangled representations" ML folks often talk about are but a special few-word case of grammars for combining distinct concept.
  - enderm 5y ago
    I can see how this could work in English. I’m not sure if there are other languages in which 3/4 of a word carries more meaning. (I’m a primary English speaker, so this concern could be unfounded.)
    - canjobear 5y ago
      All languages have something analogous to words in this way, although it can be hard to know where to draw the boundaries sometimes. Technically the smallest indivisible unit that bears meaning is the morpheme, not the word. For example the word “cats” in English consists of two morphemes, cat+s. The first morpheme can stand on its own as a word, but the second can’t.
      - solarmist 5y ago
        I agree, but I think the trickier part is that the semantics of words are even blurrier/more ambiguous than the syntax.
        canjobear 5y ago
        Yeah, hence the turn away from dictionary definitions and things like WordNet towards continuous distributional vector representations in NLP. I don’t think you could really give an uncontroversial symbolic definition for any natural word.
    - PeterisP 5y ago
      In many languages you have literally 3/4 of the word carry the meaning of the actual word and the remaining 1/4 sounds or letters devoted to grammatical markers for the gender/case/number/etc. Using a classic Latin example from Monty Python, Romani ite domum / Romanes eunt domus; the "Roman" part of of Romanes/Romani actually carries very much meaning and the -es/-i has information that's largely orthogonal to that.
  - solarmist 5y ago
    Unfortunately, words aren't that simple, but it's close. Prefixes, suffixes, in-fixes, endings, etc., all have discrete meaning as well. And going into Asian language, this is much more obvious. The discrete unit of meaning level is generally somewhere between a syllable and a word, with a few exceptions for shorter modifiers. Unfortunately, in linguistics, the concept of a "word" is only as well defined as "planet" was pre-pluto losing its status. Similarly when you look at riddles and crossword puzzle clues the idea of words being discrete also falls apart. Words, very much like variables in algebra only have meaning in relation to the other pieces of the context they are attached to. While the mechanics (all the pieces of language, syntax and semantics are not discretizable. Just talk to anyone working on a dictionary.) you talk about don't seem to hold, I do think the idea you're talking about does hold.
    - ericjang 5y ago
      Fair enough, I agree that if we really examine the comment "word as a discrete unit of meaning", the edge cases start to accumulate and the semantics rapidly break down. But barring things like prefixes/suffixes/modifiers/composite word characters in traditional Chinese, words are fairly discrete and generally regarded as the primary layer for expressing singular units of "meaning"
      - solarmist 5y ago
        They are, but only because we don't have better language to express them. Similar to a lot of the problems with Chomsky's works the composability of language is only a subset of the whole breadth of what is expressable in a given language. Or in other words, I believe the surface area of "edge cases" has a similar surface area as the rest of the language. The difference being they aren't invoked nearly as often because they require more effort and creativity. Just look at the rise of words like "hangry". There are types of mashups that show up in creative uses of language that defy nearly any rule for any language you can come up with. In many languages, if you choose any of those supposed rules you can probably construct an algorithm to generate odd, but understandable words that defy that rule.
  - Mezzie 5y ago
    Actually, the discrete unit of meaning, linguistically, is the morpheme. It's a small difference, but it matters. Some words are morphemes, but not all, and not all morphemes are words. Language, man. It's weird.
solarmist 5y ago
This is a neat idea, but I think it's missing a large and important area for generalization, and that's the process of seeking and exploring exceptions or counter-examples (see my other comments for examples). Language defines things through subtraction, inversion, comparison, and contrast as much as construction and straightforward language. Engineering and computer science rely too heavily on induction, but deduction and other non-linear processes are largely missing from these kinds of analyses/approaches. And until they are accounted for I don't think we'll reach any kind of true approach to generalization.
ncmncm 5y ago
Not to understand generalization, therefore, is not to understand language. Q E D.
gsjbjt 5y ago
Nice post! I work on NLP and I think a lot of ideas in this post resonate with what I find exciting about working on the intersection of language + the real world: large text datasets as sources of abundant prior knowledge about the world, structure of language ~ structure of concepts that matter to humans, etc. I feel like the bottleneck is getting access to paired (language, other modality) data though (if your other modality isn't images). i.e. "bolt on generalization" is an intuitively appealing concept, but then it reduces to the hard problem of "how do I learn to ground language to e.g. my robot action space?" I haven't seen a robotics + language paper that actually grapples with the grounding problem / tries to think about how to scale the data collection process for language-conditioned robotics beyond annotating your own dataset as a proof-of-concept. Unlike language modeling / CLIP-type pretraining, it seems (fundamentally?) more difficult to find natural sources of supervision of (language, action). I'd be curious about your thoughts on this! > When it comes to combining natural language with robots, the obvious take is to use it as an input-output modality for human-robot interaction. The robot would understand human language inputs and potentially converse with the human. But if you accept that “generalization is language”, then language models have a far bigger role to play than just being the “UX layer for robots”. You should check out Jacob Andreas's work, if you haven't seen it already - esp. his stuff on learning from latent language (https://arxiv.org/abs/1711.00482 https://arxiv.org/abs/1711.00482).
- ericjang 5y ago
  My hope is that sufficiently rich language models obviate the need for a lot of robot-language grounding data. LfP (https://learning-from-play.github.io/ https://learning-from-play.github.io/) was a work that inspired me a lot. They relabel a few hours of open-ended demonstrations (humans instructed to play with anything in the environment) with a lot of hindsight language descriptions, and show some degree of general capability acquired through this richer language. You can describe the same action with a lot of different descriptions, e.g. "pick up the leftmost object unless it is a cup" could also be relabeled as "pick up an apple". That being said, the LfP paper stops short of testing whether we can improve robotics solely by only scaling language - a confounding factor and central to their narrative was the role of "open-ended play data". We do need some paired data to ground (language, robot-specific sensor/actuator modalities), but perhaps we can scale everything else with language only data. Thanks to the pointer on the Andreas paper! This is indeed quite relevant to the spirit of what I'm arguing for, though I prefer the implementation realized by the Lu et al '21 paper.
motohagiography 5y ago
An AI could be said to understand language if it used language as one of a selection of tools to operate on itself, a peer or other being, or its environment. The idea of "meaning = co-ocurrance" overlooks things like need, cause, and effect that appear when language is used as a tool to operate on its environment. Most of what I read about ML and AI is about creating these monolithic models that treat networks and clusters of neurons as a single entity, but that would be like treating a species of individuals with lifecycles as a single entity. The comment in the article about how GPT models are like a shadow compared to a 3D world suggests the bottleneck to evolving them is really us, as we're trying to make just one that emmulates many of us, instead of letting one loose on the internet to divide and proliferate to evolve millions where the best few will be exponentially better. Right now we're building expert systems that are individual specimens without an ecosystem. There isn't yet a botnet of GPT nodes compromising machines and harvesting compute for training and evolving through participating in forums, but then again how would I know? (There's nothing worse than failing a modern catchpa and having a flash of existential dread at the stark possibility I may have indeed been a robot all along. Now I do them at random just to be sure.)
- visarga 5y ago
  > instead of letting one loose on the internet to divide and proliferate to evolve millions where the best few will be exponentially better I've said this before, what current AI agents lack is a dick (& pussy). If they had a dick they could have an internal goal to motivate their evolution, a goal not dependent on us anymore. The battlefield of self replication vs death is the great school of evolution, where humanity is currently the top student. AI only sent the likes of AlphaGo to the school.
  - verisimi 5y ago
    You can program them to have a dick (or pussy). Program an AI to get the highest score in a game etc, and it will be more incessant than a teenager - it won't stop ever. What AI agents lack is animation - they are inanimate, they are software; they are machines. And will always be so. We can (and do) labour under the impression of our senses that all there is in reality is physical matter. This is an erroneous assumption imo, but if that is your bedrock, you will struggle to understand why the damn machines can't do what we want. You can be unhappy about this but it won't change reality. It is true metaphysics is hard to discern perhaps (by definition) but it won't change the reality that metaphysics is a genuine element of the human existence. In fact, its the most important part of human existence - we don't feel to be automatons after all, even if we make a pretence of it sometimes. The best we will do with machines is to create a simulation of the human experience, one that might pass the Turing test even. And even then, despite all indications and evidence, the machine will not be animated by spirit.
Gimpei 5y ago
What are your thoughts on the externalism of Putnam and Kripke, i.e. that meanings aren't just defined by use, but that they are also determined by objects themselves? It feels like that puts a crimp in meaning = co-occurance, but maybe not?
- solarmist 5y ago
  I agree. Or put another way a set (or sets) of concrete examples grounds every abstract idea (including words as abstract objects). And it's turtles all the way down (or up depending).
synquid 5y ago
This seems very similar to the research program led by the late Patrick Henry Winton: https://groups.csail.mit.edu/genesis/index.html https://groups.csail.mit.edu/genesis/index.html Besides, I wish that causality had been mentioned more than once in passing. Due to the existence of the ladder of causality, many important queries cannot be answered by mere observation, or even by intervention; such queries require counterfactual reasoning, and structural causal models generalize because they describe something that is very invariant in the world.

91 points by ericjang 5y ago

Does reverse is true ? To understand generalisation is to understand language ?

I like the design of your website! What do you mean when you say words are disentangled, standalone concepts? I see words as being very much related to each other. I assume I may be misinterpreting what you mean by "disentangled, standalone concepts”. Barbara Tversky's research seems to contradict linguistic relativism. I definitely don’t think language is the foundation of cognition.

ericjang 5y ago
Thanks! Words are considered a "discrete unit of meaning", i.e. 3/4 of a word doesn't really mean much. So words like "red" and "grass" are "standalone" in the sense that the mean something by themselves. I agree that words are very much related to each other, in the sense that you can combine them. I was trying to draw a connection that the "disentangled representations" ML folks often talk about are but a special few-word case of grammars for combining distinct concept.
- enderm 5y ago
  I can see how this could work in English. I’m not sure if there are other languages in which 3/4 of a word carries more meaning. (I’m a primary English speaker, so this concern could be unfounded.)
  - canjobear 5y ago
    All languages have something analogous to words in this way, although it can be hard to know where to draw the boundaries sometimes. Technically the smallest indivisible unit that bears meaning is the morpheme, not the word. For example the word “cats” in English consists of two morphemes, cat+s. The first morpheme can stand on its own as a word, but the second can’t.
    - solarmist 5y ago
      I agree, but I think the trickier part is that the semantics of words are even blurrier/more ambiguous than the syntax.
      - canjobear 5y ago
        Yeah, hence the turn away from dictionary definitions and things like WordNet towards continuous distributional vector representations in NLP. I don’t think you could really give an uncontroversial symbolic definition for any natural word.
  - PeterisP 5y ago
    In many languages you have literally 3/4 of the word carry the meaning of the actual word and the remaining 1/4 sounds or letters devoted to grammatical markers for the gender/case/number/etc. Using a classic Latin example from Monty Python, Romani ite domum / Romanes eunt domus; the "Roman" part of of Romanes/Romani actually carries very much meaning and the -es/-i has information that's largely orthogonal to that.
- solarmist 5y ago
  Unfortunately, words aren't that simple, but it's close. Prefixes, suffixes, in-fixes, endings, etc., all have discrete meaning as well. And going into Asian language, this is much more obvious. The discrete unit of meaning level is generally somewhere between a syllable and a word, with a few exceptions for shorter modifiers. Unfortunately, in linguistics, the concept of a "word" is only as well defined as "planet" was pre-pluto losing its status. Similarly when you look at riddles and crossword puzzle clues the idea of words being discrete also falls apart. Words, very much like variables in algebra only have meaning in relation to the other pieces of the context they are attached to. While the mechanics (all the pieces of language, syntax and semantics are not discretizable. Just talk to anyone working on a dictionary.) you talk about don't seem to hold, I do think the idea you're talking about does hold.
  - ericjang 5y ago
    Fair enough, I agree that if we really examine the comment "word as a discrete unit of meaning", the edge cases start to accumulate and the semantics rapidly break down. But barring things like prefixes/suffixes/modifiers/composite word characters in traditional Chinese, words are fairly discrete and generally regarded as the primary layer for expressing singular units of "meaning"
    - solarmist 5y ago
      They are, but only because we don't have better language to express them. Similar to a lot of the problems with Chomsky's works the composability of language is only a subset of the whole breadth of what is expressable in a given language. Or in other words, I believe the surface area of "edge cases" has a similar surface area as the rest of the language. The difference being they aren't invoked nearly as often because they require more effort and creativity. Just look at the rise of words like "hangry". There are types of mashups that show up in creative uses of language that defy nearly any rule for any language you can come up with. In many languages, if you choose any of those supposed rules you can probably construct an algorithm to generate odd, but understandable words that defy that rule.
- Mezzie 5y ago
  Actually, the discrete unit of meaning, linguistically, is the morpheme. It's a small difference, but it matters. Some words are morphemes, but not all, and not all morphemes are words. Language, man. It's weird.

solarmist 5y ago

This is a neat idea, but I think it's missing a large and important area for generalization, and that's the process of seeking and exploring exceptions or counter-examples (see my other comments for examples). Language defines things through subtraction, inversion, comparison, and contrast as much as construction and straightforward language. Engineering and computer science rely too heavily on induction, but deduction and other non-linear processes are largely missing from these kinds of analyses/approaches. And until they are accounted for I don't think we'll reach any kind of true approach to generalization.

ncmncm 5y ago

Not to understand generalization, therefore, is not to understand language. Q E D.

gsjbjt 5y ago

Nice post! I work on NLP and I think a lot of ideas in this post resonate with what I find exciting about working on the intersection of language + the real world: large text datasets as sources of abundant prior knowledge about the world, structure of language ~ structure of concepts that matter to humans, etc. I feel like the bottleneck is getting access to paired (language, other modality) data though (if your other modality isn't images). i.e. "bolt on generalization" is an intuitively appealing concept, but then it reduces to the hard problem of "how do I learn to ground language to e.g. my robot action space?" I haven't seen a robotics + language paper that actually grapples with the grounding problem / tries to think about how to scale the data collection process for language-conditioned robotics beyond annotating your own dataset as a proof-of-concept. Unlike language modeling / CLIP-type pretraining, it seems (fundamentally?) more difficult to find natural sources of supervision of (language, action). I'd be curious about your thoughts on this! > When it comes to combining natural language with robots, the obvious take is to use it as an input-output modality for human-robot interaction. The robot would understand human language inputs and potentially converse with the human. But if you accept that “generalization is language”, then language models have a far bigger role to play than just being the “UX layer for robots”. You should check out Jacob Andreas's work, if you haven't seen it already - esp. his stuff on learning from latent language (https://arxiv.org/abs/1711.00482 https://arxiv.org/abs/1711.00482).

ericjang 5y ago
My hope is that sufficiently rich language models obviate the need for a lot of robot-language grounding data. LfP (https://learning-from-play.github.io/ https://learning-from-play.github.io/) was a work that inspired me a lot. They relabel a few hours of open-ended demonstrations (humans instructed to play with anything in the environment) with a lot of hindsight language descriptions, and show some degree of general capability acquired through this richer language. You can describe the same action with a lot of different descriptions, e.g. "pick up the leftmost object unless it is a cup" could also be relabeled as "pick up an apple". That being said, the LfP paper stops short of testing whether we can improve robotics solely by only scaling language - a confounding factor and central to their narrative was the role of "open-ended play data". We do need some paired data to ground (language, robot-specific sensor/actuator modalities), but perhaps we can scale everything else with language only data. Thanks to the pointer on the Andreas paper! This is indeed quite relevant to the spirit of what I'm arguing for, though I prefer the implementation realized by the Lu et al '21 paper.

motohagiography 5y ago

An AI could be said to understand language if it used language as one of a selection of tools to operate on itself, a peer or other being, or its environment. The idea of "meaning = co-ocurrance" overlooks things like need, cause, and effect that appear when language is used as a tool to operate on its environment. Most of what I read about ML and AI is about creating these monolithic models that treat networks and clusters of neurons as a single entity, but that would be like treating a species of individuals with lifecycles as a single entity. The comment in the article about how GPT models are like a shadow compared to a 3D world suggests the bottleneck to evolving them is really us, as we're trying to make just one that emmulates many of us, instead of letting one loose on the internet to divide and proliferate to evolve millions where the best few will be exponentially better. Right now we're building expert systems that are individual specimens without an ecosystem. There isn't yet a botnet of GPT nodes compromising machines and harvesting compute for training and evolving through participating in forums, but then again how would I know? (There's nothing worse than failing a modern catchpa and having a flash of existential dread at the stark possibility I may have indeed been a robot all along. Now I do them at random just to be sure.)

visarga 5y ago
> instead of letting one loose on the internet to divide and proliferate to evolve millions where the best few will be exponentially better I've said this before, what current AI agents lack is a dick (& pussy). If they had a dick they could have an internal goal to motivate their evolution, a goal not dependent on us anymore. The battlefield of self replication vs death is the great school of evolution, where humanity is currently the top student. AI only sent the likes of AlphaGo to the school.
- verisimi 5y ago
  You can program them to have a dick (or pussy). Program an AI to get the highest score in a game etc, and it will be more incessant than a teenager - it won't stop ever. What AI agents lack is animation - they are inanimate, they are software; they are machines. And will always be so. We can (and do) labour under the impression of our senses that all there is in reality is physical matter. This is an erroneous assumption imo, but if that is your bedrock, you will struggle to understand why the damn machines can't do what we want. You can be unhappy about this but it won't change reality. It is true metaphysics is hard to discern perhaps (by definition) but it won't change the reality that metaphysics is a genuine element of the human existence. In fact, its the most important part of human existence - we don't feel to be automatons after all, even if we make a pretence of it sometimes. The best we will do with machines is to create a simulation of the human experience, one that might pass the Turing test even. And even then, despite all indications and evidence, the machine will not be animated by spirit.

Gimpei 5y ago

What are your thoughts on the externalism of Putnam and Kripke, i.e. that meanings aren't just defined by use, but that they are also determined by objects themselves? It feels like that puts a crimp in meaning = co-occurance, but maybe not?

solarmist 5y ago
I agree. Or put another way a set (or sets) of concrete examples grounds every abstract idea (including words as abstract objects). And it's turtles all the way down (or up depending).

synquid 5y ago

This seems very similar to the research program led by the late Patrick Henry Winton: https://groups.csail.mit.edu/genesis/index.html https://groups.csail.mit.edu/genesis/index.html Besides, I wish that causality had been mentioned more than once in passing. Due to the existence of the ladder of causality, many important queries cannot be answered by mere observation, or even by intervention; such queries require counterfactual reasoning, and structural causal models generalize because they describe something that is very invariant in the world.