5 ms·

I like the design of your website! What do you mean when you say words are disentangled, standalone concepts? I see words as being very much related to each ot

by enderm 5y ago

I like the design of your website! What do you mean when you say words are disentangled, standalone concepts? I see words as being very much related to each other. I assume I may be misinterpreting what you mean by "disentangled, standalone concepts”. Barbara Tversky's research seems to contradict linguistic relativism. I definitely don’t think language is the foundation of cognition.

ericjang 5y ago
Thanks! Words are considered a "discrete unit of meaning", i.e. 3/4 of a word doesn't really mean much. So words like "red" and "grass" are "standalone" in the sense that the mean something by themselves. I agree that words are very much related to each other, in the sense that you can combine them. I was trying to draw a connection that the "disentangled representations" ML folks often talk about are but a special few-word case of grammars for combining distinct concept.
- enderm 5y ago
  I can see how this could work in English. I’m not sure if there are other languages in which 3/4 of a word carries more meaning. (I’m a primary English speaker, so this concern could be unfounded.)
  - canjobear 5y ago
    All languages have something analogous to words in this way, although it can be hard to know where to draw the boundaries sometimes. Technically the smallest indivisible unit that bears meaning is the morpheme, not the word. For example the word “cats” in English consists of two morphemes, cat+s. The first morpheme can stand on its own as a word, but the second can’t.
    - solarmist 5y ago
      I agree, but I think the trickier part is that the semantics of words are even blurrier/more ambiguous than the syntax.
      - canjobear 5y ago
        Yeah, hence the turn away from dictionary definitions and things like WordNet towards continuous distributional vector representations in NLP. I don’t think you could really give an uncontroversial symbolic definition for any natural word.
  - PeterisP 5y ago
    In many languages you have literally 3/4 of the word carry the meaning of the actual word and the remaining 1/4 sounds or letters devoted to grammatical markers for the gender/case/number/etc. Using a classic Latin example from Monty Python, Romani ite domum / Romanes eunt domus; the "Roman" part of of Romanes/Romani actually carries very much meaning and the -es/-i has information that's largely orthogonal to that.
- solarmist 5y ago
  Unfortunately, words aren't that simple, but it's close. Prefixes, suffixes, in-fixes, endings, etc., all have discrete meaning as well. And going into Asian language, this is much more obvious. The discrete unit of meaning level is generally somewhere between a syllable and a word, with a few exceptions for shorter modifiers. Unfortunately, in linguistics, the concept of a "word" is only as well defined as "planet" was pre-pluto losing its status. Similarly when you look at riddles and crossword puzzle clues the idea of words being discrete also falls apart. Words, very much like variables in algebra only have meaning in relation to the other pieces of the context they are attached to. While the mechanics (all the pieces of language, syntax and semantics are not discretizable. Just talk to anyone working on a dictionary.) you talk about don't seem to hold, I do think the idea you're talking about does hold.
  - ericjang 5y ago
    Fair enough, I agree that if we really examine the comment "word as a discrete unit of meaning", the edge cases start to accumulate and the semantics rapidly break down. But barring things like prefixes/suffixes/modifiers/composite word characters in traditional Chinese, words are fairly discrete and generally regarded as the primary layer for expressing singular units of "meaning"
    - solarmist 5y ago
      They are, but only because we don't have better language to express them. Similar to a lot of the problems with Chomsky's works the composability of language is only a subset of the whole breadth of what is expressable in a given language. Or in other words, I believe the surface area of "edge cases" has a similar surface area as the rest of the language. The difference being they aren't invoked nearly as often because they require more effort and creativity. Just look at the rise of words like "hangry". There are types of mashups that show up in creative uses of language that defy nearly any rule for any language you can come up with. In many languages, if you choose any of those supposed rules you can probably construct an algorithm to generate odd, but understandable words that defy that rule.
- Mezzie 5y ago
  Actually, the discrete unit of meaning, linguistically, is the morpheme. It's a small difference, but it matters. Some words are morphemes, but not all, and not all morphemes are words. Language, man. It's weird.

by enderm 5y ago

ericjang 5y ago

Thanks! Words are considered a "discrete unit of meaning", i.e. 3/4 of a word doesn't really mean much. So words like "red" and "grass" are "standalone" in the sense that the mean something by themselves. I agree that words are very much related to each other, in the sense that you can combine them. I was trying to draw a connection that the "disentangled representations" ML folks often talk about are but a special few-word case of grammars for combining distinct concept.

enderm 5y ago
I can see how this could work in English. I’m not sure if there are other languages in which 3/4 of a word carries more meaning. (I’m a primary English speaker, so this concern could be unfounded.)
- canjobear 5y ago
  All languages have something analogous to words in this way, although it can be hard to know where to draw the boundaries sometimes. Technically the smallest indivisible unit that bears meaning is the morpheme, not the word. For example the word “cats” in English consists of two morphemes, cat+s. The first morpheme can stand on its own as a word, but the second can’t.
  - solarmist 5y ago
    I agree, but I think the trickier part is that the semantics of words are even blurrier/more ambiguous than the syntax.
    - canjobear 5y ago
      Yeah, hence the turn away from dictionary definitions and things like WordNet towards continuous distributional vector representations in NLP. I don’t think you could really give an uncontroversial symbolic definition for any natural word.
- PeterisP 5y ago
  In many languages you have literally 3/4 of the word carry the meaning of the actual word and the remaining 1/4 sounds or letters devoted to grammatical markers for the gender/case/number/etc. Using a classic Latin example from Monty Python, Romani ite domum / Romanes eunt domus; the "Roman" part of of Romanes/Romani actually carries very much meaning and the -es/-i has information that's largely orthogonal to that.
solarmist 5y ago
Unfortunately, words aren't that simple, but it's close. Prefixes, suffixes, in-fixes, endings, etc., all have discrete meaning as well. And going into Asian language, this is much more obvious. The discrete unit of meaning level is generally somewhere between a syllable and a word, with a few exceptions for shorter modifiers. Unfortunately, in linguistics, the concept of a "word" is only as well defined as "planet" was pre-pluto losing its status. Similarly when you look at riddles and crossword puzzle clues the idea of words being discrete also falls apart. Words, very much like variables in algebra only have meaning in relation to the other pieces of the context they are attached to. While the mechanics (all the pieces of language, syntax and semantics are not discretizable. Just talk to anyone working on a dictionary.) you talk about don't seem to hold, I do think the idea you're talking about does hold.
- ericjang 5y ago
  Fair enough, I agree that if we really examine the comment "word as a discrete unit of meaning", the edge cases start to accumulate and the semantics rapidly break down. But barring things like prefixes/suffixes/modifiers/composite word characters in traditional Chinese, words are fairly discrete and generally regarded as the primary layer for expressing singular units of "meaning"
  - solarmist 5y ago
    They are, but only because we don't have better language to express them. Similar to a lot of the problems with Chomsky's works the composability of language is only a subset of the whole breadth of what is expressable in a given language. Or in other words, I believe the surface area of "edge cases" has a similar surface area as the rest of the language. The difference being they aren't invoked nearly as often because they require more effort and creativity. Just look at the rise of words like "hangry". There are types of mashups that show up in creative uses of language that defy nearly any rule for any language you can come up with. In many languages, if you choose any of those supposed rules you can probably construct an algorithm to generate odd, but understandable words that defy that rule.
Mezzie 5y ago
Actually, the discrete unit of meaning, linguistically, is the morpheme. It's a small difference, but it matters. Some words are morphemes, but not all, and not all morphemes are words. Language, man. It's weird.