3 ms·

> Modern language models absolutely make both sentiment and semantic distinctions They only appear to because of co-occurrence with pre-given sentiment terms.

> Modern language models absolutely make both sentiment and semantic distinctions They only appear to because of co-occurrence with pre-given sentiment terms. A conceptual relation between concept A and concept B isn't statistical. Eg., "Minorities" oppose "Racism" -- not because of any statistical occurrence of any of these terms, nor because of any statistical association at all. P(Minority|Racism, Oppose) and P(Minority| Racism, W forall words) have no baring on the truth of the proposition. It is true that in a large enough body of text if we took the powerset of all words and computing relative frequencies (ie., conditional probabilities on all possible co-occurances) we would find that "Minorities" oppose "Racism" more than, eg., "smell" it. But that fact isnt sufficient to make "semantic distinctions" -- because the proposition isn't true in virtue of that frequency. NLP systems have no means of generating text other than what is justified by those frequencies. This leads to trivial failures, such as facts in the world changing which invalidates those historical frequency-relationships. But also to absolutely fundamental failures such as the text generated itself being meaningless: the system has nothing it wishes to express, because there is no world it is in to express anything about. All sentences are just justified by previous sentences, not eg., by that there is -- right now -- a sunny day. When I say, "do you like the clothes I'm wearing?" i am not generating text justified by past frequencies. I am talking /about/ the clothes I am wearing, and that is what my words /mean/.

jmoss20 5y ago
> But that fact isnt sufficient to make "semantic distinctions" -- because the proposition isn't true in virtue of that frequency. The trick here is that language models are (currently!) demonstrating you /can/ get most of the way to semantic distinctions just by analyzing symbol-level statistics. Whether you can get all the way is an open question. I agree with you that "movie theaters" don't "sell candy" /because/ of some statistical artifact in large bodies of text. Movie theaters sell candy because people want to eat candy when they watch movies (and are willing to pay for it, etc.). But this wraps back around: the statistical artifacts happen to exist in large bodies of text because it is true. So, with enough text, and the right kind of analysis, you can tease the semantics back out. The power in language models is not that they "understand" text "the right way", from first principles, with a symbolic language model. The power is that they don't have to get most of the way there. Perhaps they'll get all the way there! And if they do, what then? Are we so sure that we don't do the same thing?
- nonameiguess 5y ago
  It's an open question, but I certainly suspect the reason humans are able to do this is we can synthesize knowledge from other sources and not have to rely solely upon learning from text. We've been to movie theaters and experienced buying candy there, which adds a great deal to our understanding of sentence containing the associated words without needing to read a hundred million sentences about movies and candy and rely only upon statistical patterns in the text to understand it.

jmoss20 5y ago

> But that fact isnt sufficient to make "semantic distinctions" -- because the proposition isn't true in virtue of that frequency. The trick here is that language models are (currently!) demonstrating you /can/ get most of the way to semantic distinctions just by analyzing symbol-level statistics. Whether you can get all the way is an open question. I agree with you that "movie theaters" don't "sell candy" /because/ of some statistical artifact in large bodies of text. Movie theaters sell candy because people want to eat candy when they watch movies (and are willing to pay for it, etc.). But this wraps back around: the statistical artifacts happen to exist in large bodies of text because it is true. So, with enough text, and the right kind of analysis, you can tease the semantics back out. The power in language models is not that they "understand" text "the right way", from first principles, with a symbolic language model. The power is that they don't have to get most of the way there. Perhaps they'll get all the way there! And if they do, what then? Are we so sure that we don't do the same thing?

nonameiguess 5y ago
It's an open question, but I certainly suspect the reason humans are able to do this is we can synthesize knowledge from other sources and not have to rely solely upon learning from text. We've been to movie theaters and experienced buying candy there, which adds a great deal to our understanding of sentence containing the associated words without needing to read a hundred million sentences about movies and candy and rely only upon statistical patterns in the text to understand it.