4 ms·

> LLMs have no understanding of the underlying reality that language describes. LLMs absolutely have concepts that extend beyond just words. As early as 2008 (

> LLMs have no understanding of the underlying reality that language describes. LLMs absolutely have concepts that extend beyond just words. As early as 2008 (back when there were no large language models, only "regular" language models), we've been able to demonstrate things that seem to me like the model is learning abstract concepts. For a classic example, see Linguistic Regularities in Continuous Space Word Representations [0], a 2013 paper that talks about a model with a latent space where one can take the vector representations of the words "king", "queen", "man", and "woman" and literally perform the arithmetic `king - (man - woman) ≈ queen`. To me, this clearly demonstrates that the model "understands" the concept of gender, represented by a vector in the model's latent space. The set of numbers that you get from the subtraction `man - woman` represent the concept of the difference between the male and female genders, without needing a specific word tied to that representation of the concept. It's debatable whether or not that counts as "understanding", but that's more of a semantic debate about what it means to understand, not a debate about the model's internal knowledge of the world and ability to do things with that knowledge. 0: https://aclanthology.org/N13-1090.pdf https://aclanthology.org/N13-1090.pdf

yladiz 3y ago
Are you sure the model in the paper linked and LLMs function similarly or identically in this regard?
- hatthew 3y ago
  No, but IMO an assertion that they have no understanding of the world has less supporting evidence than an assertion that they do have some level of understanding.
happytiger 3y ago
I would suggest that example is more likely evidence of semantic associations provided by underlying taxonomy based on input training data. There is so much contextual information in language hierarchy and associated characteristics and there is nothing stopping a LLM from using multiple recombinant factors for generation. I don’t mean to discount your excellent comment. I mean obviously these models are so complex already that their black box nature makes it difficult to derive conclusive results, but Arkham’s razor would imply that the simplest explanation — that is it recombined training data creating a heirarchy with multiple factors that provides context for the “correct” answer — that is more likely correct. In other words a LlM can confer multiple related associations to output based on training data, and that is the most likely explanation for that behavior. Just look at the poor performance with GPT-3 and the underlying data quality issues the Alpaca dataset team has discussed. I’m no expert, and don’t mean to counter you as much as contribute my own limited thinking for discussion here. Cheers!
- hatthew 3y ago
  Hmmm, do you mean that it recreates a hierarchy of words in its latent space? I would argue that such a structured spatial hierarchy is the model's understanding of the world.

yladiz 3y ago

Are you sure the model in the paper linked and LLMs function similarly or identically in this regard?

hatthew 3y ago
No, but IMO an assertion that they have no understanding of the world has less supporting evidence than an assertion that they do have some level of understanding.

happytiger 3y ago

I would suggest that example is more likely evidence of semantic associations provided by underlying taxonomy based on input training data. There is so much contextual information in language hierarchy and associated characteristics and there is nothing stopping a LLM from using multiple recombinant factors for generation. I don’t mean to discount your excellent comment. I mean obviously these models are so complex already that their black box nature makes it difficult to derive conclusive results, but Arkham’s razor would imply that the simplest explanation — that is it recombined training data creating a heirarchy with multiple factors that provides context for the “correct” answer — that is more likely correct. In other words a LlM can confer multiple related associations to output based on training data, and that is the most likely explanation for that behavior. Just look at the poor performance with GPT-3 and the underlying data quality issues the Alpaca dataset team has discussed. I’m no expert, and don’t mean to counter you as much as contribute my own limited thinking for discussion here. Cheers!

hatthew 3y ago
Hmmm, do you mean that it recreates a hierarchy of words in its latent space? I would argue that such a structured spatial hierarchy is the model's understanding of the world.