3 ms·

The interpretation of the LLM having a "world model" is a big stretch in terminology. Encoding a coherent or accurate set of spatial coordinates for places is n

by lsy 3y ago

The interpretation of the LLM having a "world model" is a big stretch in terminology. Encoding a coherent or accurate set of spatial coordinates for places is not equivalent to understanding the actuality of world space and the relationship between items in that space. By this definition an accurate spreadsheet of the lat-long of every major city would also constitute a "world model". It shouldn't be surprising to anyone that a model trained on (likely many) such spreadsheets would also encode that coordinate data. What's notable is that it still takes a team of human researchers to plot these raw numbers onto a map and to interpret them as experiential differentiators in physical space. There's no doubt that an LLM can uncover whatever is structurally included in its training data, even if that encoding is implicit. What's less believable is that the LLM somehow achieves a grounded understanding of physical aspects of experience purely from streams of raw text. I think the paper engages in some equivocation here, as the abstract differentiates between "an enormous collection of superficial statistics" and "a coherent model of the data generating process", without admitting that the first should well imply the second. But that doesn't then further imply a "world model" in the sense that we understand it as sentient beings. For us, an internal model is useful, but when our model or texts don't agree with the actuality of the world, the actuality of the world takes precedence. For an LLM there is no distinguishing between its trained representations and an actual exterior place, or even any sense that an exterior space exists.

nopinsight 3y ago
Current LLMs clearly have much coarser models of the world than humans do. Their training data do not include as many modalities and training data with spatiotemporal dimensions are inadequate. Up and coming multimodal models are changing this.
- lsy 3y ago
  "Multimodal" models only promise an additional layer of indirection and interpretation from the world through sensors into the same conceptual mechanism. An image is converted to tokens, a sensor is converted to coordinates, and it is all fed back into a correlating model. But this isn't adequate either. The famous dress illusion picture has RGB values closer to gold and white, but the actual physical dress was black and blue. The difference between perception and reality is something we don't have a handle on when it comes to these models.
  - fiso64 3y ago
    Tokens are just another representation of the image, like 3 matrices or electrical impulses in our brain. They probably encode most of the relevant information, the question is whether the linear algebra at the core of a transformer is good enough to use this data as efficiently as we do. I don't think we can currently say if it is or isn't adequate.
  - nopinsight 3y ago
    > There's no doubt that an LLM can uncover whatever is structurally included in its training data, even if that encoding is implicit. What's less believable is that the LLM somehow achieves a grounded understanding of physical aspects of experience purely from streams of raw text. I was responding to the point in your first paragraph in your upper level comment and this quoted paragraph. Multimodal models will address that. Your new point about perception could still be a problem for multimodal models, but it’s also possible that with sufficient data, the new models could address that (eg it could say people perceive the colors differently, as A and B). Multimodal models map between human language and raw sensory data. It’s fundamentally similar, in principle, to part of how we learn our first language. (Not saying we do only that though.)
ComplexSystems 3y ago
"Encoding a coherent or accurate set of spatial coordinates for places is not equivalent to understanding the actuality of world space and the relationship between items in that space." What do you mean by the "actuality" of world space? It certainly has memorized an enormous data set of geographic points for various important locations and can do some pretty sophisticated reasoning and inference from them, and with the code interpreter it can compute various metrics derived from them. What can this thing do that humans can't?

by lsy 3y ago

nopinsight 3y ago

Current LLMs clearly have much coarser models of the world than humans do. Their training data do not include as many modalities and training data with spatiotemporal dimensions are inadequate. Up and coming multimodal models are changing this.

lsy 3y ago
"Multimodal" models only promise an additional layer of indirection and interpretation from the world through sensors into the same conceptual mechanism. An image is converted to tokens, a sensor is converted to coordinates, and it is all fed back into a correlating model. But this isn't adequate either. The famous dress illusion picture has RGB values closer to gold and white, but the actual physical dress was black and blue. The difference between perception and reality is something we don't have a handle on when it comes to these models.
- fiso64 3y ago
  Tokens are just another representation of the image, like 3 matrices or electrical impulses in our brain. They probably encode most of the relevant information, the question is whether the linear algebra at the core of a transformer is good enough to use this data as efficiently as we do. I don't think we can currently say if it is or isn't adequate.
- nopinsight 3y ago
  > There's no doubt that an LLM can uncover whatever is structurally included in its training data, even if that encoding is implicit. What's less believable is that the LLM somehow achieves a grounded understanding of physical aspects of experience purely from streams of raw text. I was responding to the point in your first paragraph in your upper level comment and this quoted paragraph. Multimodal models will address that. Your new point about perception could still be a problem for multimodal models, but it’s also possible that with sufficient data, the new models could address that (eg it could say people perceive the colors differently, as A and B). Multimodal models map between human language and raw sensory data. It’s fundamentally similar, in principle, to part of how we learn our first language. (Not saying we do only that though.)

ComplexSystems 3y ago

"Encoding a coherent or accurate set of spatial coordinates for places is not equivalent to understanding the actuality of world space and the relationship between items in that space." What do you mean by the "actuality" of world space? It certainly has memorized an enormous data set of geographic points for various important locations and can do some pretty sophisticated reasoning and inference from them, and with the code interpreter it can compute various metrics derived from them. What can this thing do that humans can't?