3 ms·

We are using different meanings for terms. Building a world model for a perfect information game is different than building an mental model of the external wor

We are using different meanings for terms. Building a world model for a perfect information game is different than building an mental model of the external world. In context learning is a well known property of LLMs, while real world generalization, often described through the common sense problem is not. To me, 'mental models' are personal, internal representations of external reality, which LLMs currently lack, being limited to the corpus.

ctoth 2y ago
I believe they used the game to show how the same underlying technology would build a mental model not because the game was perfect-information, but because it was easy to probe for without a lot of other unrelated concepts getting in the way. Perhaps this is what you're looking for (similar technique, larger model?) https://www.anthropic.com/news/golden-gate-claude https://www.anthropic.com/news/golden-gate-claude > In the “mind” of Claude, we found millions of concepts that activate when the model reads relevant text or sees relevant images, which we call “features”. > One of those was the concept of the Golden Gate Bridge. We found that there’s a specific combination of neurons in Claude’s neural network that activates when it encounters a mention (or a picture) of this most famous San Francisco landmark. Sounds like a mental model to me! An internal representation of an external concept which exists in the real world.
accountnum 2y ago
No, you're not. Are you genuinely trying to suggest that LLMs, which can: - Construct arbitrary text that isn't just grammatically but semantically coherent - Derive intent, subtle intent, from user queries and responses - Emulate endless different personalities and their reactions to endless stimuli - Describe in detail the statics and dynamics of the world, including sight, smell, touch and sound do not have a model of the external world? What do you think a "corpus" means in this context? How is the "corpus" of sensory and evolutionary data that makes you up in any way different? LLMs are excellent common sense reasoners, and they generalize just fine. Why exactly do you think they get things _subtly_ wrong? Make up API syntax that looks sensible but isn't actually implemented? In order to make these guesses they need to have generalized, they need an understanding of the structure underlying naming, such that they can produce _sensible_ output even if they lack the hard facts.
- FrustratedMonky 2y ago
  You are correct. We are flooded with studies on AI now, so can't find reference. But just few months ago, saw example of AI, from video, building an internal representation of the world. An internal model of the world. Everyone saying this can't be done, it already is. Maybe can argue it wasn't an LLM, and then I'd say were nitpicking over which technology can do it or not. We already have example of tying them together, symbols and LLM's. Might be related. https://www.nature.com/articles/d41586-024-00288-1 https://www.nature.com/articles/d41586-024-00288-1 https://www.technologyreview.com/2019/04/08/103223/two-rival-ai-approaches-combine-to-let-machines-learn-about-the-world-like-a-child/ https://www.technologyreview.com/2019/04/08/103223/two-rival...

ctoth 2y ago

I believe they used the game to show how the same underlying technology would build a mental model not because the game was perfect-information, but because it was easy to probe for without a lot of other unrelated concepts getting in the way. Perhaps this is what you're looking for (similar technique, larger model?) https://www.anthropic.com/news/golden-gate-claude https://www.anthropic.com/news/golden-gate-claude > In the “mind” of Claude, we found millions of concepts that activate when the model reads relevant text or sees relevant images, which we call “features”. > One of those was the concept of the Golden Gate Bridge. We found that there’s a specific combination of neurons in Claude’s neural network that activates when it encounters a mention (or a picture) of this most famous San Francisco landmark. Sounds like a mental model to me! An internal representation of an external concept which exists in the real world.

accountnum 2y ago

No, you're not. Are you genuinely trying to suggest that LLMs, which can: - Construct arbitrary text that isn't just grammatically but semantically coherent - Derive intent, subtle intent, from user queries and responses - Emulate endless different personalities and their reactions to endless stimuli - Describe in detail the statics and dynamics of the world, including sight, smell, touch and sound do not have a model of the external world? What do you think a "corpus" means in this context? How is the "corpus" of sensory and evolutionary data that makes you up in any way different? LLMs are excellent common sense reasoners, and they generalize just fine. Why exactly do you think they get things _subtly_ wrong? Make up API syntax that looks sensible but isn't actually implemented? In order to make these guesses they need to have generalized, they need an understanding of the structure underlying naming, such that they can produce _sensible_ output even if they lack the hard facts.

FrustratedMonky 2y ago
You are correct. We are flooded with studies on AI now, so can't find reference. But just few months ago, saw example of AI, from video, building an internal representation of the world. An internal model of the world. Everyone saying this can't be done, it already is. Maybe can argue it wasn't an LLM, and then I'd say were nitpicking over which technology can do it or not. We already have example of tying them together, symbols and LLM's. Might be related. https://www.nature.com/articles/d41586-024-00288-1 https://www.nature.com/articles/d41586-024-00288-1 https://www.technologyreview.com/2019/04/08/103223/two-rival-ai-approaches-combine-to-let-machines-learn-about-the-world-like-a-child/ https://www.technologyreview.com/2019/04/08/103223/two-rival...