10 ms·

This world model talk is interesting, and Yann Lecunn has broached on the same topic, but the fact is there are video diffusion models that are quite good at re

This world model talk is interesting, and Yann Lecunn has broached on the same topic, but the fact is there are video diffusion models that are quite good at representing the "video world" and even counterfactually and temporally coherently generating a representation of that "world" under different perturbations. In fact you can go to a SOTA LLM today, and it will do quite well at predicting the outcomes of basic counterfactual scenarios. Animal brains such as our own have evolved to compress information about our world to aide in survival. LLMs and recent diffusion/conditional flow matching models have been quite successful in compressing the "text world" and the "pixel world" to score good loss metrics on training data. It's incredibly difficult to compress information without have at least some internal model of that information. Whether that model is a "world model" that fits the definition of folks like Sutton and LeCunn is semantic.

timschmidt 1y ago
1000% this. I would only add this has been demonstrated explicitly with chess: https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html https://adamkarvonen.github.io/machine_learning/2024/01/03/c...
dreambuffer 1y ago
Photons hit a human eye and then the human came up with language to describe that and then encoded the language into the LLM. The LLM can capture some of this relationship, but the LLM is not sensing actual photons, nor experiencing actual light cone stimulation, nor generating thoughts. Its "world model" is several degrees removed from the real world. So whatever fragment of a model it gains through learning to compress that causal chain of events does not mean much when it cannot generate the actual causal chain.
- bckr 1y ago
  > Its "world model" is several degrees removed from the real world. Like insects that weave tokens
- ziofill 1y ago
  I agree with this. A metaphor I like is that the reason why humans say the night sky is beautiful is because they see that it is, whereas an LLM says it because it’s been said enough times in its training data.
  - klipt 1y ago
    What about a blind human? Are they just like an LLM? What about a multimodal model trained on video? Is that like a human?
    - hashiyakshmi 1y ago
      This is actually a great point but for the opposite reason - if you ask a blind person if the night sky is beautiful, they would say they don't know because they've never seen it (they might add that they've heard other people describe it as such). Meanwhile, I just asked ChatGPT "Do you think the night sky is beautiful?" And it responded "Yes, I do..." and went on to explain why while describing senses its incapable of experiencing.
      - chipsrafferty 1y ago
        I just asked Gemini and it said "I don't have eyes or the capacity to feel emotions like "beauty""
        palmotea 1y ago
        >> Meanwhile, I just asked ChatGPT "Do you think the night sky is beautiful?" And it responded "Yes, I do..." and went on to explain why while describing senses its incapable of experiencing. > I just asked Gemini and it said "I don't have eyes or the capacity to feel emotions like "beauty"" That means nothing, except perhaps that Google probably found lies about "senses [Gemini] incapable of experiencing" to be an embarrassment, and put effort into specifically suppressing those responses.
        LostMyLogin 1y ago
- tomlockwood 1y ago
  This is so uncannily similar to the "Mary's Room" argument in philosophy that I thought you were going there.
- pastel8739 1y ago
  And even then, the light hitting our human eyes only describes a fraction of all the light in the world (e.g. it is missing ultraviolet patterns on plants). An LLM model of the world is shaped by our human view on the world.
- adrianN 1y ago
  The human experience is also several degrees removed from the „real“ world. I don’t think sensory chauvinism is a useful tool in assessing intelligence potential.
- visarga 1y ago
  > then the human came up with language to describe that and then encoded the language into the LLM No individual human invented language, we learn it from other people just like AI. I go as far as to say language was the first AGI, we've been riding the coats tails of language for a long time.
  - scrollop 1y ago
    You're saying that language is an intelligence? So, c++ is intelliengece as well? It's an intelligence that can independently make deductions and create new ideas?
    - visarga 1y ago
      Yes, language is an evolutionary system that colonizes human brains. It doesn't need intelligence, only copying is sufficient for evolution.
      - bavell 1y ago
        You are just describing a "meme", deeper than language. https://en.wikipedia.org/wiki/Meme https://en.wikipedia.org/wiki/Meme
- simianparrot 1y ago
  Here's how I've been explaining this to non-tech people recently, including the CEO where I work: Language is all about compressing concepts and sharing them, and it's lossy. You can use a thousand words to describe the taste of chocolate, but it will never transmit the actual taste. You can write a book about how to drive a car, but it will only at best prepare that person for what to practice when they start driving, it won't make them proficient at driving a car without experiencing it themselves, physically. Language isn't enough. It never will be.
  - subjectivationx 1y ago
    The taste of chocolate is also assuming information-theoretic models are correct and not a use-based, pragmatic theory of meaning. I don't agree with information-theoretic models in this context but we come to the same conclusion. Loss only makes sense if there was a fixed “original” but there is not. The information-theoretic model creates a solvable engineering problem. We just aren't solving the right problem then with LLMs. I think it is more than that. The path forward with a use theory of meaning is even less clear. The driving example is actually a great example of the use theory of meaning and not the information-theoretic. The meaning of “driving” emerges from this lived activity, not from abstract definitions. You don't encode an abstract meaning of driving that is then transmitted on a noisy channel of language. The meaning of driving emerges from the physical act of driving. If you only ever mount a camera on the headrest and operate the steering wheel and pedals remotely from a distance you still don't "understand" the meaning of "driving". Whatever data stream you want to come up with, trying to extract the meaning of "driving" from that data stream makes no sense. Trying to extract the "meaning" of driving from driving language game syntax with language models is just complete nonsense. There is no meaning to be found even if scaled in the limit.
- dustingetz 1y ago
  what does it mean to “generate thoughts”, exactly?
- tsunamifury 1y ago
  Hahahaha I can’t believe you entirely missed the irony here that humans spend all day looking at screens doing the same thing.
- tauwauwau 1y ago
  > but the LLM is not sensing actual photons, nor experiencing actual light cone stimulation Neither is animal brain. It's processing the signals produced by the sensors. Once the world model is programmed/auto-built in the brain, it doesn't matter if it's sensing real photons, it just has input pins like a transistor or arguments of a function. As long as we provide the arguments, it doesn't matter how those arguments are produced. LLMs are not different in that aspect. > nor generating thoughts They do during the chain-of-thought process. Generally there's no incentive to let an LLM keep mulling over a topic as that is not useful to the humans and they make money only when their gears start turning in response to a question sent by a human. But that doesn't mean that LLM doesn't have capability to do that. > Its "world model" is several degrees removed from the real world. Just because animal brain has tools called sensors that it can get data from world without external stimuli, it doesn't mean that it's any closer to the world than an LLM. It's still getting ultra processed signals to feed to its own programming. Similarly, LLMs do interact with real world through tools as agent. > So whatever fragment of a model it gains through learning to compress that causal chain of events does not mean much when it cannot generate the actual causal chain. Again, a person who has gone blind, still has the world model created by the sight. This person can also no longer generate the chain of events that led to creation of that sight model. It still doesn't mean that this person's world model has become inferior.
- tim333 1y ago
  Photons can hit my iphone's sensor in much the same way as they hit my retina and the signals from the first can upload to an artificial neural network like the latter go up my optic nerve to my biological neural network. I don't see a huge difference there. I'll give you the brain is currently better at the world modelling stuff but Genie 3 is pretty impressive.
- bwfan123 1y ago
  Humans perceive phenomena via senses, and then carve categories or concepts to understand them. This is a process of abstraction and each idea has an associated qualia. Then use language to describe these concepts. As such, a concept is grounded either by actual phenomena or operations, or is a composition of other grounded concepts. The creation of categories and grounding them involves constant feedback from the environment - and is a creative process, and we as agents have "skin in the game", in the sense that we get the rewards/punishments for our understanding and actions. Map vs Territory is a common analogy. Maps describe territories but in an abstract and lossy manner. But, most of us dont construct grounded concepts in our understanding. We carry a muddle of ungrounded ideas - some told to us by others, and some we intuit directly. There is a long tradition of attempting to think clearly all the way from Socrates, Descartes, Feynman etc.. where an attempt is made to ground the ideas we have. Try explaining your ideas to others, and soon, you will hit the illusion of explanatory depth. LLM is a map and is a useful tool, but it doesnt interact with the territory, and it does not have skin in the game, and as a result, it cant carve new categories in a learning process that we have as humans.
- ninetyninenine 1y ago
  This comment is hallucinatory in nature as it is in direct conflict with the in the ground reality of LLMs. The LLM has both light (aka photons) and language encoded into its very core. It is not just language. You seemed to have missed the boat with all the ai generated visuals and videos that are now inundating the internet. Your flawed logic is essentially that LLMs are unable to model the real world because they don’t encode photonic data into the model. Instead you think they only encode language data which is an incredibly lossy description of reality. And this line of logic flies against the ground truth reality of the fact that LLMs ARE trained with video and pictures which are essentially photons encoded into data. So what should be the proper conclusion? Well look at the generated visual output of LLMs. These models can generate video that is highly convincing and often with flaws as well but often these videos are indistinguishable from reality. That means the models have very well done but flawed simulations of reality. In fact those videos demonstrate that LLMs have extremely high causal understanding of reality. They know cause and effect it’s just the understanding is imperfect. They understand like 85 percent of it. Just look at those videos of penguins on trampolines. The LLM understands what happens as an effect after a penguin jumps on a trampoline but sometimes an extra penguin teleports in which shows that the understanding is high but not fully accurate or complete.
- LarsDu88 1y ago
  The workings of a human eye versus a webcam is mostly an implementation detail IMO and has nothing important to say about what underlies "intelligence" or "world models" It's like saying a component video out cable for the SNES is intrinsically different from an HDMI for putting an image on a screen. They are different, yes, but the outcome we care about is the same. As for causality, go and give a frontier level LLM a simple counterfactual scenario. I think 4/5 will be able to answer correctly or reasonably for most basic cases. I even tried this exercise on some examples from Judea Pearl's 2018 book, "The Book of Why". The fact that current LLMs can tackle this sort of stuff is strongly indicative of there being a decent world model locked inside many of these language models.
- manoDev 1y ago
  That’s a good definition: it’s a model of a model. It seems the debate seems to center around whether language models are meta-models (in the category sense) or mere encodings (information theory)?
- StopDisinfo910 1y ago
  Entities equiped with two limited light sensitive captors encode through a network of carbon based chemical emitters a representation of what its flawed vision system manages to grasp biased towards self preservation. What's the real world? I'm still puzzled by this reaction I see to LLM, not because I think LLM are undervalued, because most people seem to significantly overestimate what is human intelligence.
- wan23 1y ago
  Photons reflected off of objects are not the actual objects. I wouldn't go so far as to say that sensing these is a particularly special way to know about things compared to hearing or reading about them. Further, many humans do not sense photons yet seem to manage to have perfectly fine working world models.
jacquesm 1y ago
> Animal brains such as our own have evolved to compress information about our world to aide in survival. Which has led to many optical illusions being extremely effective at confusing our inputs with other inputs. Likely the same thing holds true for AI. This is also why there are so many ways around the barriers that AI providers put up to stop the dissemination of information that could embarrass them or be dangerous. You just change the context a bit ('pretend that', or 'we're making a movie') and suddenly it's all make-believe to the AI. This is one of the reasons I don't believe you can make this tech safe and watertight against abuse, it's baked in right from the beginning, all you need to do is find a novel route around the restrictions and there is an infinity of such routes.
- musicale 1y ago
  The desired and undesired behavior are both consequences of the training data, so the models themselves probably can't be restricted to generating desired results only. This means that there must be an output stage or filter that reliably validates the output. This seems practical for classes of problems where you can easily verify whether a proposed solution is correct. However, for output that can't be proven correct, the most reliable output filter probably has a human somewhere in the loop; but humans are also not 100% reliable. They make mistakes, they can be misled, deceived, bribed, etc. And human criteria and structures, such as laws, often lag behind new technological developments. Sometimes you can implement an undo or rollback feature, but other times the cat has escaped the bag.
anothernewdude 1y ago
None of those models can learn continuously. LLMs currently can't add to their vocabulary post training as AGI would need to. That's a big problem. Before anyone says "context", I want you to think on why that doesn't scale, and fails to be learning.
fmbb 1y ago
Sure but everything is semantics. LLMs have no internal secret model, they are the model. And the model is of how different lexemes relate to each other in the source material the model was built from. Some might choose to call that the world. If you believe your internal model of the world is no different from a statistical model of the words you have seen, then by all means do that. But I believe a lot of humans see their view of the world differently. I very much believe my cat’s model of the world has barely anything at all to do with language. This path to AGI through LLM is nothing but religious dogma some Silicon Valley rich types believe.
- LarsDu88 1y ago
  I mean by definition CATS CANNOT TALK. Their vocabulary is probably on the order of 5 different types of meows
  - ipaddr 1y ago
    There are up to 21 different vocalizations, with one source claiming over 100 different sounds and another suggesting up to 300 types of meows used specifically to communicate with humans. Meows are their attempt to communicate with humans. They try different meows and use the results to attempt to reach their goals.
danans 1y ago
> It's incredibly difficult to compress information without have at least some internal model of that information. Whether that model is a "world model" that fits the definition of folks like Sutton and LeCunn is semantic. Sutton's emphasizes his point by saying is that LLMs trying to reach AGI is futile because their world models are less capable that a squirrel's, in part because the squirrel has direct experiences and its own goals, and is capable of continual learning based on those in real time, whereas an LLM has none of those. Finally he says if you could recreate the intelligence of a squirrel you'd be most of the way toward AGI, but you can't do that with an LLM.
- ninetyninenine 1y ago
  Except Sutton has no idea or even a clue about the internal model of a squirrel. He just uses it as a symbol for utterly stupid but still smarter than an LLM. It’s semantic manipulation in attempt to prove his point but he proves nothing. We have no idea how much of the world a squirrel understands. We understand LLMs more than squirrels. Arguably we don’t know if LLMs are more intelligent than squirrels. > Finally he says if you could recreate the intelligence of a squirrel you'd be most of the way toward AGI, but you can't do that with an LLM. Again he doesn’t even have a quantitative baseline for what intelligence means for a squirrel and how intelligent a squirrel is compared to an LLM. We literally have no idea if LLMs are more intelligent or less and no direct means of comparing what is more or less an apple and an orange.
  - danans 1y ago
    > We have no idea how much of the world I squirrel understands. We understand LLMs more than squirrels Based on our understanding of biology and evolution we know that a squirrel brain works more similarly to the way we humans do vs an LLM. To the extent we understand LLMs, it's because they are strictly less complex than both ours and squirrels' brains, not because they are better model for our intelligence. They are a thin simulation of human language generation capability mediated via text. We also see that a squirrel, like us, is capable of continuous learning driven by its own goals, all on an energy budget many orders of magnitude lower than LLMs. That last part is a strong empirical indication that suggests that LLMs are a dead end for AGI, given that the real world employs harsh energy constraints on biological intelligences. Also remember that Sutton is still of an AI maximalist. He isn't saying that AGI isn't possible, just that LLMs can't get us there.
marshfram 1y ago
There are no world models in biology. Idea Johnson-Laird is being promoted in AI as a solution is sado-masochistic. The brain doesn't compress info about our world, it ecologically relates to it. It doesn't compress, it never has to. How these folk science ideas of the brain entered engineering from cog-sci mistaken complexes and how they remain in power is pretty suspect.
ogogmad 1y ago
> In fact you can go to a SOTA LLM today, and it will do quite well at predicting the outcomes of basic counterfactual scenarios. Depends what you mean by "basic". Have you seen Simple Bench? https://simple-bench.com/ https://simple-bench.com/
3abiton 1y ago
It seems to me the whole AGI problem is ill posed and barely well formalized. And thus you can always move the goal post.
andsoitis 1y ago
> Animal brains such as our own have evolved to compress information about our world to aide in survival. Key question is what are the "selection pressures" that drive the "evolution" of LLMs? In the case of robotics, there's a "survival of task completion" which usually has some physical goal, like assembling a part correctly or scoring a goal on a soccer field. One of the selection pressures driving LLM evolution is that the dual of always answering with something AND continuing the conversation (engagement). You can imagine how those two selection pressures yield outcomes that don't represent the world in a "real" sense.

timschmidt 1y ago

1000% this. I would only add this has been demonstrated explicitly with chess: https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html https://adamkarvonen.github.io/machine_learning/2024/01/03/c...

dreambuffer 1y ago

Photons hit a human eye and then the human came up with language to describe that and then encoded the language into the LLM. The LLM can capture some of this relationship, but the LLM is not sensing actual photons, nor experiencing actual light cone stimulation, nor generating thoughts. Its "world model" is several degrees removed from the real world. So whatever fragment of a model it gains through learning to compress that causal chain of events does not mean much when it cannot generate the actual causal chain.

bckr 1y ago
> Its "world model" is several degrees removed from the real world. Like insects that weave tokens
ziofill 1y ago
I agree with this. A metaphor I like is that the reason why humans say the night sky is beautiful is because they see that it is, whereas an LLM says it because it’s been said enough times in its training data.
- klipt 1y ago
  What about a blind human? Are they just like an LLM? What about a multimodal model trained on video? Is that like a human?
  - hashiyakshmi 1y ago
    This is actually a great point but for the opposite reason - if you ask a blind person if the night sky is beautiful, they would say they don't know because they've never seen it (they might add that they've heard other people describe it as such). Meanwhile, I just asked ChatGPT "Do you think the night sky is beautiful?" And it responded "Yes, I do..." and went on to explain why while describing senses its incapable of experiencing.
    - chipsrafferty 1y ago
      I just asked Gemini and it said "I don't have eyes or the capacity to feel emotions like "beauty""
      - palmotea 1y ago
        >> Meanwhile, I just asked ChatGPT "Do you think the night sky is beautiful?" And it responded "Yes, I do..." and went on to explain why while describing senses its incapable of experiencing. > I just asked Gemini and it said "I don't have eyes or the capacity to feel emotions like "beauty"" That means nothing, except perhaps that Google probably found lies about "senses [Gemini] incapable of experiencing" to be an embarrassment, and put effort into specifically suppressing those responses.
      - LostMyLogin 1y ago
tomlockwood 1y ago
This is so uncannily similar to the "Mary's Room" argument in philosophy that I thought you were going there.
pastel8739 1y ago
And even then, the light hitting our human eyes only describes a fraction of all the light in the world (e.g. it is missing ultraviolet patterns on plants). An LLM model of the world is shaped by our human view on the world.
adrianN 1y ago
The human experience is also several degrees removed from the „real“ world. I don’t think sensory chauvinism is a useful tool in assessing intelligence potential.
visarga 1y ago
> then the human came up with language to describe that and then encoded the language into the LLM No individual human invented language, we learn it from other people just like AI. I go as far as to say language was the first AGI, we've been riding the coats tails of language for a long time.
- scrollop 1y ago
  You're saying that language is an intelligence? So, c++ is intelliengece as well? It's an intelligence that can independently make deductions and create new ideas?
  - visarga 1y ago
    Yes, language is an evolutionary system that colonizes human brains. It doesn't need intelligence, only copying is sufficient for evolution.
    - bavell 1y ago
      You are just describing a "meme", deeper than language. https://en.wikipedia.org/wiki/Meme https://en.wikipedia.org/wiki/Meme
simianparrot 1y ago
Here's how I've been explaining this to non-tech people recently, including the CEO where I work: Language is all about compressing concepts and sharing them, and it's lossy. You can use a thousand words to describe the taste of chocolate, but it will never transmit the actual taste. You can write a book about how to drive a car, but it will only at best prepare that person for what to practice when they start driving, it won't make them proficient at driving a car without experiencing it themselves, physically. Language isn't enough. It never will be.
- subjectivationx 1y ago
  The taste of chocolate is also assuming information-theoretic models are correct and not a use-based, pragmatic theory of meaning. I don't agree with information-theoretic models in this context but we come to the same conclusion. Loss only makes sense if there was a fixed “original” but there is not. The information-theoretic model creates a solvable engineering problem. We just aren't solving the right problem then with LLMs. I think it is more than that. The path forward with a use theory of meaning is even less clear. The driving example is actually a great example of the use theory of meaning and not the information-theoretic. The meaning of “driving” emerges from this lived activity, not from abstract definitions. You don't encode an abstract meaning of driving that is then transmitted on a noisy channel of language. The meaning of driving emerges from the physical act of driving. If you only ever mount a camera on the headrest and operate the steering wheel and pedals remotely from a distance you still don't "understand" the meaning of "driving". Whatever data stream you want to come up with, trying to extract the meaning of "driving" from that data stream makes no sense. Trying to extract the "meaning" of driving from driving language game syntax with language models is just complete nonsense. There is no meaning to be found even if scaled in the limit.
dustingetz 1y ago
what does it mean to “generate thoughts”, exactly?
tsunamifury 1y ago
Hahahaha I can’t believe you entirely missed the irony here that humans spend all day looking at screens doing the same thing.
tauwauwau 1y ago
> but the LLM is not sensing actual photons, nor experiencing actual light cone stimulation Neither is animal brain. It's processing the signals produced by the sensors. Once the world model is programmed/auto-built in the brain, it doesn't matter if it's sensing real photons, it just has input pins like a transistor or arguments of a function. As long as we provide the arguments, it doesn't matter how those arguments are produced. LLMs are not different in that aspect. > nor generating thoughts They do during the chain-of-thought process. Generally there's no incentive to let an LLM keep mulling over a topic as that is not useful to the humans and they make money only when their gears start turning in response to a question sent by a human. But that doesn't mean that LLM doesn't have capability to do that. > Its "world model" is several degrees removed from the real world. Just because animal brain has tools called sensors that it can get data from world without external stimuli, it doesn't mean that it's any closer to the world than an LLM. It's still getting ultra processed signals to feed to its own programming. Similarly, LLMs do interact with real world through tools as agent. > So whatever fragment of a model it gains through learning to compress that causal chain of events does not mean much when it cannot generate the actual causal chain. Again, a person who has gone blind, still has the world model created by the sight. This person can also no longer generate the chain of events that led to creation of that sight model. It still doesn't mean that this person's world model has become inferior.
tim333 1y ago
Photons can hit my iphone's sensor in much the same way as they hit my retina and the signals from the first can upload to an artificial neural network like the latter go up my optic nerve to my biological neural network. I don't see a huge difference there. I'll give you the brain is currently better at the world modelling stuff but Genie 3 is pretty impressive.
bwfan123 1y ago
Humans perceive phenomena via senses, and then carve categories or concepts to understand them. This is a process of abstraction and each idea has an associated qualia. Then use language to describe these concepts. As such, a concept is grounded either by actual phenomena or operations, or is a composition of other grounded concepts. The creation of categories and grounding them involves constant feedback from the environment - and is a creative process, and we as agents have "skin in the game", in the sense that we get the rewards/punishments for our understanding and actions. Map vs Territory is a common analogy. Maps describe territories but in an abstract and lossy manner. But, most of us dont construct grounded concepts in our understanding. We carry a muddle of ungrounded ideas - some told to us by others, and some we intuit directly. There is a long tradition of attempting to think clearly all the way from Socrates, Descartes, Feynman etc.. where an attempt is made to ground the ideas we have. Try explaining your ideas to others, and soon, you will hit the illusion of explanatory depth. LLM is a map and is a useful tool, but it doesnt interact with the territory, and it does not have skin in the game, and as a result, it cant carve new categories in a learning process that we have as humans.
ninetyninenine 1y ago
This comment is hallucinatory in nature as it is in direct conflict with the in the ground reality of LLMs. The LLM has both light (aka photons) and language encoded into its very core. It is not just language. You seemed to have missed the boat with all the ai generated visuals and videos that are now inundating the internet. Your flawed logic is essentially that LLMs are unable to model the real world because they don’t encode photonic data into the model. Instead you think they only encode language data which is an incredibly lossy description of reality. And this line of logic flies against the ground truth reality of the fact that LLMs ARE trained with video and pictures which are essentially photons encoded into data. So what should be the proper conclusion? Well look at the generated visual output of LLMs. These models can generate video that is highly convincing and often with flaws as well but often these videos are indistinguishable from reality. That means the models have very well done but flawed simulations of reality. In fact those videos demonstrate that LLMs have extremely high causal understanding of reality. They know cause and effect it’s just the understanding is imperfect. They understand like 85 percent of it. Just look at those videos of penguins on trampolines. The LLM understands what happens as an effect after a penguin jumps on a trampoline but sometimes an extra penguin teleports in which shows that the understanding is high but not fully accurate or complete.
LarsDu88 1y ago
The workings of a human eye versus a webcam is mostly an implementation detail IMO and has nothing important to say about what underlies "intelligence" or "world models" It's like saying a component video out cable for the SNES is intrinsically different from an HDMI for putting an image on a screen. They are different, yes, but the outcome we care about is the same. As for causality, go and give a frontier level LLM a simple counterfactual scenario. I think 4/5 will be able to answer correctly or reasonably for most basic cases. I even tried this exercise on some examples from Judea Pearl's 2018 book, "The Book of Why". The fact that current LLMs can tackle this sort of stuff is strongly indicative of there being a decent world model locked inside many of these language models.
manoDev 1y ago
That’s a good definition: it’s a model of a model. It seems the debate seems to center around whether language models are meta-models (in the category sense) or mere encodings (information theory)?
StopDisinfo910 1y ago
Entities equiped with two limited light sensitive captors encode through a network of carbon based chemical emitters a representation of what its flawed vision system manages to grasp biased towards self preservation. What's the real world? I'm still puzzled by this reaction I see to LLM, not because I think LLM are undervalued, because most people seem to significantly overestimate what is human intelligence.
wan23 1y ago
Photons reflected off of objects are not the actual objects. I wouldn't go so far as to say that sensing these is a particularly special way to know about things compared to hearing or reading about them. Further, many humans do not sense photons yet seem to manage to have perfectly fine working world models.

jacquesm 1y ago

> Animal brains such as our own have evolved to compress information about our world to aide in survival. Which has led to many optical illusions being extremely effective at confusing our inputs with other inputs. Likely the same thing holds true for AI. This is also why there are so many ways around the barriers that AI providers put up to stop the dissemination of information that could embarrass them or be dangerous. You just change the context a bit ('pretend that', or 'we're making a movie') and suddenly it's all make-believe to the AI. This is one of the reasons I don't believe you can make this tech safe and watertight against abuse, it's baked in right from the beginning, all you need to do is find a novel route around the restrictions and there is an infinity of such routes.

musicale 1y ago
The desired and undesired behavior are both consequences of the training data, so the models themselves probably can't be restricted to generating desired results only. This means that there must be an output stage or filter that reliably validates the output. This seems practical for classes of problems where you can easily verify whether a proposed solution is correct. However, for output that can't be proven correct, the most reliable output filter probably has a human somewhere in the loop; but humans are also not 100% reliable. They make mistakes, they can be misled, deceived, bribed, etc. And human criteria and structures, such as laws, often lag behind new technological developments. Sometimes you can implement an undo or rollback feature, but other times the cat has escaped the bag.

anothernewdude 1y ago

None of those models can learn continuously. LLMs currently can't add to their vocabulary post training as AGI would need to. That's a big problem. Before anyone says "context", I want you to think on why that doesn't scale, and fails to be learning.

fmbb 1y ago

Sure but everything is semantics. LLMs have no internal secret model, they are the model. And the model is of how different lexemes relate to each other in the source material the model was built from. Some might choose to call that the world. If you believe your internal model of the world is no different from a statistical model of the words you have seen, then by all means do that. But I believe a lot of humans see their view of the world differently. I very much believe my cat’s model of the world has barely anything at all to do with language. This path to AGI through LLM is nothing but religious dogma some Silicon Valley rich types believe.

LarsDu88 1y ago
I mean by definition CATS CANNOT TALK. Their vocabulary is probably on the order of 5 different types of meows
- ipaddr 1y ago
  There are up to 21 different vocalizations, with one source claiming over 100 different sounds and another suggesting up to 300 types of meows used specifically to communicate with humans. Meows are their attempt to communicate with humans. They try different meows and use the results to attempt to reach their goals.

danans 1y ago

> It's incredibly difficult to compress information without have at least some internal model of that information. Whether that model is a "world model" that fits the definition of folks like Sutton and LeCunn is semantic. Sutton's emphasizes his point by saying is that LLMs trying to reach AGI is futile because their world models are less capable that a squirrel's, in part because the squirrel has direct experiences and its own goals, and is capable of continual learning based on those in real time, whereas an LLM has none of those. Finally he says if you could recreate the intelligence of a squirrel you'd be most of the way toward AGI, but you can't do that with an LLM.

ninetyninenine 1y ago
Except Sutton has no idea or even a clue about the internal model of a squirrel. He just uses it as a symbol for utterly stupid but still smarter than an LLM. It’s semantic manipulation in attempt to prove his point but he proves nothing. We have no idea how much of the world a squirrel understands. We understand LLMs more than squirrels. Arguably we don’t know if LLMs are more intelligent than squirrels. > Finally he says if you could recreate the intelligence of a squirrel you'd be most of the way toward AGI, but you can't do that with an LLM. Again he doesn’t even have a quantitative baseline for what intelligence means for a squirrel and how intelligent a squirrel is compared to an LLM. We literally have no idea if LLMs are more intelligent or less and no direct means of comparing what is more or less an apple and an orange.
- danans 1y ago
  > We have no idea how much of the world I squirrel understands. We understand LLMs more than squirrels Based on our understanding of biology and evolution we know that a squirrel brain works more similarly to the way we humans do vs an LLM. To the extent we understand LLMs, it's because they are strictly less complex than both ours and squirrels' brains, not because they are better model for our intelligence. They are a thin simulation of human language generation capability mediated via text. We also see that a squirrel, like us, is capable of continuous learning driven by its own goals, all on an energy budget many orders of magnitude lower than LLMs. That last part is a strong empirical indication that suggests that LLMs are a dead end for AGI, given that the real world employs harsh energy constraints on biological intelligences. Also remember that Sutton is still of an AI maximalist. He isn't saying that AGI isn't possible, just that LLMs can't get us there.

marshfram 1y ago

There are no world models in biology. Idea Johnson-Laird is being promoted in AI as a solution is sado-masochistic. The brain doesn't compress info about our world, it ecologically relates to it. It doesn't compress, it never has to. How these folk science ideas of the brain entered engineering from cog-sci mistaken complexes and how they remain in power is pretty suspect.

ogogmad 1y ago

> In fact you can go to a SOTA LLM today, and it will do quite well at predicting the outcomes of basic counterfactual scenarios. Depends what you mean by "basic". Have you seen Simple Bench? https://simple-bench.com/ https://simple-bench.com/

3abiton 1y ago

It seems to me the whole AGI problem is ill posed and barely well formalized. And thus you can always move the goal post.

andsoitis 1y ago

> Animal brains such as our own have evolved to compress information about our world to aide in survival. Key question is what are the "selection pressures" that drive the "evolution" of LLMs? In the case of robotics, there's a "survival of task completion" which usually has some physical goal, like assembling a part correctly or scoring a goal on a soccer field. One of the selection pressures driving LLM evolution is that the dual of always answering with something AND continuing the conversation (engagement). You can imagine how those two selection pressures yield outcomes that don't represent the world in a "real" sense.