4 ms·

> Models are transitive- if x models y, and y models z, then x models z. The upshot of these facts are that if you have a really good statistical model of how w

by rland 6y ago

> Models are transitive- if x models y, and y models z, then x models z. The upshot of these facts are that if you have a really good statistical model of how words relate to each other, that model is also implicitly a model of the world. This right here is a great way of putting the success of GPT-3 into context. We think GPT is smart, because when it says something eerily human-like we apply our model of the world onto what it is saying. A conversation like this: > Me: So, what happened when you fell off the balance beam? > GPT: It hurt. > Me: Why'd it hurt so bad? > GPT: The beam was high up and I feel awkwardly. > Me: Wow, that sounds awful. In this conversation, one of us is thinking far harder than the other. GPT can have conversations like this now, which is impressive. But only I can model the beam, the fall, and the physical reality. When I say "that sounds awful," I actually do a miniature physics simulation in my head, imagining losing my balance and falling off a high beam, landing, the physical pain, etc. GPT does none of that. In either case, when it asks the question or when it answers it, it is entirely ignorant of this sort of "shadow" model that's being constructed. Generalizing a bit, our "shadow" model of reality in every single domain is far more powerful than language's approximation. That's why we won't be able to use GPT to do a medical diagnosis or create a piece of architecture or whatever else people are saying it's going to do now.

ilaksh 6y ago
Right. I recently saw a podcast with Dileep George of Vicarious AI. I don't know if his specific techniques will work fully generally but at the high level he is talking about something quite similar to what you are saying in terms of grounding language understanding and having real models of the world. So I am definitely following him. More broadly there is a growing group of researchers who are working on trying to achieve better world modeling (although of course there have been many for decades, it just seems that the number who are coming from deep learning towards AGI with more emphasis on world modeling is increasing). Such as Ferran Alet. Sort of the Tenenbaum school I guess. Or Melanie Mitchell has been saying similar for quite awhile. They have recent talks on Good AI's YouTube channel.
dwohnitmok 6y ago
The author's quote is asserting the opposite of what you're saying here. Filling in "GPT" for x, "text" for y, and "world" for z, the author is stating "if GPT models text, and text models the world, then GPT models the world." In particular, the author directly addresses your point that "Generalizing a bit, our 'shadow' model of reality in every single domain is far more powerful than language's approximation." > Modality, uncertainty, vagueness and other complexities enter but the isomorphism between world and language is there, even if inexact. My own personal view tends to lean towards the author's. GPT-3 , despite its many astonishing achievements, has many many limitations. Yet I don't see any indication as of yet that any of those limitations are fundamental ones inherent to the approach behind the GPT series, which is why GPT-3 both excites and frightens me.
- akakww 6y ago
  GPT models the world through text* There are many other things that go into modelling the world, like drawing pictures, visualising things, communicating in non-text ways essentially. GPT can only model the world in the ways that text can model the world, which is limited.
  - dwohnitmok 6y ago
    Although a differentiating factor in the near-term, in the grand scheme of things that is merely an implementation detail. The rise of computing and its myriad encoders is an existence proof that the world we humans know can be, to an arbitrary degree of precision, modeled by 0s and 1s and what are 0s and 1s but the purest essence of text? Although fraught with many large implementation difficulties, a GPT trained on binary data does not represent a fundamental difficulty vs one trained on text.
    - randcraw 6y ago
      No, and not even close. Text cannot model many things. Try learning surgery using a textbook without pictures. Try driving a racecar competitively or learning a foreign language after only reading a book, even one with pictures. Models of all kinds are always poor surrogates for reality, and models that cannot employ logic or causality CLEARLY cannot model a world in which mechanisms cause all change. Statistical models can indeed describe many observations, but can never break out from the echo chamber of copy-catting patterns it has already seen. If a probabilistic engine like a deep net hasn’t been exposed to a concept during training, it will never induce its existence. Imagination requires initiative, the proposal of an unknown and unfamiliar outcome via logical inference or causal induction. Until deep nets can employ both of these skills, they will never master many skills humans use routinely to explain the world or extend our understanding of how it works.
lmm 6y ago
> In this conversation, one of us is thinking far harder than the other. GPT can have conversations like this now, which is impressive. But only I can model the beam, the fall, and the physical reality. When I say "that sounds awful," I actually do a miniature physics simulation in my head, imagining losing my balance and falling off a high beam, landing, the physical pain, etc. GPT does none of that. In either case, when it asks the question or when it answers it, it is entirely ignorant of this sort of "shadow" model that's being constructed. GPT-3 could presumably write a paragraph like that one. You can claim to have a working physics model in your head, but why should I believe that unless it becomes evident from the things that you communicate to me? I've certainly met humans who could have a superficially legitimate conversation about objects in motion while harbouring enormous misconceptions about the physics involved. Maybe the biggest takeaway from GPT-3 should be that we should raise our standards for human conversation, demanding more precise language and giving less credit to flourishes that make the meaning ambiguous.
- akakww 6y ago
  wow that last point is profound, it would almost solve the problem of misinformation, bogus headlines, double speak, etc. Simply raising the standard for what good speech is.
- rland 6y ago
  I mean the miniature physics simulation of me actually imagining another embodied human falling off of a beam. There is a huge knowledge of the physical world encoded into that. Think about it, you can viscerally imagine balancing on a beam, losing your footing, falling, and the subsequent forces and accelerations on all parts of your body during and after the fall, in real time. All of that information is in our model, and none of it is in GPT's. Everyone has this, it's builtin. I'm not talking about Newton's Laws here. The physical mechanics of being an embodied thing isn't the only example of this. Any experiential knowledge, any knowledge with feedback suffers from this. And any knowledge that is difficult to put into words, like emotional knowledge, interpersonal knowledge, etc., will be off-limits. That's my point.
  - mercer 6y ago

by rland 6y ago

ilaksh 6y ago

Right. I recently saw a podcast with Dileep George of Vicarious AI. I don't know if his specific techniques will work fully generally but at the high level he is talking about something quite similar to what you are saying in terms of grounding language understanding and having real models of the world. So I am definitely following him. More broadly there is a growing group of researchers who are working on trying to achieve better world modeling (although of course there have been many for decades, it just seems that the number who are coming from deep learning towards AGI with more emphasis on world modeling is increasing). Such as Ferran Alet. Sort of the Tenenbaum school I guess. Or Melanie Mitchell has been saying similar for quite awhile. They have recent talks on Good AI's YouTube channel.

dwohnitmok 6y ago

The author's quote is asserting the opposite of what you're saying here. Filling in "GPT" for x, "text" for y, and "world" for z, the author is stating "if GPT models text, and text models the world, then GPT models the world." In particular, the author directly addresses your point that "Generalizing a bit, our 'shadow' model of reality in every single domain is far more powerful than language's approximation." > Modality, uncertainty, vagueness and other complexities enter but the isomorphism between world and language is there, even if inexact. My own personal view tends to lean towards the author's. GPT-3 , despite its many astonishing achievements, has many many limitations. Yet I don't see any indication as of yet that any of those limitations are fundamental ones inherent to the approach behind the GPT series, which is why GPT-3 both excites and frightens me.

akakww 6y ago
GPT models the world through text* There are many other things that go into modelling the world, like drawing pictures, visualising things, communicating in non-text ways essentially. GPT can only model the world in the ways that text can model the world, which is limited.
- dwohnitmok 6y ago
  Although a differentiating factor in the near-term, in the grand scheme of things that is merely an implementation detail. The rise of computing and its myriad encoders is an existence proof that the world we humans know can be, to an arbitrary degree of precision, modeled by 0s and 1s and what are 0s and 1s but the purest essence of text? Although fraught with many large implementation difficulties, a GPT trained on binary data does not represent a fundamental difficulty vs one trained on text.
  - randcraw 6y ago
    No, and not even close. Text cannot model many things. Try learning surgery using a textbook without pictures. Try driving a racecar competitively or learning a foreign language after only reading a book, even one with pictures. Models of all kinds are always poor surrogates for reality, and models that cannot employ logic or causality CLEARLY cannot model a world in which mechanisms cause all change. Statistical models can indeed describe many observations, but can never break out from the echo chamber of copy-catting patterns it has already seen. If a probabilistic engine like a deep net hasn’t been exposed to a concept during training, it will never induce its existence. Imagination requires initiative, the proposal of an unknown and unfamiliar outcome via logical inference or causal induction. Until deep nets can employ both of these skills, they will never master many skills humans use routinely to explain the world or extend our understanding of how it works.

lmm 6y ago

> In this conversation, one of us is thinking far harder than the other. GPT can have conversations like this now, which is impressive. But only I can model the beam, the fall, and the physical reality. When I say "that sounds awful," I actually do a miniature physics simulation in my head, imagining losing my balance and falling off a high beam, landing, the physical pain, etc. GPT does none of that. In either case, when it asks the question or when it answers it, it is entirely ignorant of this sort of "shadow" model that's being constructed. GPT-3 could presumably write a paragraph like that one. You can claim to have a working physics model in your head, but why should I believe that unless it becomes evident from the things that you communicate to me? I've certainly met humans who could have a superficially legitimate conversation about objects in motion while harbouring enormous misconceptions about the physics involved. Maybe the biggest takeaway from GPT-3 should be that we should raise our standards for human conversation, demanding more precise language and giving less credit to flourishes that make the meaning ambiguous.

akakww 6y ago
wow that last point is profound, it would almost solve the problem of misinformation, bogus headlines, double speak, etc. Simply raising the standard for what good speech is.
rland 6y ago
I mean the miniature physics simulation of me actually imagining another embodied human falling off of a beam. There is a huge knowledge of the physical world encoded into that. Think about it, you can viscerally imagine balancing on a beam, losing your footing, falling, and the subsequent forces and accelerations on all parts of your body during and after the fall, in real time. All of that information is in our model, and none of it is in GPT's. Everyone has this, it's builtin. I'm not talking about Newton's Laws here. The physical mechanics of being an embodied thing isn't the only example of this. Any experiential knowledge, any knowledge with feedback suffers from this. And any knowledge that is difficult to put into words, like emotional knowledge, interpersonal knowledge, etc., will be off-limits. That's my point.
- mercer 6y ago