4 ms·

What does it mean to actually understand why the sky is blue? What are the specifics characteristics of a hypothetical AI system that you would feel comfortab

What does it mean to actually understand why the sky is blue? What are the specifics characteristics of a hypothetical AI system that you would feel comfortable giving the label of “understanding” to?

Night_Thastus 3y ago
From facts and processes we know, we can derive novel information and conclusions. If you really understand why the sky is blue, you might be able to come to conclusions about why other things appear a certain color, like human eye color. GPT can't make those kinds of reasoning or extensions. It can only regurgitate what is already known and has been stated before, somewhere before in its training set. It's very impressive, I just think people over-hype it into something it is not.
- dwaltrip 3y ago
  Can you propose more tests of “understanding”? The more concrete the better. Or is more like “you know it when you see it”? I tested GPT-4 with your example of sky color / human eyes. It performed quite well and seemed to have a pretty coherent grasp of the subject and related associations. Link to the convo: https://chat.openai.com/share/77add48f-abdc-4734-ac55-05b8d1ba9093 https://chat.openai.com/share/77add48f-abdc-4734-ac55-05b8d1... However, I could see how one might argue the reasoning is not that complex. I strongly maintain that there is some form of reasoning going on here, for all meaningful definitions of the word. But it is a complex and tricky thing to analyze. Lastly, your original comment veered dangerously close to claiming that models trained to predict text cannot — by definition! — ever acquire any form of reasoning or understanding of the world. This is a very strong claim that I don’t think can be substantiated, at least with the tools and knowledge we have today.
  - Night_Thastus 3y ago
    >Lastly, your original comment veered dangerously close to claiming that models trained to predict text cannot — by definition! — ever acquire any form of reasoning or understanding of the world. They can't, and I stand by that. An LLM is always going to be an LLM, it's never magically going to turn into a gAI. Now, some of the lessons learned from developing all these LLMs could be used to develop a gAI, but that's a long way away and would need to be built from the ground up for that purpose by design.
    - dwaltrip 3y ago
      Can you please argue your case? I argue that the model’s weights could contain a rich representation of the world, even if it looks different from that of humans. And that it can use this capability of world modeling to also make new connections. These “world modeling” / “reasoning” abilities seem to have some similarities with that of humans. We see echoes of our own abilities in gpt-4 and the like. But they also have their own peculiarities and limitations, which isn’t surprising in hindsight.

Night_Thastus 3y ago

From facts and processes we know, we can derive novel information and conclusions. If you really understand why the sky is blue, you might be able to come to conclusions about why other things appear a certain color, like human eye color. GPT can't make those kinds of reasoning or extensions. It can only regurgitate what is already known and has been stated before, somewhere before in its training set. It's very impressive, I just think people over-hype it into something it is not.

dwaltrip 3y ago
Can you propose more tests of “understanding”? The more concrete the better. Or is more like “you know it when you see it”? I tested GPT-4 with your example of sky color / human eyes. It performed quite well and seemed to have a pretty coherent grasp of the subject and related associations. Link to the convo: https://chat.openai.com/share/77add48f-abdc-4734-ac55-05b8d1ba9093 https://chat.openai.com/share/77add48f-abdc-4734-ac55-05b8d1... However, I could see how one might argue the reasoning is not that complex. I strongly maintain that there is some form of reasoning going on here, for all meaningful definitions of the word. But it is a complex and tricky thing to analyze. Lastly, your original comment veered dangerously close to claiming that models trained to predict text cannot — by definition! — ever acquire any form of reasoning or understanding of the world. This is a very strong claim that I don’t think can be substantiated, at least with the tools and knowledge we have today.
- Night_Thastus 3y ago
  >Lastly, your original comment veered dangerously close to claiming that models trained to predict text cannot — by definition! — ever acquire any form of reasoning or understanding of the world. They can't, and I stand by that. An LLM is always going to be an LLM, it's never magically going to turn into a gAI. Now, some of the lessons learned from developing all these LLMs could be used to develop a gAI, but that's a long way away and would need to be built from the ground up for that purpose by design.
  - dwaltrip 3y ago
    Can you please argue your case? I argue that the model’s weights could contain a rich representation of the world, even if it looks different from that of humans. And that it can use this capability of world modeling to also make new connections. These “world modeling” / “reasoning” abilities seem to have some similarities with that of humans. We see echoes of our own abilities in gpt-4 and the like. But they also have their own peculiarities and limitations, which isn’t surprising in hindsight.