3 ms·

Well I actually read Neel Nanda's writings on it which acknowledge weaknesses and potential gaps. Because I'm not qualified to judge it myself. But that's hard

Well I actually read Neel Nanda's writings on it which acknowledge weaknesses and potential gaps. Because I'm not qualified to judge it myself. But that's hardly the point. The question is whether or not "general intelligence" is an emergent property from stupider processes, and my view is "Yes, almost certainly, isn't that the most likely explanation for our own intelligence?" If it is, and we keep seeing LLMs building more robust approximations of real world models, it's pretty insane to say "No, there is without doubt a wall we're going to hit. It's invisible but I know it's there."

psb217 2y ago
My point was mainly that this claim: "we keep seeing LLMs building more robust approximations of real world models" is hard to evaluate without a well-formed definition of what it means to have a world model. Eg, a more restrictive definition of having a world model might include the ability to adapt reasoning to account for changes in the modeled world. Eg, an LLM with a proper model of chess by this definition would be able to quickly adapt to account for a rule change like "rooks and bishops can't move more than 4 squares at a time". I don't think there are any major walls either, but I think there are at least a few more plateaus we'll hit and spend time wandering around before finding the right direction for continued progress. Meanwhile, businesses/society/etc can work to catch up with the rapid progress made on the way to the current plateau.
- digging 2y ago
  I think we're largely in agreement then, actually. I'm seeing "world models" as a spectrum. World models aren't even consistent among adult humans. I claim LLMs are moving up that ladder, and whether or not they've crosses a threshold into "real" world models I do not actually claim to know. Of course I also agree that it's very possible, maybe even likely, that LLMs aren't able to cross that threshold. > this claim ... is hard to evaluate without a well-formed definition of what it means to have a world model Absolutely yes, but that only makes it more imperative that we're analyzing things critically, rigorously, and honestly. Again you and I may be on the same side here. Mainly my point was that asserting the intrinsic non-intelligence of LLMs is a very bad take, as it's not supported by evidence and, if anything, it contradicts some (admittedly very difficult to parse) evidence we do have that LLMs might be able to develop a general capability for constructing mental models of the world.

psb217 2y ago

My point was mainly that this claim: "we keep seeing LLMs building more robust approximations of real world models" is hard to evaluate without a well-formed definition of what it means to have a world model. Eg, a more restrictive definition of having a world model might include the ability to adapt reasoning to account for changes in the modeled world. Eg, an LLM with a proper model of chess by this definition would be able to quickly adapt to account for a rule change like "rooks and bishops can't move more than 4 squares at a time". I don't think there are any major walls either, but I think there are at least a few more plateaus we'll hit and spend time wandering around before finding the right direction for continued progress. Meanwhile, businesses/society/etc can work to catch up with the rapid progress made on the way to the current plateau.

digging 2y ago
I think we're largely in agreement then, actually. I'm seeing "world models" as a spectrum. World models aren't even consistent among adult humans. I claim LLMs are moving up that ladder, and whether or not they've crosses a threshold into "real" world models I do not actually claim to know. Of course I also agree that it's very possible, maybe even likely, that LLMs aren't able to cross that threshold. > this claim ... is hard to evaluate without a well-formed definition of what it means to have a world model Absolutely yes, but that only makes it more imperative that we're analyzing things critically, rigorously, and honestly. Again you and I may be on the same side here. Mainly my point was that asserting the intrinsic non-intelligence of LLMs is a very bad take, as it's not supported by evidence and, if anything, it contradicts some (admittedly very difficult to parse) evidence we do have that LLMs might be able to develop a general capability for constructing mental models of the world.