3 ms·

>GPT can never be an intelligent agent because it’s not an agent For all the handwaving people like to do, this is the bottom line, and it's not clear to me w

>GPT can never be an intelligent agent because it’s not an agent For all the handwaving people like to do, this is the bottom line, and it's not clear to me why researchers haven't spent more time clearly formulating this, vs a lot of proving that various tasks and games don't actually require intelligence. As best I can formulate it, "AI" can't "escape" its programming, and just a neural network even less so. There is no mental model of the world to rationalize against, and if there was, it would be pre-defined or coded in some way that it still couldn't be escaped. Whatever tasks a ML model has been deliberately programmed with just have nothing in common with intelligence. Scaling doesnt change that. I subscribe to the idea that with ML we may have understood how some of the sensory mechanisms work, like visual recognition, but we're not actually closer to the intelligence part.

qrian 4y ago
The same author has a story describing how AI might escape: https://www.gwern.net/fiction/Clippy https://www.gwern.net/fiction/Clippy Not that I agree with the author but 'AIs are sandboxed and resourced constrained therefore safe' doesn't feel very convincing to me after reading that story.
d110af5ccf 4y ago
> "AI" can't "escape" its programming I don't think you can back that claim up. ML models as often used can't escape their programming. But I don't see any fundamental reason you can't loop one back on itself. Isn't this already done to some extent by using a model to assess its own output versus the current environmental state? Recurrent models combined with reinforcement learning actually look a lot like what you're claiming can't be done. > There is no mental model of the world to rationalize against There's a theory and some associated research about a model that performs well necessitating an internal model that accurately reflects the environment in which it operates. Or something like that. Sorry I can't recall the paper off the top of my head. > Whatever tasks a ML model has been deliberately programmed with Yeah, no, that's not the case. See the entire field of reinforcement learning.
- astrange 4y ago
  > Isn't this already done to some extent by using a model to assess its own output versus the current environmental state? Recurrent models combined with reinforcement learning actually look a lot like what you're claiming can't be done. Yes, you can do this. And I believe you could recreate human intelligence if you made something exactly like a human and put it in an environment exactly like a human - but for some reason the AGI theorists think AGI via AI research is scary and "N"GI via having children isn't. But GPT doesn't have those features, which is why it's not one of them. > Yeah, no, that's not the case. See the entire field of reinforcement learning. ML models (especially big ones) fluidly combine learning with memorizing, which is cool, but it's more like "compression" than "programming".

qrian 4y ago

The same author has a story describing how AI might escape: https://www.gwern.net/fiction/Clippy https://www.gwern.net/fiction/Clippy Not that I agree with the author but 'AIs are sandboxed and resourced constrained therefore safe' doesn't feel very convincing to me after reading that story.

d110af5ccf 4y ago

> "AI" can't "escape" its programming I don't think you can back that claim up. ML models as often used can't escape their programming. But I don't see any fundamental reason you can't loop one back on itself. Isn't this already done to some extent by using a model to assess its own output versus the current environmental state? Recurrent models combined with reinforcement learning actually look a lot like what you're claiming can't be done. > There is no mental model of the world to rationalize against There's a theory and some associated research about a model that performs well necessitating an internal model that accurately reflects the environment in which it operates. Or something like that. Sorry I can't recall the paper off the top of my head. > Whatever tasks a ML model has been deliberately programmed with Yeah, no, that's not the case. See the entire field of reinforcement learning.

astrange 4y ago
> Isn't this already done to some extent by using a model to assess its own output versus the current environmental state? Recurrent models combined with reinforcement learning actually look a lot like what you're claiming can't be done. Yes, you can do this. And I believe you could recreate human intelligence if you made something exactly like a human and put it in an environment exactly like a human - but for some reason the AGI theorists think AGI via AI research is scary and "N"GI via having children isn't. But GPT doesn't have those features, which is why it's not one of them. > Yeah, no, that's not the case. See the entire field of reinforcement learning. ML models (especially big ones) fluidly combine learning with memorizing, which is cool, but it's more like "compression" than "programming".