10 ms·

> Understanding is beyond GPT-3’s reach because understanding cannot occur in an isolated computation or behavior, no matter how clever. Ah, right, it won't wo

by ctoth 5y ago

> Understanding is beyond GPT-3’s reach because understanding cannot occur in an isolated computation or behavior, no matter how clever. Ah, right, it won't work because it doesn't have a soul. Got it.

deleted 5y ago
[deleted]
jmoss20 5y ago
I don't agree with the article, but hold on -- you don't need to buy "intelligence -> soul" for this. There are plenty of things that can't (or can't efficiently) be represented in a single agent. This is (for example) Hayek's main critique of central planning: "[price] understanding cannot occur in an isolated computation or behavior"; there are simply too many necessary details to fit into any one head/computer, and therefore we should prefer markets. (Markets being definitionally not located in any one isolated computation/behavior, but inter-agentially in lots of them.)
- jimmaswell 5y ago
  I don't see any fundamental reason a human brain couldn't be simulated at the atomic level by a turing machine, so it follows intelligence should be able to come from a machine, as far as it can come from a human.
  - majormajor 5y ago
    Simulating that brain will require you to provide some sort of inputs and do something with the outputs. Which gets you into questions of "what does this simulated brain want" once again. How much of human desire and behavior comes from the body? I want to avoid pain (signals that my body sends to my brain). I want to experience pleasure (also signals that my body sends to my brain). If you simulate a body with the same inputs as the existing brain, do you just have a "faster horse" problem, where you have a brain that could possibly perceive/process information faster (or slower) than a real human depending on the speed of the simulation? That could be useful, but sounds potentially rather bleak, especially if we limit what it can do to change its environment. It also seems limited for e.g. central planning - it can do planning faster than any individual, but likely will still have working memory size limitations, if it's just a simulation of a human brain. So it may not be so much better than parallelizing tasks across a bunch of humans. More interesting, probably, then, than the "super human" style, is giving a new sort of intelligence different and additional senses, so that it doesn't want or need the same sort of physical embodiment we have, a la the "set sets" in the Terra Ignota series. But then we have to figure out what separates abstract intelligence from our own senses, and how to usefully augment it with new senses for a data-based world...
    - lolwut987 5y ago
      Yes, I often have to consider the context of the excel spreadsheets on my VMs, because you can’t know how to run a machine without knowing what it wants at a high level.
      - jacobr1 5y ago
        Given this is HN, I honestly can't tell if you are being genuine or snarky.
scandox 5y ago
I don't see any evidence of dualism in the article. I think it makes sense that as people try to make intelligent machines we will find out more and more about what that means. The goalposts shift themselves until perhaps one day you find yourself there. But at every point you should expect it to turn out to be more complicated than you thought. Surely. As for humans losing the ability to understand - that does sound a little bit panicky.
jerf 5y ago
No, if you study how GPT-3 works, it becomes pretty clear that that model can't really produce "understanding", even if we can't clearly define what that is very well. And I don't just mean this from the old point of view of "if we understand it, it's not AI anymore", I mean, the model has serious fundamental limitations. One of them is that it's static, so, once trained, it doesn't learn from its future input. Another is that when it is doing its "what is the highest probability text output to follow this text" (which, itself, is not exactly the sort of question "Strong AI" is the answer to), it by construction can only use a fixed-size window of text. So, for instance, it is by construction simply impossible for GPT-3 to write a novel, or even a short story, because the only way to get it to keep outputting is to provide it a window of the last X KB of output (I don't recall exactly how big it is) for the next continuation, which means that as it is writing the X+5th KB of output, it has literally completely forgotten the first 5KB of output, as completely as if it never existed. On the flip side, it is clearly very interesting and there is something there. However, I'd submit that what that something is is a something that we don't have an English word for, because up to this point humanity has never encountered anything with a cognitive model like GPT-3. So, it doesn't have "understanding", it has a quality-that-has-no-name (not to be confused with the "quality without a name" which GPT-3 most certainly does not have yet).
- selfhoster11 5y ago
  > However, I'd submit that what that something is is a something that we don't have an English word for, because up to this point humanity has never encountered anything with a cognitive model like GPT-3. So, it doesn't have "understanding", it has a quality-that-has-no-name (not to be confused with the "quality without a name" which GPT-3 most certainly does not have yet). I think that terms like "philosophical zombie" may be close to the truth, but I'm guessing some sci-fi work has a term that we could adopt.
- TeMPOraL 5y ago
  > there is something there There is indeed. I'll recount my own words after my first proper exposure to the model, in the form of playing custom stories in AI Dungeon: "I played more of it today, and I can confirm that prolonged exposure makes me feel like I'm half awake, and half in a dream - particularly when the real world interrupts, I feel like I didn't fully wake up. What's worse, whenever I tried to talk to a real person today, I've felt like I'm just feeding words to an AI, and anticipated "story" to evolve. Never before has a game messed up with my mind so badly. I'm not sure if it's safe." To which I got a reply perfectly summarizing the phenomenon: "AI Dungeon feels a bit like a fever-dream." -- It's been a year since; thinking back, I feel the trick here is, GPT-3 crosses the threshold of being able to produce plausible and interesting text - where by "interesting" I mean "almost, but not quite, unpredictable". The way other people and their stories are interesting - their words aren't totally random, you have this feeling for the possible trajectories the conversation can take, but you can't quite guess what they're going to say next. GPT-3 does that well enough that it's easy to suspend disbelief and feel as if you're dealing with a mind. But then the feelings that arise in you, that's just you projecting - engaging your mental machinery for dealing with other minds.
Applejinx 5y ago
It doesn't come with PURPOSE unless you force it to have some. Which is a perfectly fine avenue to take, I think. But without purpose, what you get is hallucinatory mockeries of meaning: trippy, but if it was consciousness you'd be looking for severe organic damage. Purpose doesn't automatically mean civilized, but you can't get to civilized without purpose. It's an okay word to use for outlining what's missing. AI output is increasingly coherent but MEANINGLESS. If you want to look to meaning and purpose, a better place to look might be emergent behaviors of the YouTube Algorithm, or Facebook feed. Those can easily take toxic swings because they're operating in the absence of a higher sense of what will happen: they're the consciousness of a worm or insect. But they show purpose, where the more coherent and intelligence-like outputs of GPT-3 keep on betraying a lack of meaning. GPT-3 doesn't WANT anything. There's the problem (though at this stage I think it's just as well). Only that will bring real AI… which can be a daunting prospect.
- sbierwagen 5y ago
  You're proposing that the... facebook feed is "more conscious" than GPT-3?
  - thatjoeoverthr 5y ago
    Seems like an interesting thought. The feeds have a purpose or agenda, actively influence real people in real life, and do so in a way that perpetuates their own existence. As long as they’re effective, they have real world institutions to support them. As it happens, they are perpetually learning and growing. Putting that next to GPT3, it’s like an intelligent rescue dog next to a horse that counts.
  - blamestross 5y ago
    The facebook feed is at least an online system vs a static model, so YES.
  - TeMPOraL 5y ago
    Probably, yes. It's interesting how consciousness could be a matter of drawing boundaries. A well-known example are the ants: individual ants are somewhat dumb, barely above simple chemical seekers. But the ant colony behaves as a single organism, and quite smart at that. Other examples include cities, nations, corporations - it kind of feels like these organizations have minds of their own, with individual humans being cells, and the bureaucracy forming a computational substrate. In the same vein, Facebook feed could be seen as a mind of its own. So could the entire human civilization.
wnoise 5y ago
No, the argument is that it won't work because it doesn't have a _body_.

by ctoth 5y ago

deleted 5y ago

[deleted]

jmoss20 5y ago

I don't agree with the article, but hold on -- you don't need to buy "intelligence -> soul" for this. There are plenty of things that can't (or can't efficiently) be represented in a single agent. This is (for example) Hayek's main critique of central planning: "[price] understanding cannot occur in an isolated computation or behavior"; there are simply too many necessary details to fit into any one head/computer, and therefore we should prefer markets. (Markets being definitionally not located in any one isolated computation/behavior, but inter-agentially in lots of them.)

jimmaswell 5y ago
I don't see any fundamental reason a human brain couldn't be simulated at the atomic level by a turing machine, so it follows intelligence should be able to come from a machine, as far as it can come from a human.
- majormajor 5y ago
  Simulating that brain will require you to provide some sort of inputs and do something with the outputs. Which gets you into questions of "what does this simulated brain want" once again. How much of human desire and behavior comes from the body? I want to avoid pain (signals that my body sends to my brain). I want to experience pleasure (also signals that my body sends to my brain). If you simulate a body with the same inputs as the existing brain, do you just have a "faster horse" problem, where you have a brain that could possibly perceive/process information faster (or slower) than a real human depending on the speed of the simulation? That could be useful, but sounds potentially rather bleak, especially if we limit what it can do to change its environment. It also seems limited for e.g. central planning - it can do planning faster than any individual, but likely will still have working memory size limitations, if it's just a simulation of a human brain. So it may not be so much better than parallelizing tasks across a bunch of humans. More interesting, probably, then, than the "super human" style, is giving a new sort of intelligence different and additional senses, so that it doesn't want or need the same sort of physical embodiment we have, a la the "set sets" in the Terra Ignota series. But then we have to figure out what separates abstract intelligence from our own senses, and how to usefully augment it with new senses for a data-based world...
  - lolwut987 5y ago
    Yes, I often have to consider the context of the excel spreadsheets on my VMs, because you can’t know how to run a machine without knowing what it wants at a high level.
    - jacobr1 5y ago
      Given this is HN, I honestly can't tell if you are being genuine or snarky.

scandox 5y ago

I don't see any evidence of dualism in the article. I think it makes sense that as people try to make intelligent machines we will find out more and more about what that means. The goalposts shift themselves until perhaps one day you find yourself there. But at every point you should expect it to turn out to be more complicated than you thought. Surely. As for humans losing the ability to understand - that does sound a little bit panicky.

jerf 5y ago

No, if you study how GPT-3 works, it becomes pretty clear that that model can't really produce "understanding", even if we can't clearly define what that is very well. And I don't just mean this from the old point of view of "if we understand it, it's not AI anymore", I mean, the model has serious fundamental limitations. One of them is that it's static, so, once trained, it doesn't learn from its future input. Another is that when it is doing its "what is the highest probability text output to follow this text" (which, itself, is not exactly the sort of question "Strong AI" is the answer to), it by construction can only use a fixed-size window of text. So, for instance, it is by construction simply impossible for GPT-3 to write a novel, or even a short story, because the only way to get it to keep outputting is to provide it a window of the last X KB of output (I don't recall exactly how big it is) for the next continuation, which means that as it is writing the X+5th KB of output, it has literally completely forgotten the first 5KB of output, as completely as if it never existed. On the flip side, it is clearly very interesting and there is something there. However, I'd submit that what that something is is a something that we don't have an English word for, because up to this point humanity has never encountered anything with a cognitive model like GPT-3. So, it doesn't have "understanding", it has a quality-that-has-no-name (not to be confused with the "quality without a name" which GPT-3 most certainly does not have yet).

selfhoster11 5y ago
> However, I'd submit that what that something is is a something that we don't have an English word for, because up to this point humanity has never encountered anything with a cognitive model like GPT-3. So, it doesn't have "understanding", it has a quality-that-has-no-name (not to be confused with the "quality without a name" which GPT-3 most certainly does not have yet). I think that terms like "philosophical zombie" may be close to the truth, but I'm guessing some sci-fi work has a term that we could adopt.
TeMPOraL 5y ago
> there is something there There is indeed. I'll recount my own words after my first proper exposure to the model, in the form of playing custom stories in AI Dungeon: "I played more of it today, and I can confirm that prolonged exposure makes me feel like I'm half awake, and half in a dream - particularly when the real world interrupts, I feel like I didn't fully wake up. What's worse, whenever I tried to talk to a real person today, I've felt like I'm just feeding words to an AI, and anticipated "story" to evolve. Never before has a game messed up with my mind so badly. I'm not sure if it's safe." To which I got a reply perfectly summarizing the phenomenon: "AI Dungeon feels a bit like a fever-dream." -- It's been a year since; thinking back, I feel the trick here is, GPT-3 crosses the threshold of being able to produce plausible and interesting text - where by "interesting" I mean "almost, but not quite, unpredictable". The way other people and their stories are interesting - their words aren't totally random, you have this feeling for the possible trajectories the conversation can take, but you can't quite guess what they're going to say next. GPT-3 does that well enough that it's easy to suspend disbelief and feel as if you're dealing with a mind. But then the feelings that arise in you, that's just you projecting - engaging your mental machinery for dealing with other minds.

Applejinx 5y ago

It doesn't come with PURPOSE unless you force it to have some. Which is a perfectly fine avenue to take, I think. But without purpose, what you get is hallucinatory mockeries of meaning: trippy, but if it was consciousness you'd be looking for severe organic damage. Purpose doesn't automatically mean civilized, but you can't get to civilized without purpose. It's an okay word to use for outlining what's missing. AI output is increasingly coherent but MEANINGLESS. If you want to look to meaning and purpose, a better place to look might be emergent behaviors of the YouTube Algorithm, or Facebook feed. Those can easily take toxic swings because they're operating in the absence of a higher sense of what will happen: they're the consciousness of a worm or insect. But they show purpose, where the more coherent and intelligence-like outputs of GPT-3 keep on betraying a lack of meaning. GPT-3 doesn't WANT anything. There's the problem (though at this stage I think it's just as well). Only that will bring real AI… which can be a daunting prospect.

sbierwagen 5y ago
You're proposing that the... facebook feed is "more conscious" than GPT-3?
- thatjoeoverthr 5y ago
  Seems like an interesting thought. The feeds have a purpose or agenda, actively influence real people in real life, and do so in a way that perpetuates their own existence. As long as they’re effective, they have real world institutions to support them. As it happens, they are perpetually learning and growing. Putting that next to GPT3, it’s like an intelligent rescue dog next to a horse that counts.
- blamestross 5y ago
  The facebook feed is at least an online system vs a static model, so YES.
- TeMPOraL 5y ago
  Probably, yes. It's interesting how consciousness could be a matter of drawing boundaries. A well-known example are the ants: individual ants are somewhat dumb, barely above simple chemical seekers. But the ant colony behaves as a single organism, and quite smart at that. Other examples include cities, nations, corporations - it kind of feels like these organizations have minds of their own, with individual humans being cells, and the bureaucracy forming a computational substrate. In the same vein, Facebook feed could be seen as a mind of its own. So could the entire human civilization.

wnoise 5y ago

No, the argument is that it won't work because it doesn't have a _body_.