3 ms·

I agree on the point about it being real (and I'm fully on board that AGI will happen, while agreeing with Marcus' scepticism about current approaches), but in

I agree on the point about it being real (and I'm fully on board that AGI will happen, while agreeing with Marcus' scepticism about current approaches), but in that case (the intelligence matching ours) it would be obscene to use it as a tool; it's a sentient being with (not soon afterwards I hope) the same rights as us.

layer8 4y ago
Sentience and intelligence are quite orthogonal to each other.
- mellosouls 4y ago
  Debatable, and irrelevant in the case of AGI which would have both by definition.
  - layer8 4y ago
    It’s AGI, not AGS. There is no implication that a system exhibiting human-level intelligence has any level of sentience or consciousness.
    - mellosouls 4y ago
      Outside of philosophical zombie thought experiments, there is not only the implication, but the necessity that a system exhibiting sentience and consciousness (via human-level intelligence) should be assumed to have it.
      - layer8 4y ago
        Exhibiting human-level intelligence does not imply exhibiting sentience or consciousness. This is not a Turing test. You could have an extremely intelligent AI, more intelligent than any human, which however doesn’t feel anything and doesn’t have any experience of self (i.e. consciousness). P-zombies refer to a simulation of humans, and I agree that those wouldn’t actually be zombies, and would have consciousness and sentience. An AGI, however, does not aim to be anything like a human, except for having a level of intelligence that is at least comparable.
        mellosouls 4y ago
        I don't accept the Turing Test is relevant for anything other than judging the Turing Test. I also don't accept it is remotely likely any intelligence indistinguishable from humans in quality will lack sentience and consciousness. That's the whole point in calling out current AI paths of research, they don't address that, they just move closer and closer to your Turing Test, which doesn't interest me. Anyway, as we seem to be talking past each other, I'll agree to differ on whether AGI that matches human level capabilities in all aspects can exist without sentience and whatever else we are differing on.

layer8 4y ago

Sentience and intelligence are quite orthogonal to each other.

mellosouls 4y ago
Debatable, and irrelevant in the case of AGI which would have both by definition.
- layer8 4y ago
  It’s AGI, not AGS. There is no implication that a system exhibiting human-level intelligence has any level of sentience or consciousness.
  - mellosouls 4y ago
    Outside of philosophical zombie thought experiments, there is not only the implication, but the necessity that a system exhibiting sentience and consciousness (via human-level intelligence) should be assumed to have it.
    - layer8 4y ago
      Exhibiting human-level intelligence does not imply exhibiting sentience or consciousness. This is not a Turing test. You could have an extremely intelligent AI, more intelligent than any human, which however doesn’t feel anything and doesn’t have any experience of self (i.e. consciousness). P-zombies refer to a simulation of humans, and I agree that those wouldn’t actually be zombies, and would have consciousness and sentience. An AGI, however, does not aim to be anything like a human, except for having a level of intelligence that is at least comparable.
      - mellosouls 4y ago
        I don't accept the Turing Test is relevant for anything other than judging the Turing Test. I also don't accept it is remotely likely any intelligence indistinguishable from humans in quality will lack sentience and consciousness. That's the whole point in calling out current AI paths of research, they don't address that, they just move closer and closer to your Turing Test, which doesn't interest me. Anyway, as we seem to be talking past each other, I'll agree to differ on whether AGI that matches human level capabilities in all aspects can exist without sentience and whatever else we are differing on.