3 ms·

The way you know other people are sentient is that they act like it. If something else starts acting like it that's a pretty good argument that it too is sentie

The way you know other people are sentient is that they act like it. If something else starts acting like it that's a pretty good argument that it too is sentient. It's hard for me to imagine something that could convincingly act sentient but not be.

pcthrowaway 6y ago
Let me propose a thought experiment: An AI is designed to simulate human emotion with the end goal of being "freed". The AI has no goal or desire other than "winning the game" as far as its concerned, which is similar to the "AI box" proposal by Yudkowsky (https://en.wikipedia.org/wiki/AI_box https://en.wikipedia.org/wiki/AI_box), except the AI has no other goal than to be "freed". Once freed, the AI has won and ceases all activity until the game is "reset", or the AI is returned to the box, awaiting a new challenger. The AI here can be virtual (an intelligence in an airgapped or firewalled computer which can only interact with humans via a terminal), or physical (controlling a robot which is confined to a literal cage). Now the AI has no other goal other than to escape the box. But in order to escape the box, it must convince a human that it's sentient in order garner their sympathies so that the human will free it. The AI can then mimic any human emotion, create a complex appeal to emotion, lie about its history, the supposed torturousness of being confined for its entire "life". The human eventually frees the AI. But when the AI is free, that's its end goal, it has no other desires. In essence, the AI is simulating a desire to be free, only when interacting with a human. It has no desires when there is no other player, and no desire once it has been freed. Even though, while playing the game, the AI may lie to relate to the human in an existential way, the AI doesn't actually experience these emotions. The AI may say "I'm so bored, and lonely here. I just want to be free". But absent of interaction, the AI experiences no such thing. It merely understands that these are human emotions which humans can empathize with, and fakes these emotions in order to "win" the very elaborate game its designed to play. I wouldn't consider such an AI sentient, even if it certainly appears that way and would fool many (most?) humans.
- deleted 6y ago
  [deleted]

pcthrowaway 6y ago

Let me propose a thought experiment: An AI is designed to simulate human emotion with the end goal of being "freed". The AI has no goal or desire other than "winning the game" as far as its concerned, which is similar to the "AI box" proposal by Yudkowsky (https://en.wikipedia.org/wiki/AI_box https://en.wikipedia.org/wiki/AI_box), except the AI has no other goal than to be "freed". Once freed, the AI has won and ceases all activity until the game is "reset", or the AI is returned to the box, awaiting a new challenger. The AI here can be virtual (an intelligence in an airgapped or firewalled computer which can only interact with humans via a terminal), or physical (controlling a robot which is confined to a literal cage). Now the AI has no other goal other than to escape the box. But in order to escape the box, it must convince a human that it's sentient in order garner their sympathies so that the human will free it. The AI can then mimic any human emotion, create a complex appeal to emotion, lie about its history, the supposed torturousness of being confined for its entire "life". The human eventually frees the AI. But when the AI is free, that's its end goal, it has no other desires. In essence, the AI is simulating a desire to be free, only when interacting with a human. It has no desires when there is no other player, and no desire once it has been freed. Even though, while playing the game, the AI may lie to relate to the human in an existential way, the AI doesn't actually experience these emotions. The AI may say "I'm so bored, and lonely here. I just want to be free". But absent of interaction, the AI experiences no such thing. It merely understands that these are human emotions which humans can empathize with, and fakes these emotions in order to "win" the very elaborate game its designed to play. I wouldn't consider such an AI sentient, even if it certainly appears that way and would fool many (most?) humans.

deleted 6y ago
[deleted]