5 ms·

If we manage to create an artificial sentience then we need to legislate and protect their rights. AGI won't be a free lunch.

drdeca 5y ago
Just because something is AGI probably needn't imply that it is sapient. It might have no internal experience, no feeling of desire, only a utility function which it optimizes.
- krageon 5y ago
  This reasoning is why it will need rights. If it doesn't, people thinking these thoughts (which are impossible to answer, like they are for every other potential sapience. Not just AGI) will abuse that reasoning to create yet another creative modern form of slavery.
  - drdeca 5y ago
    I do agree that it probably makes sense to err on the side of being too quick to estimate that something is likely sapient than to err on the side of being too slow to do so. However, that doesn't make it impossible for a plan-producer to be both superhuman and non-sapient. Nor do I think the default is for an AGI to be sapient. (Though, possibly the default could be such that it isn't but in a way in which we couldn't be sufficiently confident that it isn't, and therefore be obligated to behave as if it is sapient) But, in the FOOM scenario, I would expect that "whether or not to treat it as if it has rights" would be the least of our worries. (the greatest of our worries would be, roughly, "whether it kills everyone".)
9wzYQbTYsAIc 5y ago
Yeesh, hard enough to get people to consider non-humans as people let alone a computer program.
- salawat 5y ago
  Or worse, the level of indirection in question where a computer program stands in for a work vertical that represents a large swathe of the population's livelihood. The unsexy part of CS. The Should I vs the could I.
unlikelymordant 5y ago
Honest question, why would we need to do that? They arent human or animal, it seems to me like saying we need to protect the rights of chairs or PCs. I personally think letting AI have e.g. property rights would be catastrophic if you can just spin up another one.
- drdeca 5y ago
  Whether an AGI should have rights depends on the specifics of the AGI. Being capable of acting in general environments, planning and taking actions in the world, etc. I think probably aren't in themselves enough for the agent to have moral patienthood. But, that's not to say that it wouldn't be eventually possible to make an AI which was a moral patient (like, suppose it was not just able to plan taking itself and its own planning capabilities into account, but was in addition, also sapient, in the sense of having an internal experience like that of our own. In this case (e.g. perhaps a computer emulation of a human mind) then I'm fairly confident that such an agent would be a moral patient. Such an agent being copyable as data would no more prevent them from being a moral patient than having a magic duplicator ray which could duplicate a living human including their body, would render all people not moral-patients, namely, not at all.) edit: that's not to say that it wouldn't be catastrophic, but, having such a duplicator ray might also be catastrophic for the same reasons, and the presence of such a magical duplicator array shouldn't render all people suddenly not moral-patients, and so the copyability of AIs shouldn't by itself prevent them from being moral patients either.
  - unlikelymordant 5y ago
    If there was a magical duplicator ray, governments would want to heavily regulate its use. A few million extra bodies would completely crash an economy, but might be good if you wanted to fight a war. I do think the duplicator ray is a great analogy. There would be a lot of tricky moral questions raised if such a technology existed. On the topic of a human mind in a computer, i dont think they should be afforded property rights either. If you can copy them into 10 bodies, and each one can own a house, then they are taking real resources from actual humans. This extends to owning other things too.

drdeca 5y ago

Just because something is AGI probably needn't imply that it is sapient. It might have no internal experience, no feeling of desire, only a utility function which it optimizes.

krageon 5y ago
This reasoning is why it will need rights. If it doesn't, people thinking these thoughts (which are impossible to answer, like they are for every other potential sapience. Not just AGI) will abuse that reasoning to create yet another creative modern form of slavery.
- drdeca 5y ago
  I do agree that it probably makes sense to err on the side of being too quick to estimate that something is likely sapient than to err on the side of being too slow to do so. However, that doesn't make it impossible for a plan-producer to be both superhuman and non-sapient. Nor do I think the default is for an AGI to be sapient. (Though, possibly the default could be such that it isn't but in a way in which we couldn't be sufficiently confident that it isn't, and therefore be obligated to behave as if it is sapient) But, in the FOOM scenario, I would expect that "whether or not to treat it as if it has rights" would be the least of our worries. (the greatest of our worries would be, roughly, "whether it kills everyone".)

9wzYQbTYsAIc 5y ago

Yeesh, hard enough to get people to consider non-humans as people let alone a computer program.

salawat 5y ago
Or worse, the level of indirection in question where a computer program stands in for a work vertical that represents a large swathe of the population's livelihood. The unsexy part of CS. The Should I vs the could I.

unlikelymordant 5y ago

Honest question, why would we need to do that? They arent human or animal, it seems to me like saying we need to protect the rights of chairs or PCs. I personally think letting AI have e.g. property rights would be catastrophic if you can just spin up another one.

drdeca 5y ago
Whether an AGI should have rights depends on the specifics of the AGI. Being capable of acting in general environments, planning and taking actions in the world, etc. I think probably aren't in themselves enough for the agent to have moral patienthood. But, that's not to say that it wouldn't be eventually possible to make an AI which was a moral patient (like, suppose it was not just able to plan taking itself and its own planning capabilities into account, but was in addition, also sapient, in the sense of having an internal experience like that of our own. In this case (e.g. perhaps a computer emulation of a human mind) then I'm fairly confident that such an agent would be a moral patient. Such an agent being copyable as data would no more prevent them from being a moral patient than having a magic duplicator ray which could duplicate a living human including their body, would render all people not moral-patients, namely, not at all.) edit: that's not to say that it wouldn't be catastrophic, but, having such a duplicator ray might also be catastrophic for the same reasons, and the presence of such a magical duplicator array shouldn't render all people suddenly not moral-patients, and so the copyability of AIs shouldn't by itself prevent them from being moral patients either.
- unlikelymordant 5y ago
  If there was a magical duplicator ray, governments would want to heavily regulate its use. A few million extra bodies would completely crash an economy, but might be good if you wanted to fight a war. I do think the duplicator ray is a great analogy. There would be a lot of tricky moral questions raised if such a technology existed. On the topic of a human mind in a computer, i dont think they should be afforded property rights either. If you can copy them into 10 bodies, and each one can own a house, then they are taking real resources from actual humans. This extends to owning other things too.