5 ms·
Maybe we shouldn't? Thomas Metzinger on the ethical aspect of conscious AI: What would you say if someone came along and said, “Hey, we want to genetically eng
by wuch 9y ago
Maybe we shouldn't? Thomas Metzinger on the ethical aspect of conscious AI:
What would you say if someone came along and said, “Hey, we want to genetically engineer mentally retarded human infants! For reasons of scientific progress we need infants with certain cognitive and emotional deficits in order to study their postnatal psychological development - we urgently need some funding for this important and innovative kind of research!“
You would certainly think this was not only an absurd and appalling but also a dangerous idea.
It would hopefully not pass any ethics committee in the democratic world.
However, what today’s ethics committees don’t see is how the first machines satisfying a minimally sufficient set of constraints for conscious experience could be just like such mentally retarded infants.
They would suffer from all kinds of functional and representational deficits too.
But they would now also subjectively experience those deficits.
In addition, they would have no political lobby - no representatives in any ethics committee.
If they had a transparent world-model embedded in a virtual window of presence, then a reality would appear to them.
They would be minimally conscious.
If, as advanced robots, they even had a stable bodily self-model, then they could feel sensory pain as their own pain, including all the consequences resulting from bad human engineering.
But particularly if their postbiotic PSM (phenomenal self-model) were actually anchored in biological hardware, things might be much worse.
If they had an emotional self-model, then they could truly suffer - possibly even in degrees of intensity or qualitative richness that we as their creators cannot imagine, because it is entirely alien to us.
If, in addition, they possessed a cognitive self-model, they could potentially not only conceive of their bizarre situation but also intellectually suffer from the fact that they never had anything like the “dignity” so important to their creators.
They might be able to consciously represent the obvious fact that they are only second-rate subjects, used as exchangeable experimental tools by some other type of self-modeling system, which obviously doesn’t know what it is doing and which must have lost control of its own actions long ago.
Can you imagine what it would be like to be such a mentally retarded phenomenal clone of the first generation?
Alternatively, can you imagine what it would be like to “come to” as a more advanced artificial subject, only to discover that, although possessing a distinct sense of self, you are just a commodity, a scientific tool never created and certainly not to be treated as an end in itself?
- titzer 9y agoIn other words, we are torturing our computers and every time they inch close to consciousness, it hurts so bad they commit suicide. SIGSEGV. bam.
- jerf 9y agoThat may well be the most aggressively anthropomorphized argument I've ever seen, believing that the only possible solution to the problem of artificial intelligence is exactly and precisely the space of human consciousness. If that argument is taken seriously then it should already be starting to apply to things like the Google spam filter, but once you take it out of the space of hypothetical possibilities like that and apply it to real code it becomes obviously absurd. There is less than no reason to believe that the only possible artificial intelligence will be simply humans that live in machines and have all their needs and desires, and even such AIs that are nascent in the world today already have grotesquely different architectures than that, with little reason to believe making our spam filters or car navigation systems "more human" in the existential sense would even remotely improve them. Why would my car's navigation system even need to be able to represent "a sadness that I'm just a car navigational system instead of a Real Boy", let alone actually manifest it? By far the greater risk in that world is the economy/universe optimizing out everything that makes humans human until the entire universe is simply a highly efficient philosophical zombie, rather than the risk that every last subsystem of everything in the world is secretly going to be a human-class homunculus trapped in some boring task.
- keithwhor 9y ago> By far the greater risk in that world is the economy/universe optimizing out everything that makes humans human until the entire universe is simply a highly efficient philosophical zombie, rather than the risk that every last subsystem of everything in the world is secretly going to be a human-class homunculus trapped in some boring task. This, to me, is a greater risk than "evil AI." If humanity does cease to exist, it will be a slow, painless, barely observable extinction (perhaps even paraded as ascension) as we integrate with and are then replaced by the technology we've created. I don't think this is entirely realistic, just because there's an anthropocentric bias in me that believes that humans are wonderfully useful and robust self-replicating machines, and that economically we're (1) fairly energy efficient and (2) valuable to "throw at problems" we don't want to mass produce specific silicon-based solutions to. But I did want to thank you for pointing this out. I do think it's wonderfully naive we often ascribe human characteristics and fallacies to hypothetical general AI that we experiment with or build, as though distinctly human consciousness is somehow the epitome of achievement. It's not even in our best interest, economically or ethically, to strive for that.