I don't think ethics applies unless the AI actually has conscious experience. The trouble is that conscious experience is only detectable from the inside. So we

I don't think ethics applies unless the AI actually has conscious experience. The trouble is that conscious experience is only detectable from the inside. So we need to test it from the inside. Here's a way: Attain the technology to upload a human to hardware via the "ship of theseus" method, replacing a few neurons at a time with hardware that replicates the activities of the originals. But when you actually upload people, have them report their experiences as you go. Vary the order in which you replace parts. If people never report anything weird, then I might start to trust that the hardware really does support conscious experience. But if, say, you replace the visual cortex and people say they know where everything is but aren't actually experiencing visual qualia, then I'll take that as evidence that the hardware does not support conscious experience, and any AI based on that hardware is a philosophical zombie, replicating behaviors but not experiencing qualia. That will be my default assumption until we test it, because no matter how complicated the computer program, mathematically it's still a Turing machine, and I don't see how a Turing machine moving back and forth on a tape can end up having qualia.