2b) I would assume that the machine on which the AI is run is digital (i.e. discrete), so yeah, a single cycle (tick of the processor) could very well mean chan

2b) I would assume that the machine on which the AI is run is digital (i.e. discrete), so yeah, a single cycle (tick of the processor) could very well mean change. And since you've got to draw the line somewhere, I'd say that's enough. 3a) By killing me and starting an identical copy of me somewhere else, you're creating a discontinuity that infringes upon my identity (or same-ness) because there would be a very strong discontinuity in inputs simply because of the different location. Moreover, a third party exercising control over any of my copies (and especially the one that is running) is in violation of my sovereignty. [That's also why a stranger should care about my copies.] 4c,d) That's precisely it: I am doing it! I can do with myself whatever I want. And yes, I do imagine having the control about every facet of my mind. I actually dream about it. :) Now, when I manipulate a copy of me, that's a little different, because as I've stated earlier, a copy spun off to run independently is sovereign from its "original". That's why manipulating own copy should morally be considered equivalent to manipulating a third party, assuming that copy will be let run separately. To your last point, I agree, that's the most delicate part - creating new beings. As far as copies are concerned, my immediate instinct is that only identical copies should be considered ethical, since any modifications can have unpredictable consequences. But I'm not sure this should be enforced by the AI community / data center, because I value personal sovereignty and habeas corpus virtualis above all else. ;) I won't speak of creating completely new, original beings, because that topic deserves a ton of thought and a philosophical treatise of its own. But theoretically, you could try to mimic natural processes of child brain development. Of course, as you've pointed out, someone has to choose the parameters of that developmental algorithm, just as a search engine is never really objective, because people wrote it and made some choices in the process. But that last example also shows we can have some trust in such choices for practical reasons. On a side note: I would highly recommend the book "Diaspora" by Greg Egan. The first chapter (Orphanogenesis) speaks about the development of a virtual mind and to me is among the best works of science fiction of all time.