6 ms·

2b) It never lived, it was a "frozen corpse", and since you can make copies at will, just having a copy doesn't constitute a new life until it's run. How long

2b) It never lived, it was a "frozen corpse", and since you can make copies at will, just having a copy doesn't constitute a new life until it's run. How long does it have to run before it's alive? If the interval is short enough then there will be no change in state whatsoever. So in what way does the mind state have to change in order for it to have lived? What if that change is so simple that you can calculate it in your head? Have you created a new life just by thinking about what will happen next in the simulation? By writing it down? 3a) Since there's zero information difference, I'd say it is OK. So you would have no objection to being killed if there was an identical copy of you in existence somewhere? Why does that matter? Why do you care more about this copy of you than you do about anybody else? Why should a stranger care that there is at least one copy of you? 4c,d) Very interesting. On the one hand, in case of many modifications, you can't really foresee the consequences, so all should be considered unethical to prevent unintended suffering. But you are modifying your own mind all the time, by forming memories and learning skills. To some degree, you choose what you think and experience. Imagine that you have ultimate control over every facet of your mind. You live in a computer, so there is no "natural" mental life. You must choose exactly how your mind will progress. If you make a frozen copy of yourself, it hasn't diverged yet, so you can modify it however you like, since it's you. And if new, unique people are to be created at all, someone will have to choose how their mind works. There is no longer a "natural" reproductive process to decide this. How can choices be made in the new person's interests when it is those very interests that are being chosen?

hastur 15y ago
2b) I would assume that the machine on which the AI is run is digital (i.e. discrete), so yeah, a single cycle (tick of the processor) could very well mean change. And since you've got to draw the line somewhere, I'd say that's enough. 3a) By killing me and starting an identical copy of me somewhere else, you're creating a discontinuity that infringes upon my identity (or same-ness) because there would be a very strong discontinuity in inputs simply because of the different location. Moreover, a third party exercising control over any of my copies (and especially the one that is running) is in violation of my sovereignty. [That's also why a stranger should care about my copies.] 4c,d) That's precisely it: I am doing it! I can do with myself whatever I want. And yes, I do imagine having the control about every facet of my mind. I actually dream about it. :) Now, when I manipulate a copy of me, that's a little different, because as I've stated earlier, a copy spun off to run independently is sovereign from its "original". That's why manipulating own copy should morally be considered equivalent to manipulating a third party, assuming that copy will be let run separately. To your last point, I agree, that's the most delicate part - creating new beings. As far as copies are concerned, my immediate instinct is that only identical copies should be considered ethical, since any modifications can have unpredictable consequences. But I'm not sure this should be enforced by the AI community / data center, because I value personal sovereignty and habeas corpus virtualis above all else. ;) I won't speak of creating completely new, original beings, because that topic deserves a ton of thought and a philosophical treatise of its own. But theoretically, you could try to mimic natural processes of child brain development. Of course, as you've pointed out, someone has to choose the parameters of that developmental algorithm, just as a search engine is never really objective, because people wrote it and made some choices in the process. But that last example also shows we can have some trust in such choices for practical reasons. On a side note: I would highly recommend the book "Diaspora" by Greg Egan. The first chapter (Orphanogenesis) speaks about the development of a virtual mind and to me is among the best works of science fiction of all time.

hastur 15y ago

2b) I would assume that the machine on which the AI is run is digital (i.e. discrete), so yeah, a single cycle (tick of the processor) could very well mean change. And since you've got to draw the line somewhere, I'd say that's enough. 3a) By killing me and starting an identical copy of me somewhere else, you're creating a discontinuity that infringes upon my identity (or same-ness) because there would be a very strong discontinuity in inputs simply because of the different location. Moreover, a third party exercising control over any of my copies (and especially the one that is running) is in violation of my sovereignty. [That's also why a stranger should care about my copies.] 4c,d) That's precisely it: I am doing it! I can do with myself whatever I want. And yes, I do imagine having the control about every facet of my mind. I actually dream about it. :) Now, when I manipulate a copy of me, that's a little different, because as I've stated earlier, a copy spun off to run independently is sovereign from its "original". That's why manipulating own copy should morally be considered equivalent to manipulating a third party, assuming that copy will be let run separately. To your last point, I agree, that's the most delicate part - creating new beings. As far as copies are concerned, my immediate instinct is that only identical copies should be considered ethical, since any modifications can have unpredictable consequences. But I'm not sure this should be enforced by the AI community / data center, because I value personal sovereignty and habeas corpus virtualis above all else. ;) I won't speak of creating completely new, original beings, because that topic deserves a ton of thought and a philosophical treatise of its own. But theoretically, you could try to mimic natural processes of child brain development. Of course, as you've pointed out, someone has to choose the parameters of that developmental algorithm, just as a search engine is never really objective, because people wrote it and made some choices in the process. But that last example also shows we can have some trust in such choices for practical reasons. On a side note: I would highly recommend the book "Diaspora" by Greg Egan. The first chapter (Orphanogenesis) speaks about the development of a virtual mind and to me is among the best works of science fiction of all time.