A while ago, this article was on HN: http://www.theatlantic.com/technology/archive/12/03/were-underestimating-the-risk-of-human-extinction/253821/ http://www.th

A while ago, this article was on HN: http://www.theatlantic.com/technology/archive/12/03/were-underestimating-the-risk-of-human-extinction/253821/ http://www.theatlantic.com/technology/archive/12/03/were-und... The article is based on an interview with Nick Bostrom, a philosophy professor at Oxford. His thoughts on the computer simulation scenario are very interesting. I've reproduced them below, but I recommend reading the whole interview; it's very fascinating. From the interview: "Can you explain the simulation argument, and how it presents a very particular existential risk? Bostrom: The simulation argument addresses whether we are in fact living in a simulation as opposed to some basement level physical reality. It tries to show that at least one of three propositions is true, but it doesn't tell us which one. Those three are: 1) Almost all civilizations like ours go extinct before reaching technological maturity. 2) Almost all technologically mature civilizations lose interest in creating ancestor simulations: computer simulations detailed enough that the simulated minds within them would be conscious. 3) We're almost certainly living in a computer simulation. The full argument requires sophisticated probabilistic reasoning, but the basic argument is fairly easy to grasp without resorting to mathematics. Suppose that the first proposition is false, which would mean that some significant portion of civilizations at our stage eventually reach technological maturity. Suppose that the second proposition is also false, which would mean that some significant fraction of those (technologically mature) civilizations retain an interest in using some non-negligible fraction of their resources for the purpose of creating these ancestor simulations. You can then show that it would be possible for a technologically mature civilization to create astronomical numbers of these simulations. So if this significant fraction of civilizations made it through to this stage where they decided to use their capabilities to create these ancestor simulations, then there would be many more simulations created than there are original histories, meaning that almost all observers with our types of experiences would be living in simulations. Going back to the observation selection effect, if almost all kinds of observers with our kinds of experiences are living in simulations, then we should think that we are living in a simulation, that we are one of the typical observers, rather than one of the rare, exceptional basic level reality observers. The connection to existential risk is twofold. First, the first of those three possibilities, that almost all civilizations like ours go extinct before reaching technological maturity obviously bears directly on how much existential risk we face. If proposition 1 is true then the obvious implication is that we will succumb to an existential catastrophe before reaching technological maturity. The other relationship with existential risk has to do with proposition 3: if we are living in a computer simulation then there are certain exotic ways in which we might experience an existential catastrophe which we wouldn't fear if we are living in basement level physical reality. The simulation could be shut off, for instance. Or there might be other kinds of interventions in our simulated reality."