4 ms·

Surely it would immediately fail the test because which is more likely: the guy is a nutter, or there exists another universe with another turing machine which

by ldar15 15y ago

Surely it would immediately fail the test because which is more likely: the guy is a nutter, or there exists another universe with another turing machine which includes its states and all the states required for ours and all the states required for the 3^^^3 people.

bdr 15y ago
It doesn't matter which is more likely, only that there is some positive probability of the second option.
- ldar15 15y ago
  It doesn't matter which is more likely, only that there is some positive probability of the second option. No, it requires that the weighting (probability * consequence) is greater in the second case than the first. FTA: 1. But, small as this probability is, it isn't anywhere near as small as 3^^^^3 is large. 2. If the probabilities of various scenarios considered did not exactly cancel out, the AI's action in the case of Pascal's Mugging would be overwhelmingly dominated by whatever tiny differentials existed in the various tiny probabilities under which 3^^^^3 units of expected utility were actually at stake. I disagree with statement 1. If the author's definition of Occam's razor is believed by the AI, then the probability that the mugger is telling the truth is given by the inverse of the complexity of the system that incorporates our world, the other world and in addition the ability to simulate 3^^^^3 deaths. So it doesn't matter if X is 3^^^3 or 3^^^^^3, because the very probability of it being so is the inverse to the damage done. Our world: A Outer world only: B Temporary 3^^^3 death world: C Complexity when mugger telling truth = ABC Potential consequence: kC Weighting for mugger telling truth: C/ABC = 1/AB Complexity when mugger is lying = A. Weighting for mugger lying: 1/A Since A and B are staggeringly large numbers, the AI must choose 1/A over 1/AB. Then there's a whole bunch of other stuff, such as, even if the AI was certain that the mugger could carry out its threat, should the AI yield to threats of violence? Another question: if life can so trivially be created and destroyed, what does it matter? Can we trust the mugger? But the main issue, and this should be the response of anyone threatened with some claimed ability to cause harm and suffering in "the next world", is "prove it".

by ldar15 15y ago

bdr 15y ago

It doesn't matter which is more likely, only that there is some positive probability of the second option.

ldar15 15y ago
It doesn't matter which is more likely, only that there is some positive probability of the second option. No, it requires that the weighting (probability * consequence) is greater in the second case than the first. FTA: 1. But, small as this probability is, it isn't anywhere near as small as 3^^^^3 is large. 2. If the probabilities of various scenarios considered did not exactly cancel out, the AI's action in the case of Pascal's Mugging would be overwhelmingly dominated by whatever tiny differentials existed in the various tiny probabilities under which 3^^^^3 units of expected utility were actually at stake. I disagree with statement 1. If the author's definition of Occam's razor is believed by the AI, then the probability that the mugger is telling the truth is given by the inverse of the complexity of the system that incorporates our world, the other world and in addition the ability to simulate 3^^^^3 deaths. So it doesn't matter if X is 3^^^3 or 3^^^^^3, because the very probability of it being so is the inverse to the damage done. Our world: A Outer world only: B Temporary 3^^^3 death world: C Complexity when mugger telling truth = ABC Potential consequence: kC Weighting for mugger telling truth: C/ABC = 1/AB Complexity when mugger is lying = A. Weighting for mugger lying: 1/A Since A and B are staggeringly large numbers, the AI must choose 1/A over 1/AB. Then there's a whole bunch of other stuff, such as, even if the AI was certain that the mugger could carry out its threat, should the AI yield to threats of violence? Another question: if life can so trivially be created and destroyed, what does it matter? Can we trust the mugger? But the main issue, and this should be the response of anyone threatened with some claimed ability to cause harm and suffering in "the next world", is "prove it".