3 ms·

FWIW, I think you might be misreading the audience of the post. The majority of readers at LW subscribe to not-directly-causal decision theories under which th

FWIW, I think you might be misreading the audience of the post. The majority of readers at LW subscribe to not-directly-causal decision theories under which they would cooperate with another rational agent in the one-shot PD; it may distract you, but not most. I agree with you, however, that it isn't the greatest analogy.

ZeroGravitas 16y ago
Are you talking about Superrationality (http://en.wikipedia.org/wiki/Superrationality http://en.wikipedia.org/wiki/Superrationality)? Still strikes me as changing the rules and then pretending you've said something new about the original unchanged form. Kind of like saying you'd choose white in the "You must choose black game". I'm a bit disappointed if this kind of sloppy logic/mysticism is popular on LessWrong.
- mquander 16y ago
  Yes, I'm talking about superrationality and decision theories that accommodate it, like evidential decision theory (you should make decisions that serve as evidence for the outcome you desire; e.g. you should cooperate, because doing so is evidence that a corresponding EDT-using rational partner will also cooperate) and Eliezer's pet timeless decision theory (roughly, your decisions should be treated as if you were making the decision for all sufficiently similar minds in the same situation.)
  - ZeroGravitas 16y ago
    Thanks for the pointers, I found some stuff on their wiki with these keywords and it looks like some interesting reading. However, from a quick look through this interpretation makes it an even worse analogy, even if you overlook not flagging it up as an unconventional take on PD for outsiders.
    - mquander 16y ago
      Yeah, I agree! All I took away from the analogy was "you had in mind one thing that you wanted to do, accidentally did the other thing, and suffered for it."
- ZeroGravitas 16y ago
  Googling for mentions of this on LessWrong it seems Elizier agrees that the standard formulation is confusing but remains strangely coy about what a rational actor should/would do in response: http://lesswrong.com/lw/tn/the_true_prisoners_dilemma/ http://lesswrong.com/lw/tn/the_true_prisoners_dilemma/ Actually, in a follow-up he's clearer in a comment: "As someone who rejects defection as the inevitable rational solution to both the one-shot PD and the iterated PD, I'm interested in the inconsistency of those who accept defection as the rational equilibrium in the one-shot PD, but find excuses to reject it in the finitely iterated known-horizon PD." from http://lesswrong.com/lw/to/the_truly_iterated_prisoners_dilemma/ http://lesswrong.com/lw/to/the_truly_iterated_prisoners_dile... Haven't found an actual explanation of why he thinks co-operation is rational in a one-shot PD yet though. He's demonstrated that he understands the problem enough to describe why it's counter-intuitive, so I'm going to assume that whatever his reason is it's something a bit crazy, rather than just dumb.
  - pjscott 16y ago
    If two "rational" actors go into a one-shot prisoner's dilemma and both defect, then they're worse off than "irrational" actors who both cooperate. What the hell kind of rationality is that? If you're going up against an agent whom you think will defect, then sure, it's rational for you to defect as well -- but defecting is not always the right answer. I'm guessing that Eliezer's being relatively quiet about this because he's still working on a grand unified theory of how to approach such problems: http://wiki.lesswrong.com/wiki/Timeless_decision_theory http://wiki.lesswrong.com/wiki/Timeless_decision_theory
    - ZeroGravitas 16y ago
      It's rational as defined by economists i.e. nothing to do with rationality at all. Similarly, don't be fooled into believing that perfect competition is perfect or free markets are free. It's just jargon, shorthand for a bunch of assumptions about a model, almost none of which actually hold in real life but just might, if you squint the right way and bear all the caveats in mind, have some useful information to impart. from http://en.wikipedia.org/wiki/Rational_choice_theory http://en.wikipedia.org/wiki/Rational_choice_theory "The "rationality" described by rational choice theory is different from the colloquial and most philosophical uses of the word. For most people, "rationality" means "sane," "in a thoughtful clear-headed manner," or knowing and doing what's healthy in the long term. Rational choice theory uses a specific and narrower definition of "rationality" simply to mean that an individual acts as if balancing costs against benefits to arrive at action that maximizes personal advantage.[4] For example, this may involve kissing someone, cheating on a test, using cocaine, or murdering someone. In rational choice theory, all decisions, crazy or sane, are postulated as mimicking such a "rational" process." Also, note that co-operating with the other person gives the greatest total gain (i.e. adding yours and the partners) but the greatest individual gain is when one defects and the other tries to co-operate. So betraying the other is the best possible outcome, and he knows that, and knows you know that. That's why you can't trust them to co-operate, they have no incentive to do so.

ZeroGravitas 16y ago

Are you talking about Superrationality (http://en.wikipedia.org/wiki/Superrationality http://en.wikipedia.org/wiki/Superrationality)? Still strikes me as changing the rules and then pretending you've said something new about the original unchanged form. Kind of like saying you'd choose white in the "You must choose black game". I'm a bit disappointed if this kind of sloppy logic/mysticism is popular on LessWrong.

mquander 16y ago
Yes, I'm talking about superrationality and decision theories that accommodate it, like evidential decision theory (you should make decisions that serve as evidence for the outcome you desire; e.g. you should cooperate, because doing so is evidence that a corresponding EDT-using rational partner will also cooperate) and Eliezer's pet timeless decision theory (roughly, your decisions should be treated as if you were making the decision for all sufficiently similar minds in the same situation.)
- ZeroGravitas 16y ago
  Thanks for the pointers, I found some stuff on their wiki with these keywords and it looks like some interesting reading. However, from a quick look through this interpretation makes it an even worse analogy, even if you overlook not flagging it up as an unconventional take on PD for outsiders.
  - mquander 16y ago
    Yeah, I agree! All I took away from the analogy was "you had in mind one thing that you wanted to do, accidentally did the other thing, and suffered for it."
ZeroGravitas 16y ago
Googling for mentions of this on LessWrong it seems Elizier agrees that the standard formulation is confusing but remains strangely coy about what a rational actor should/would do in response: http://lesswrong.com/lw/tn/the_true_prisoners_dilemma/ http://lesswrong.com/lw/tn/the_true_prisoners_dilemma/ Actually, in a follow-up he's clearer in a comment: "As someone who rejects defection as the inevitable rational solution to both the one-shot PD and the iterated PD, I'm interested in the inconsistency of those who accept defection as the rational equilibrium in the one-shot PD, but find excuses to reject it in the finitely iterated known-horizon PD." from http://lesswrong.com/lw/to/the_truly_iterated_prisoners_dilemma/ http://lesswrong.com/lw/to/the_truly_iterated_prisoners_dile... Haven't found an actual explanation of why he thinks co-operation is rational in a one-shot PD yet though. He's demonstrated that he understands the problem enough to describe why it's counter-intuitive, so I'm going to assume that whatever his reason is it's something a bit crazy, rather than just dumb.
- pjscott 16y ago
  If two "rational" actors go into a one-shot prisoner's dilemma and both defect, then they're worse off than "irrational" actors who both cooperate. What the hell kind of rationality is that? If you're going up against an agent whom you think will defect, then sure, it's rational for you to defect as well -- but defecting is not always the right answer. I'm guessing that Eliezer's being relatively quiet about this because he's still working on a grand unified theory of how to approach such problems: http://wiki.lesswrong.com/wiki/Timeless_decision_theory http://wiki.lesswrong.com/wiki/Timeless_decision_theory
  - ZeroGravitas 16y ago
    It's rational as defined by economists i.e. nothing to do with rationality at all. Similarly, don't be fooled into believing that perfect competition is perfect or free markets are free. It's just jargon, shorthand for a bunch of assumptions about a model, almost none of which actually hold in real life but just might, if you squint the right way and bear all the caveats in mind, have some useful information to impart. from http://en.wikipedia.org/wiki/Rational_choice_theory http://en.wikipedia.org/wiki/Rational_choice_theory "The "rationality" described by rational choice theory is different from the colloquial and most philosophical uses of the word. For most people, "rationality" means "sane," "in a thoughtful clear-headed manner," or knowing and doing what's healthy in the long term. Rational choice theory uses a specific and narrower definition of "rationality" simply to mean that an individual acts as if balancing costs against benefits to arrive at action that maximizes personal advantage.[4] For example, this may involve kissing someone, cheating on a test, using cocaine, or murdering someone. In rational choice theory, all decisions, crazy or sane, are postulated as mimicking such a "rational" process." Also, note that co-operating with the other person gives the greatest total gain (i.e. adding yours and the partners) but the greatest individual gain is when one defects and the other tries to co-operate. So betraying the other is the best possible outcome, and he knows that, and knows you know that. That's why you can't trust them to co-operate, they have no incentive to do so.