4 ms·
I think the key word here is acausal? How can it coerce you in a way that you can’t just, be committed to not cooperating with, without first having a causal in
by drdeca 2y ago
I think the key word here is acausal? How can it coerce you in a way that you can’t just, be committed to not cooperating with, without first having a causal influence on you?
Acausal blackmail only works if one agent U predicts the likely future (or, otherwise not-yet-having-causal-influence) existence of another agent V, who would take actions so that if U’s actions aren’t in accordance with V’s preferences, then V’s actions will do harm to U(‘s interests) (eventually). But, this only works if U predicts the likely possible existence of V and V’s blackmail.
If V is having a causal influence of U, in order to do the blackmail, that’s just ordinary coercion. And, if U doesn’t anticipate the existence (and preferences) of V, then U won’t cooperate with any such attempts at acausal blackmail.
(… is “blackmail” really the right word? It isn’t like there’s a threat to reveal a secret, which I typically think of as central to the notion of blackmail.)
- khafra 2y agoSomething can be "acausal," and still change the probability you assign to various outcomes in your future event space. The classic example is in the paper "Defeating Dr. Evil with self-locating belief": https://www.princeton.edu/~adame/papers/drevil/drevil.pdf https://www.princeton.edu/~adame/papers/drevil/drevil.pdf
- throwanem 2y agoOh, good grief. I don't agree with how the other nearby commenter said it, but I do agree with what they said, especially in light of the nearby context on Yudkowsky that is also novel to me. This all evinces a vast and vastly unbalanced excess of cleverness.
- drdeca 2y agoEven if it would be rational to change the probabilities one assigns to one’s future event space, that doesn’t mean one can’t commit to not considering such reasons. Now, if it’s irrational to do so, then it’s irrational to do so, even though it is possible. But I’m not so sure it is irrational. If one is considering situations with things as powerful and oppositional as that, it seems like, unless one has a full solid theory of acausal trade ready and has shown that it is beneficial, that it is probably best to blanket refuse all acausal threats, so that they don’t influence what actually happens here.
- khafra 2y agoTo be precise, you should precommit to not trading with entities who threaten punishment--e.g. taking an action that costs them, simply because it also costs you. Unfortunately (or perhaps fortunately, given how we would misuse such an ability), strong precommitments are not available to humans. Our ability to self-modify is vague and bounded. In our organizations and other intelligent tools, we probably should make such precommitments.