5 ms·

Personally, I find the idea that "humans could be bargained into a poor position" to be fairly weak. It's certainly possible, but negotiations with hidden agen

Personally, I find the idea that "humans could be bargained into a poor position" to be fairly weak. It's certainly possible, but negotiations with hidden agendas have been part and parcel of humanity for all time, and are a major basis for all foreign (and most internal) affairs. The paper presents no predicament that China couldn't put the US in tomorrow with a suitably (apparently) one-sided deal, and yet countries don't as a rule isolate themselves to avoid hearing potentially deadly trade offers.

pdonis 9y ago
> I find the idea that "humans could be bargained into a poor position" to be fairly weak That's because you're assuming that whatever ETI or AI we are bargaining with has roughly our intelligence. That is what has made it possible for humans to negotiate with each other instead of isolating themselves to avoid hearing potentially deadly offers. But any ETI or AI capable of sending us a complex message will probably be much more intelligent than we are. In that case, we would basically be in the same position as, say, a dog trying to negotiate with humans; we simply would not even be able to comprehend what the other side was doing or thinking.
- wyattpeak 9y ago
  No, I'm not, I'm saying that magic intelligence doesn't change the rules of the game. If I were negotiating with an alien intelligence, I would assume that it could think circles around me, so I wouldn't be about to accept any bargain that I couldn't - fully - think through the ramifications of. I'm not going to accept complicated bargains, only extremely simple ones. This would most probably result in making no bargains at all. So be it. I reject the notion that a being capable of reason, holding all the cards, is unable to create an unwinnable state for its opponent, when inaction is a possibility. The most intelligent computer in the universe can't beat you at chess if you don't play.
  - TheOtherHobbes 9y ago
    How would you know you’re playing chess, or even that you’re negotiating? Any sufficiently advanced alien invasion could be indistinguishable from local politics. I think the paper fails because it assumes a magical hypersmart AI would arrive as a message at a radio telescope. But if you’re going to magically assume a magical hypersmart AI, you may as well assume it can get here magically without being noticed.
    - jacobush 9y ago
      I wonder from an anthropological view, how these discussions relate to those of the past, about demons.
    - wyattpeak 9y ago
      Agreed, there are a million more convincing ways an alien intelligence could take over, most likely - as you say - without us ever noticing. I'm just jumping off from the particular assumption that the paper makes.
  - pdonis 9y ago
    > I reject the notion that a being capable of reason, holding all the cards, is unable to create an unwinnable state for its opponent, when inaction is a possibility. If this is true for you, it's also true for the AI. Which would be a contradiction, similar to the old classic irresistible force meeting immovable object. So the notion you're rejecting is the only consistent possibility.
    - wyattpeak 9y ago
      No, on two counts. Firstly, we begin with a power imbalance, we're not on a level playing field. Secondly, there's nothing inconsistent about both parties being able to create an unwinnable state for the other. As I stated earlier in my comment, the most likely outcome is no dealing whatsoever. The only rational alternative is an agreement whose effects we can both perfectly foresee.

pdonis 9y ago

> I find the idea that "humans could be bargained into a poor position" to be fairly weak That's because you're assuming that whatever ETI or AI we are bargaining with has roughly our intelligence. That is what has made it possible for humans to negotiate with each other instead of isolating themselves to avoid hearing potentially deadly offers. But any ETI or AI capable of sending us a complex message will probably be much more intelligent than we are. In that case, we would basically be in the same position as, say, a dog trying to negotiate with humans; we simply would not even be able to comprehend what the other side was doing or thinking.

wyattpeak 9y ago
No, I'm not, I'm saying that magic intelligence doesn't change the rules of the game. If I were negotiating with an alien intelligence, I would assume that it could think circles around me, so I wouldn't be about to accept any bargain that I couldn't - fully - think through the ramifications of. I'm not going to accept complicated bargains, only extremely simple ones. This would most probably result in making no bargains at all. So be it. I reject the notion that a being capable of reason, holding all the cards, is unable to create an unwinnable state for its opponent, when inaction is a possibility. The most intelligent computer in the universe can't beat you at chess if you don't play.
- TheOtherHobbes 9y ago
  How would you know you’re playing chess, or even that you’re negotiating? Any sufficiently advanced alien invasion could be indistinguishable from local politics. I think the paper fails because it assumes a magical hypersmart AI would arrive as a message at a radio telescope. But if you’re going to magically assume a magical hypersmart AI, you may as well assume it can get here magically without being noticed.
  - jacobush 9y ago
    I wonder from an anthropological view, how these discussions relate to those of the past, about demons.
  - wyattpeak 9y ago
    Agreed, there are a million more convincing ways an alien intelligence could take over, most likely - as you say - without us ever noticing. I'm just jumping off from the particular assumption that the paper makes.
- pdonis 9y ago
  > I reject the notion that a being capable of reason, holding all the cards, is unable to create an unwinnable state for its opponent, when inaction is a possibility. If this is true for you, it's also true for the AI. Which would be a contradiction, similar to the old classic irresistible force meeting immovable object. So the notion you're rejecting is the only consistent possibility.
  - wyattpeak 9y ago
    No, on two counts. Firstly, we begin with a power imbalance, we're not on a level playing field. Secondly, there's nothing inconsistent about both parties being able to create an unwinnable state for the other. As I stated earlier in my comment, the most likely outcome is no dealing whatsoever. The only rational alternative is an agreement whose effects we can both perfectly foresee.