4 ms·

"In a game in which dishonesty is commonplace, it is notable that we were able to achieve human-level performance by controlling the agent's dialogue through th

"In a game in which dishonesty is commonplace, it is notable that we were able to achieve human-level performance by controlling the agent's dialogue through the strategic reasoning module to be largely honest and helpful to its speaking partners."

sdenton4 4y ago
This is one of the key insights in playing Diplomacy well. You can turn honest opposition in the early game into trust, which is helpful for flipping allegiances in the late game. ('Yes, I've been trying to murder you the entire game, but I've been honest about it, and never lied to you once. Now I see that we've got a real opportunity to dislodge Russia and sweep the eastern half of the board if we work together... What do you say?') Dishonesty means people don't want to work with you, because you are not a dependable partner. So you gotta make sure that any lies you tell pay off /big./
bpodgursky 4y ago
Well, yes, "largely" honest, until the critical moment when you stab to take 18 supply centers to win the game. You don't win Diplomacy by staying honest, you win by choosing the right moment to make the move and burn that trust. It's the 5% dishonesty that matters.
- wolverine876 4y ago
  It's interesting and I think significant that people are uncomfortable with positive values, such as honesty, doing good for others, responsibility to others, right (not wrong), etc. We shoot them down, similar to how people instinctively cut off a conversation going in an uncomfortable direction, often without even realizing it. I can't attribute that the parent - I don't know the individual - but I expected a chorus of such responses. I feel like I can see them squirming in their seats ... But I feel it too. There's societal pressure against doing good. It's often more socially comfortable to throw a cigarette butt on the ground than to pick one up and discard it. That's bizarre, it's perverse incentives, and we can and should change it.
  - zemvpferreira 4y ago
    It's perfectly reasonable to advise against good values in a game that's won by people who strategically abandon theirs. If you want to win, that's the strategy to follow. Not necessarily a lesson for life unless you view life as a finite game to be won. But in that case, it's probably the best strategy as well. I don't view life like that and I'm guessing you don't as well, but I'd advise anyone to be cautious of their interactions with people who play to win.
  - bpodgursky 4y ago
    When people are concerned about AI alignment, they are not concerned about an AI that is a pathological liar, lying for no reason. They are concerned about an AI which is believable day-to-day but is secretly optimizing for its own goals, which are only exposed when it eventually makes a deceitful move it cannot disguise, but succeeds because it now has sufficient control. That's literally what the AI is doing here; being honest until the moment it is able to stab everyone else and throw them under the table.
  - svnt 4y ago
    I think what you’ve identified is a societal immune response to predictable morality being systematically exploited by amoral actors to gain wealth and standing. I also think there is a big gap between what most people advise and what they ultimately do that biases toward the right thing. That is, people indicate they will not be moral in order to dissuade others who might try to exploit it, but in the end they generally behave morally. Personally I will advise other people to be cautious while repeatedly leaving myself open to being taken advantage of. It almost never happens, and you learn to identify those who will. Of course this all pivots when/if you enter corporate leadership and part of your job is to use the morality/comfort of others as part of the container for getting your role accomplished.
    - wolverine876 4y ago
      > I also think there is a big gap between what most people advise and what they ultimately do that biases toward the right thing. That is, people indicate they will not be moral in order to dissuade others who might try to exploit it, but in the end they generally behave morally. That is very interesting. Have you seen any research on it? > while repeatedly leaving myself open to being taken advantage of. It almost never happens, and you learn to identify those who will. My thinking: There is no perfectly safe solution. People who think I'm taking naive risks don't get such great results themselves. I think being 'open' is generally safer - humans generally follow the lead of those around them, and I get better responses. > you learn to identify those who will. Yes, it cannot be overstated: Being honest, you develop expertise in the skills of executing honestly, and as with any skill that expertise enables you to evaluate those skills in others. Do otherwise, you acquire other skills.
- greesil 4y ago
  So Diplomacy is just an acrimonious version of iterated prisoner's dilemma?

sdenton4 4y ago

This is one of the key insights in playing Diplomacy well. You can turn honest opposition in the early game into trust, which is helpful for flipping allegiances in the late game. ('Yes, I've been trying to murder you the entire game, but I've been honest about it, and never lied to you once. Now I see that we've got a real opportunity to dislodge Russia and sweep the eastern half of the board if we work together... What do you say?') Dishonesty means people don't want to work with you, because you are not a dependable partner. So you gotta make sure that any lies you tell pay off /big./

bpodgursky 4y ago

Well, yes, "largely" honest, until the critical moment when you stab to take 18 supply centers to win the game. You don't win Diplomacy by staying honest, you win by choosing the right moment to make the move and burn that trust. It's the 5% dishonesty that matters.

wolverine876 4y ago
It's interesting and I think significant that people are uncomfortable with positive values, such as honesty, doing good for others, responsibility to others, right (not wrong), etc. We shoot them down, similar to how people instinctively cut off a conversation going in an uncomfortable direction, often without even realizing it. I can't attribute that the parent - I don't know the individual - but I expected a chorus of such responses. I feel like I can see them squirming in their seats ... But I feel it too. There's societal pressure against doing good. It's often more socially comfortable to throw a cigarette butt on the ground than to pick one up and discard it. That's bizarre, it's perverse incentives, and we can and should change it.
- zemvpferreira 4y ago
  It's perfectly reasonable to advise against good values in a game that's won by people who strategically abandon theirs. If you want to win, that's the strategy to follow. Not necessarily a lesson for life unless you view life as a finite game to be won. But in that case, it's probably the best strategy as well. I don't view life like that and I'm guessing you don't as well, but I'd advise anyone to be cautious of their interactions with people who play to win.
- bpodgursky 4y ago
  When people are concerned about AI alignment, they are not concerned about an AI that is a pathological liar, lying for no reason. They are concerned about an AI which is believable day-to-day but is secretly optimizing for its own goals, which are only exposed when it eventually makes a deceitful move it cannot disguise, but succeeds because it now has sufficient control. That's literally what the AI is doing here; being honest until the moment it is able to stab everyone else and throw them under the table.
- svnt 4y ago
  I think what you’ve identified is a societal immune response to predictable morality being systematically exploited by amoral actors to gain wealth and standing. I also think there is a big gap between what most people advise and what they ultimately do that biases toward the right thing. That is, people indicate they will not be moral in order to dissuade others who might try to exploit it, but in the end they generally behave morally. Personally I will advise other people to be cautious while repeatedly leaving myself open to being taken advantage of. It almost never happens, and you learn to identify those who will. Of course this all pivots when/if you enter corporate leadership and part of your job is to use the morality/comfort of others as part of the container for getting your role accomplished.
  - wolverine876 4y ago
    > I also think there is a big gap between what most people advise and what they ultimately do that biases toward the right thing. That is, people indicate they will not be moral in order to dissuade others who might try to exploit it, but in the end they generally behave morally. That is very interesting. Have you seen any research on it? > while repeatedly leaving myself open to being taken advantage of. It almost never happens, and you learn to identify those who will. My thinking: There is no perfectly safe solution. People who think I'm taking naive risks don't get such great results themselves. I think being 'open' is generally safer - humans generally follow the lead of those around them, and I get better responses. > you learn to identify those who will. Yes, it cannot be overstated: Being honest, you develop expertise in the skills of executing honestly, and as with any skill that expertise enables you to evaluate those skills in others. Do otherwise, you acquire other skills.
greesil 4y ago
So Diplomacy is just an acrimonious version of iterated prisoner's dilemma?