5 ms·

> You beat him! >> No sir, it is a stalemate. > What did you do? >> I was playing for a standoff; a draw. While Kolrami was dedicated to winning, I was able

> You beat him! >> No sir, it is a stalemate. > What did you do? >> I was playing for a standoff; a draw. While Kolrami was dedicated to winning, I was able to pass up obvious avenues of advancement and settle for a balance. Theoretically, I should be able to challenge him indefinitely. > Then you have beaten him! >> In the strictest sense, I did not win. > (groans) Data! >> I busted him up. > (everybody claps)

cwillu 2y ago
You… have made a mockery of me.
- moffkalast 2y ago
  It is possible to commit no mistakes and still lose. That's not a weakness. That's life.
  - cwillu 2y ago
    But, knowing that he knows that we know that he knows, he might choose to return to his usual pattern.
taneq 2y ago
No idea if they did this on purpose but this is exactly what can happen with board game AIs when they know they will win. Unless the evaluation function explicitly promotes winning sooner they will get into an unbeatable position and then just fardle around because they have no reason to win now if they know they can do it later.
- cjbgkagh 2y ago
  Future payoffs are almost always discounted, even if for no other reason than the future has a greater deal of uncertainty. I.e even if it was not explicit which it almost always is, it would still be implicit. Their conservative style is usually due to having a better fitness function. Humans tend to not be able to model uncertainty as accurately and this results in more aggressive play, a bird in the hand is worth two in the bush.
  - taneq 2y ago
    Typically yeah, but when you're trying to make it work at all it can be easy to forget to add a bit of a gradient towards "winning sooner is better". And this happens even at the top level, the example I was thinking about as I typed that was one of the AlphaGo exhibition games against Lee Sedol (the first, maybe?) where it got into a crushing position then seemingly messed around.
    - cjbgkagh 2y ago
      There is zero chance AlphaGo devs forgot about discounting. Usually you relax the discount to allow for optimal play, most likely the fitness function flailed a bit in the long tail.
  - kqr 2y ago
    Indeed. Humans use "points ahead" as a proxy for "chance of win" so we tend to play lines that increase our lead more, even when they are a tiny bit riskier. Good software does not -- it aims for maximum chance of win, which usually means slower, less aggressive moves to turn uncertain situations into more well-defined ones.
- nilslindemann 2y ago
  Example: https://lichess.org/study/kPWZgp6s/nwqy2Hwg https://lichess.org/study/kPWZgp6s/nwqy2Hwg
- gweinberg 2y ago
  Doesn't the board get filled up with stones? I could see how a go player might think a win is a win so it doesn't mater how many stones you win by, but I don;t see how you would go about delaying winning.
  - deleted 2y ago
    [deleted]
  - deleted 2y ago
    [deleted]
  - zahlman 2y ago
    >Doesn't the board get filled up with stones? To some extent, but a player who's way ahead could still have a lot of latitude to play pointless moves without endangering the win. In the case of Go it's generally not so much "delaying winning" as just embarrassing the opponent by playing obviously suboptimal moves (that make it clearer that some key group is dead, for example). Although it's possible to start irrelevant, time-wasting ko positions - if the opponent accepts the offer to fight over them.
- zahlman 2y ago
  When I was a child, I didn't understand that episode as Data demonstrating his superiority at the game by deliberately keeping it evenly-matched, or that the alien opponent somehow realized that Data could win at any time and simply chose not to. Rather, I figured Data had come up with some hitherto-unknown strategy that allowed for making the game arbitrarily long; and that the alien had a choice between deliberately losing, accidentally losing (the way the game is depicted, it gets more complex the longer you play) or continuing to play (where an android wouldn't be limited by biology). (No, I didn't phrase my understanding like that, or speak it aloud.)
dataviz1000 2y ago
Wasn't this the plot to War Games (1983)?
- ncr100 2y ago
  Q: If the AIs are trained on adversarial policies, will this strategy also start to fail in these game-playing scenarios? EDIT: Discussed later on https://news.ycombinator.com/item?id=42503110 https://news.ycombinator.com/item?id=42503110
  - renewiltord 2y ago
    > The core vulnerability uncovered by our attack persists even in KataGo agents adversarially trained to defend against our attack
    - ncr100 2y ago
      Thanks!

cwillu 2y ago

You… have made a mockery of me.

moffkalast 2y ago
It is possible to commit no mistakes and still lose. That's not a weakness. That's life.
- cwillu 2y ago
  But, knowing that he knows that we know that he knows, he might choose to return to his usual pattern.

taneq 2y ago

No idea if they did this on purpose but this is exactly what can happen with board game AIs when they know they will win. Unless the evaluation function explicitly promotes winning sooner they will get into an unbeatable position and then just fardle around because they have no reason to win now if they know they can do it later.

cjbgkagh 2y ago
Future payoffs are almost always discounted, even if for no other reason than the future has a greater deal of uncertainty. I.e even if it was not explicit which it almost always is, it would still be implicit. Their conservative style is usually due to having a better fitness function. Humans tend to not be able to model uncertainty as accurately and this results in more aggressive play, a bird in the hand is worth two in the bush.
- taneq 2y ago
  Typically yeah, but when you're trying to make it work at all it can be easy to forget to add a bit of a gradient towards "winning sooner is better". And this happens even at the top level, the example I was thinking about as I typed that was one of the AlphaGo exhibition games against Lee Sedol (the first, maybe?) where it got into a crushing position then seemingly messed around.
  - cjbgkagh 2y ago
    There is zero chance AlphaGo devs forgot about discounting. Usually you relax the discount to allow for optimal play, most likely the fitness function flailed a bit in the long tail.
- kqr 2y ago
  Indeed. Humans use "points ahead" as a proxy for "chance of win" so we tend to play lines that increase our lead more, even when they are a tiny bit riskier. Good software does not -- it aims for maximum chance of win, which usually means slower, less aggressive moves to turn uncertain situations into more well-defined ones.
nilslindemann 2y ago
Example: https://lichess.org/study/kPWZgp6s/nwqy2Hwg https://lichess.org/study/kPWZgp6s/nwqy2Hwg
gweinberg 2y ago
Doesn't the board get filled up with stones? I could see how a go player might think a win is a win so it doesn't mater how many stones you win by, but I don;t see how you would go about delaying winning.
- deleted 2y ago
  [deleted]
- deleted 2y ago
  [deleted]
- zahlman 2y ago
  >Doesn't the board get filled up with stones? To some extent, but a player who's way ahead could still have a lot of latitude to play pointless moves without endangering the win. In the case of Go it's generally not so much "delaying winning" as just embarrassing the opponent by playing obviously suboptimal moves (that make it clearer that some key group is dead, for example). Although it's possible to start irrelevant, time-wasting ko positions - if the opponent accepts the offer to fight over them.
zahlman 2y ago
When I was a child, I didn't understand that episode as Data demonstrating his superiority at the game by deliberately keeping it evenly-matched, or that the alien opponent somehow realized that Data could win at any time and simply chose not to. Rather, I figured Data had come up with some hitherto-unknown strategy that allowed for making the game arbitrarily long; and that the alien had a choice between deliberately losing, accidentally losing (the way the game is depicted, it gets more complex the longer you play) or continuing to play (where an android wouldn't be limited by biology). (No, I didn't phrase my understanding like that, or speak it aloud.)

dataviz1000 2y ago

Wasn't this the plot to War Games (1983)?

ncr100 2y ago
Q: If the AIs are trained on adversarial policies, will this strategy also start to fail in these game-playing scenarios? EDIT: Discussed later on https://news.ycombinator.com/item?id=42503110 https://news.ycombinator.com/item?id=42503110
- renewiltord 2y ago
  > The core vulnerability uncovered by our attack persists even in KataGo agents adversarially trained to defend against our attack
  - ncr100 2y ago
    Thanks!