3 ms·

No idea if they did this on purpose but this is exactly what can happen with board game AIs when they know they will win. Unless the evaluation function explici

by taneq 2y ago

No idea if they did this on purpose but this is exactly what can happen with board game AIs when they know they will win. Unless the evaluation function explicitly promotes winning sooner they will get into an unbeatable position and then just fardle around because they have no reason to win now if they know they can do it later.

cjbgkagh 2y ago
Future payoffs are almost always discounted, even if for no other reason than the future has a greater deal of uncertainty. I.e even if it was not explicit which it almost always is, it would still be implicit. Their conservative style is usually due to having a better fitness function. Humans tend to not be able to model uncertainty as accurately and this results in more aggressive play, a bird in the hand is worth two in the bush.
- taneq 2y ago
  Typically yeah, but when you're trying to make it work at all it can be easy to forget to add a bit of a gradient towards "winning sooner is better". And this happens even at the top level, the example I was thinking about as I typed that was one of the AlphaGo exhibition games against Lee Sedol (the first, maybe?) where it got into a crushing position then seemingly messed around.
  - cjbgkagh 2y ago
    There is zero chance AlphaGo devs forgot about discounting. Usually you relax the discount to allow for optimal play, most likely the fitness function flailed a bit in the long tail.
- kqr 2y ago
  Indeed. Humans use "points ahead" as a proxy for "chance of win" so we tend to play lines that increase our lead more, even when they are a tiny bit riskier. Good software does not -- it aims for maximum chance of win, which usually means slower, less aggressive moves to turn uncertain situations into more well-defined ones.
nilslindemann 2y ago
Example: https://lichess.org/study/kPWZgp6s/nwqy2Hwg https://lichess.org/study/kPWZgp6s/nwqy2Hwg
gweinberg 2y ago
Doesn't the board get filled up with stones? I could see how a go player might think a win is a win so it doesn't mater how many stones you win by, but I don;t see how you would go about delaying winning.
- deleted 2y ago
  [deleted]
- deleted 2y ago
  [deleted]
- zahlman 2y ago
  >Doesn't the board get filled up with stones? To some extent, but a player who's way ahead could still have a lot of latitude to play pointless moves without endangering the win. In the case of Go it's generally not so much "delaying winning" as just embarrassing the opponent by playing obviously suboptimal moves (that make it clearer that some key group is dead, for example). Although it's possible to start irrelevant, time-wasting ko positions - if the opponent accepts the offer to fight over them.
zahlman 2y ago
When I was a child, I didn't understand that episode as Data demonstrating his superiority at the game by deliberately keeping it evenly-matched, or that the alien opponent somehow realized that Data could win at any time and simply chose not to. Rather, I figured Data had come up with some hitherto-unknown strategy that allowed for making the game arbitrarily long; and that the alien had a choice between deliberately losing, accidentally losing (the way the game is depicted, it gets more complex the longer you play) or continuing to play (where an android wouldn't be limited by biology). (No, I didn't phrase my understanding like that, or speak it aloud.)

by taneq 2y ago

cjbgkagh 2y ago

Future payoffs are almost always discounted, even if for no other reason than the future has a greater deal of uncertainty. I.e even if it was not explicit which it almost always is, it would still be implicit. Their conservative style is usually due to having a better fitness function. Humans tend to not be able to model uncertainty as accurately and this results in more aggressive play, a bird in the hand is worth two in the bush.

taneq 2y ago
Typically yeah, but when you're trying to make it work at all it can be easy to forget to add a bit of a gradient towards "winning sooner is better". And this happens even at the top level, the example I was thinking about as I typed that was one of the AlphaGo exhibition games against Lee Sedol (the first, maybe?) where it got into a crushing position then seemingly messed around.
- cjbgkagh 2y ago
  There is zero chance AlphaGo devs forgot about discounting. Usually you relax the discount to allow for optimal play, most likely the fitness function flailed a bit in the long tail.
kqr 2y ago
Indeed. Humans use "points ahead" as a proxy for "chance of win" so we tend to play lines that increase our lead more, even when they are a tiny bit riskier. Good software does not -- it aims for maximum chance of win, which usually means slower, less aggressive moves to turn uncertain situations into more well-defined ones.

nilslindemann 2y ago

Example: https://lichess.org/study/kPWZgp6s/nwqy2Hwg https://lichess.org/study/kPWZgp6s/nwqy2Hwg

gweinberg 2y ago

Doesn't the board get filled up with stones? I could see how a go player might think a win is a win so it doesn't mater how many stones you win by, but I don;t see how you would go about delaying winning.

deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
zahlman 2y ago
>Doesn't the board get filled up with stones? To some extent, but a player who's way ahead could still have a lot of latitude to play pointless moves without endangering the win. In the case of Go it's generally not so much "delaying winning" as just embarrassing the opponent by playing obviously suboptimal moves (that make it clearer that some key group is dead, for example). Although it's possible to start irrelevant, time-wasting ko positions - if the opponent accepts the offer to fight over them.

zahlman 2y ago

When I was a child, I didn't understand that episode as Data demonstrating his superiority at the game by deliberately keeping it evenly-matched, or that the alien opponent somehow realized that Data could win at any time and simply chose not to. Rather, I figured Data had come up with some hitherto-unknown strategy that allowed for making the game arbitrarily long; and that the alien had a choice between deliberately losing, accidentally losing (the way the game is depicted, it gets more complex the longer you play) or continuing to play (where an android wouldn't be limited by biology). (No, I didn't phrase my understanding like that, or speak it aloud.)