5 ms·

This is really interesting because i ran into a pokemon bot the other day were its training led to calibration of 50% winrste at all levels of play on Pokémon s

This is really interesting because i ran into a pokemon bot the other day were its training led to calibration of 50% winrste at all levels of play on Pokémon showdown. It was a complete accident.

Someone 2y ago
But that doesn’t imply that that bot played like an average human. Making a computer have a 50% score against a 1300-rated human is way easier than making it play like a 1300-rated human. For the former, you can take a top-of-the-line program and have it flip a coin in every game whether to make a random move every move or not.
qq66 2y ago
Playing a chess bot that works this way feels like playing a Magnus Carlsen who's trying to let you win.
School-Cotton 2y ago
It's not hard to make a chess bot that plays at a 1300 strength, i.e. its rating would converge to 1300 if it were allowed to compete. But it will not play like a 1300-rated human. It would play like a superhuman genius on most moves and then make beginner-level blunders at random moments. Making one that realistically plays like a human is an unsolved problem.
- darkerside 2y ago
  So like a 1500 rated human?
- SpaceManNabs 2y ago
  ah that makes sense. thanks!
- rieska 2y ago
  Of course, you are right. But (the linked site) at least has a bot that plays the opening like a human of chosen rating perfectly. It stops working after the opening-stage (since it just copies moves from humans in the lichess game database), but it is still very impressive. For later game stages, some other method would have to be used (unless we play multiple orders of magnintude more games on lichess). Now that i think about it, i remember the people in the alphago documentary talking about the bot giving its moves percentage scores in both how high winning % the move had and how high % chance that a human would have made the same move that it just played. I wonder why they never showed what a full game of the most human-like moves from alphago would look like. Maybe it actually worked, by feeding it all the pro games in existence, and training it to play the high human % instead of the higest win probability moves like they did in the end. https://www.chessassess.com/openings https://www.chessassess.com/openings
- sheepscreek 2y ago
  I think this can be achieved with some ease with a machine learning model. You will have to train it on games between 1300-rated players and below. A transformer model might work even better in terms of the evenness of play (behaving like a 1300 rated player throughout the game).
  - lmm 2y ago
    > I think this can be achieved with some ease with a machine learning model. What evidence lead you to think that, and how surprised would you be to be wrong?
- sourcepluck 2y ago
  Ahem https://lichess.org/@/lichess/blog/introducing-maia-a-human-like-neural-network-chess-engine/X9PUixUA https://lichess.org/@/lichess/blog/introducing-maia-a-human-...
- deleted 2y ago
  [deleted]

This is really interesting because i ran into a pokemon bot the other day were its training led to calibration of 50% winrste at all levels of play on Pokémon showdown. It was a complete accident.

Someone 2y ago

But that doesn’t imply that that bot played like an average human. Making a computer have a 50% score against a 1300-rated human is way easier than making it play like a 1300-rated human. For the former, you can take a top-of-the-line program and have it flip a coin in every game whether to make a random move every move or not.

qq66 2y ago

Playing a chess bot that works this way feels like playing a Magnus Carlsen who's trying to let you win.

School-Cotton 2y ago

It's not hard to make a chess bot that plays at a 1300 strength, i.e. its rating would converge to 1300 if it were allowed to compete. But it will not play like a 1300-rated human. It would play like a superhuman genius on most moves and then make beginner-level blunders at random moments. Making one that realistically plays like a human is an unsolved problem.

darkerside 2y ago
So like a 1500 rated human?
SpaceManNabs 2y ago
ah that makes sense. thanks!
rieska 2y ago
Of course, you are right. But (the linked site) at least has a bot that plays the opening like a human of chosen rating perfectly. It stops working after the opening-stage (since it just copies moves from humans in the lichess game database), but it is still very impressive. For later game stages, some other method would have to be used (unless we play multiple orders of magnintude more games on lichess). Now that i think about it, i remember the people in the alphago documentary talking about the bot giving its moves percentage scores in both how high winning % the move had and how high % chance that a human would have made the same move that it just played. I wonder why they never showed what a full game of the most human-like moves from alphago would look like. Maybe it actually worked, by feeding it all the pro games in existence, and training it to play the high human % instead of the higest win probability moves like they did in the end. https://www.chessassess.com/openings https://www.chessassess.com/openings
sheepscreek 2y ago
I think this can be achieved with some ease with a machine learning model. You will have to train it on games between 1300-rated players and below. A transformer model might work even better in terms of the evenness of play (behaving like a 1300 rated player throughout the game).
- lmm 2y ago
  > I think this can be achieved with some ease with a machine learning model. What evidence lead you to think that, and how surprised would you be to be wrong?
sourcepluck 2y ago
Ahem https://lichess.org/@/lichess/blog/introducing-maia-a-human-like-neural-network-chess-engine/X9PUixUA https://lichess.org/@/lichess/blog/introducing-maia-a-human-...
deleted 2y ago
[deleted]