7 ms·

Related: there was a recent article in the New York Times about the problems with this process: https://www.nytimes.com/2021/07/13/sports/chess-karjakin-mishra-

Related: there was a recent article in the New York Times about the problems with this process: https://www.nytimes.com/2021/07/13/sports/chess-karjakin-mishra-grandmasters.html https://www.nytimes.com/2021/07/13/sports/chess-karjakin-mis...

bradleyjg 5y ago
After reading that article it seems like it would be better to replace the “grandmaster norm” part of the process with standardized tournaments against a calibrated computer engine.
- bttrfl 5y ago
  Why?
  - bradleyjg 5y ago
    You can’t pay off a computer to throw a game.
- gurchik 5y ago
  While it might solve issues with the norm process, it introduces more problems, specifically that two players of equal rating can have vastly different performance against a computer because they don't play like humans do.
  - hugh-avherald 5y ago
    Why is that a problem?
    - X6S1x6Okd1st 5y ago
      Because the purpose is to establish how good people are at playing other people, not engines.
- dannyz 5y ago
  While computer engines that are significantly better than the best chess players are plentiful, computer engines that are human rated and play human like moves do not exist. From the definition of ELO you could make a 1500 rated chess engine that plays perfectly half of its games and plays random moves the other half of the games, and it would stay at 1500 ELO if it continued to play against 1500 ELO players. This is an extreme example but it is not too far off from what chess engines do to reduce their ELO.
  - bradleyjg 5y ago
    I haven’t followed the new wave of DL based engines but in the old Alpha-Beta algorithm days engines could be tuned by giving them a fixed processing budget per move (or a more sophisticated processing budget to allow for bursting). Is that not a viable strategy on the new engines?
    - sweezyjeezy 5y ago
      I think it's really hard to make it play like a human. Even GMs miss two-move tactics sometimes, wheras computers are pretty perfect at this. On the other hand traditional algorithms might not be very good at positional chess when on a budget, wheras even weak human players can understand basic concepts like putting your rook on an open file. Then it gets even more complicated with the NNUE position evaluation models - maybe even with a one-move budget the neural net is still implicitly strong at understanding several moves ahead, and could still beat a strong human player.
  - sweezyjeezy 5y ago
    Yeah exactly - it's easier to build an engine that plays as well as possible compared to an engine that plays reasonable/imperfect moves like a strong human player.
  - spywaregorilla 5y ago
    > computer engines that are human rated and play human like moves do not exist Without knowing much about chess, I'd be inclined to say this can't possibly be true. Predicting human chess moves given an ELO rating sounds like a much easier task than training it to be good at the game. It is a different task alltogether, but until you reach relatively high elo scores, I'd expect a fake human ai would have a much easier time simply because it only needs to plan a few moves ahead to mimic the mindset of a lower level player.
- sd8f9iu 5y ago
  Computers are far stronger than humans and win every game. They cannot be artificially weakened as they do not play like weaker humans, but rather make a mixture of incredibly strong moves and random mistakes. As such, it would be a tournament of trying to lose the best game against a computer, which is not an accurate simulation of what human-human chess is like.
  - thom 5y ago
    This isn’t really how most engines work. Reducing the depth that engines calculate to, or altering how they weight certain aspects of evaluation, are both perfectly natural ways to reduce an engine’s strength. They still won’t play like a human, but they don’t need to be forced into random mistakes.
    - gurchik 5y ago
      Sorry if I misunderstood but it sounds like you're saying engines aren't programmed to make random mistakes in order to lower playing strength, but this is how Stockfish works. Lowering the calculation depth and calculation time is used to reduce difficulty, but in addition to those it is programmed to make mistakes: > Internally, MultiPV is enabled, and with a certain probability depending on the Skill Level a weaker move will be played. Within the constraints of the depth and calculation times it determines the best move, but depending on MultiPV it will deliberately not play that move and play an inferior one instead.
      - thom 5y ago
        This still doesn’t mean that engines can’t be made to play weaker moves without making deliberate mistakes, which is the only point I was making.
    - sd8f9iu 5y ago
      Your experience with engines then is very different from my own and that of the wider chess community. The general consensus is that playing computers at any strength is both counterproductive and unfulfilling. They do indeed make random blunders if you lower their search depth. The engine will play brilliant moves for any tactic that falls within the search depth, but fail miserably for tactics that fall one ply outside of it. Humans function very differently to computers: they evaluate a fraction of the positions, but use a much more sophisticated evaluation function. There is no way of emulating human weakness with such a vastly different style of computation. Here is a game I just played that illustrates this phenomenon: https://lichess.org/PiIFqI2c/black https://lichess.org/PiIFqI2c/black The engine plays well before making a series of random blunders no human would make. You can try this yourself by playing the different Stockfish levels on lichess.org. You will be unable to find a level that makes for an enjoyable game. There are some new engines that use neural networks to try and play similarly to how humans do [1]. I can't comment on their success, but their lack of wide adoption by the chess community signals to me that it is still incomparable to humans. [1] https://maiachess.com/ https://maiachess.com/
michaelt 5y ago
TLDR, as the article is paywalled: * You have to win a bunch of games against grandmasters, earning 'GM Norms', in order to become a grandmaster yourself. * It is normal, at some contests, for lower-ranked players to pay to attend, while GMs are paid for attending. * It is widely rumoured some contests are "norm factories" where fee-paying players get matched up with washed-up GMs who essentially throw the game. * There are even rumours of contests where an aspiring grandmaster's parents offered cash to their opponents. There are even rumours of contests that took place only on paper.
- nextlevelchess 5y ago
  yeah, sadly there are some dirty things going on when it comes to Norms. I guess with a better leadership and a clear policy of not tolerating pre-arranged games, this should change. But as with doping, it will never be erased 100%.
- Andrew_nenakhov 5y ago
  Pay money to advance in ratings? Nonsense! Just come up with a clever way to employ Stockfish over the board!

bradleyjg 5y ago

After reading that article it seems like it would be better to replace the “grandmaster norm” part of the process with standardized tournaments against a calibrated computer engine.

bttrfl 5y ago
Why?
- bradleyjg 5y ago
  You can’t pay off a computer to throw a game.
gurchik 5y ago
While it might solve issues with the norm process, it introduces more problems, specifically that two players of equal rating can have vastly different performance against a computer because they don't play like humans do.
- hugh-avherald 5y ago
  Why is that a problem?
  - X6S1x6Okd1st 5y ago
    Because the purpose is to establish how good people are at playing other people, not engines.
dannyz 5y ago
While computer engines that are significantly better than the best chess players are plentiful, computer engines that are human rated and play human like moves do not exist. From the definition of ELO you could make a 1500 rated chess engine that plays perfectly half of its games and plays random moves the other half of the games, and it would stay at 1500 ELO if it continued to play against 1500 ELO players. This is an extreme example but it is not too far off from what chess engines do to reduce their ELO.
- bradleyjg 5y ago
  I haven’t followed the new wave of DL based engines but in the old Alpha-Beta algorithm days engines could be tuned by giving them a fixed processing budget per move (or a more sophisticated processing budget to allow for bursting). Is that not a viable strategy on the new engines?
  - sweezyjeezy 5y ago
    I think it's really hard to make it play like a human. Even GMs miss two-move tactics sometimes, wheras computers are pretty perfect at this. On the other hand traditional algorithms might not be very good at positional chess when on a budget, wheras even weak human players can understand basic concepts like putting your rook on an open file. Then it gets even more complicated with the NNUE position evaluation models - maybe even with a one-move budget the neural net is still implicitly strong at understanding several moves ahead, and could still beat a strong human player.
- sweezyjeezy 5y ago
  Yeah exactly - it's easier to build an engine that plays as well as possible compared to an engine that plays reasonable/imperfect moves like a strong human player.
- spywaregorilla 5y ago
  > computer engines that are human rated and play human like moves do not exist Without knowing much about chess, I'd be inclined to say this can't possibly be true. Predicting human chess moves given an ELO rating sounds like a much easier task than training it to be good at the game. It is a different task alltogether, but until you reach relatively high elo scores, I'd expect a fake human ai would have a much easier time simply because it only needs to plan a few moves ahead to mimic the mindset of a lower level player.
sd8f9iu 5y ago
Computers are far stronger than humans and win every game. They cannot be artificially weakened as they do not play like weaker humans, but rather make a mixture of incredibly strong moves and random mistakes. As such, it would be a tournament of trying to lose the best game against a computer, which is not an accurate simulation of what human-human chess is like.
- thom 5y ago
  This isn’t really how most engines work. Reducing the depth that engines calculate to, or altering how they weight certain aspects of evaluation, are both perfectly natural ways to reduce an engine’s strength. They still won’t play like a human, but they don’t need to be forced into random mistakes.
  - gurchik 5y ago
    Sorry if I misunderstood but it sounds like you're saying engines aren't programmed to make random mistakes in order to lower playing strength, but this is how Stockfish works. Lowering the calculation depth and calculation time is used to reduce difficulty, but in addition to those it is programmed to make mistakes: > Internally, MultiPV is enabled, and with a certain probability depending on the Skill Level a weaker move will be played. Within the constraints of the depth and calculation times it determines the best move, but depending on MultiPV it will deliberately not play that move and play an inferior one instead.
    - thom 5y ago
      This still doesn’t mean that engines can’t be made to play weaker moves without making deliberate mistakes, which is the only point I was making.
  - sd8f9iu 5y ago
    Your experience with engines then is very different from my own and that of the wider chess community. The general consensus is that playing computers at any strength is both counterproductive and unfulfilling. They do indeed make random blunders if you lower their search depth. The engine will play brilliant moves for any tactic that falls within the search depth, but fail miserably for tactics that fall one ply outside of it. Humans function very differently to computers: they evaluate a fraction of the positions, but use a much more sophisticated evaluation function. There is no way of emulating human weakness with such a vastly different style of computation. Here is a game I just played that illustrates this phenomenon: https://lichess.org/PiIFqI2c/black https://lichess.org/PiIFqI2c/black The engine plays well before making a series of random blunders no human would make. You can try this yourself by playing the different Stockfish levels on lichess.org. You will be unable to find a level that makes for an enjoyable game. There are some new engines that use neural networks to try and play similarly to how humans do [1]. I can't comment on their success, but their lack of wide adoption by the chess community signals to me that it is still incomparable to humans. [1] https://maiachess.com/ https://maiachess.com/

michaelt 5y ago

TLDR, as the article is paywalled: * You have to win a bunch of games against grandmasters, earning 'GM Norms', in order to become a grandmaster yourself. * It is normal, at some contests, for lower-ranked players to pay to attend, while GMs are paid for attending. * It is widely rumoured some contests are "norm factories" where fee-paying players get matched up with washed-up GMs who essentially throw the game. * There are even rumours of contests where an aspiring grandmaster's parents offered cash to their opponents. There are even rumours of contests that took place only on paper.

nextlevelchess 5y ago
yeah, sadly there are some dirty things going on when it comes to Norms. I guess with a better leadership and a clear policy of not tolerating pre-arranged games, this should change. But as with doping, it will never be erased 100%.
Andrew_nenakhov 5y ago
Pay money to advance in ratings? Nonsense! Just come up with a clever way to employ Stockfish over the board!