5 ms·

Yea generally I wouldn't characterise the report as being a glimmering indication of the scientific quality of the chess.com cheat detection team. Even the gra

by maegul 4y ago

Yea generally I wouldn't characterise the report as being a glimmering indication of the scientific quality of the chess.com cheat detection team. Even the graphs look a lot like copy-paste jobs from excel, which is fine obviously, but not the sort of thing I'd expect competent scientists/data-scientists/developers to be doing, especially for a big public report where some of the formatting issues are just about readability and basic presentation quality. Like Figures F and G (page 14) ... why are they styled so differently? Same question applies to many of the graphs, which isn't to mention label font sizes either. Beyond that, yea I'd say good amounts of the data and logic are pretty basic and wouldn't be surprising from an amateur on the internet. The main thing chess.com have seems to be their Strength Score algorithm, which IMO is probably developed and maintained by a team different from the one that prepared this report.

Orothrim 4y ago
Good design skills have zero correlation with good data analysis skills.
- bombcar 4y ago
  I might even go so far as to say it’s negative correlation. I’ve seen wonderful and factual data analysis presented in the most horrible default templates and hacked together screenshots.
- maegul 4y ago
  Honestly wonder how much of a HN crowd viewpoint that is. I'm not sure there's too much "correlation" between the two. Instead I'd say that if your job involves communication and data, then that necessitates decent visualisations and whatever "design skills" that entails. Honestly, you don't need "design skills" to make a decent graph ... just the wherewithal to think about what would make a graph clearer/easier to understand the same we expect of ourselves with writing.
  - frumper 4y ago
    That wherewithal is the design skill. Some people have it, some don't. Some are good with data too, some not.
    - maegul 4y ago
      Ok ... then aren't we stretching the term "design skill" too thinly then, to the point of disrespecting the actual skillset? I suspect there'd be a tendency to presume a design process is more like solving a problem with a clear techniques for arriving at an objectively correct solution, when in reality, it can and often if just the iterative process of assessing-adjusting-repeat. Being better at these things to the point of sometimes reducing the iterative loop to 0 repeats is "design skill". But the essential base-line ingredient, IMO, is being willing to enter the iterative cycle at least to some extent.
treis 4y ago
IMHO unless line three of the table on page 5 is a typo it conclusively proves that their analysis is bunk. In every other line they think he cheated 100% of the games. But for line three it's only 12 out of 32. Of the 11 events in that table he had his 2nd highest rating in that event. That obviously shouldn't be possible. If he only cheated in 1/3 of the games in the event then his performance should have been much worse than events where he cheated 100% of the time. And I don't understand how they can say this: >However, while Hans has had a record-setting and remarkable rise in rating and strength, in our view there is a lack of concrete statistical evidence that he cheated in his game with Magnus or in any other over-the-board (“OTB”)—i.e., in-person—games. So what the hell? They pored over his OTB games, found no evidence of cheating, but then wrote 70 more pages? And even the circumstantial evidence is weak. They base a lot on Hans having unusually late improvements. But it's not like they're an order of magnitude better. He's slightly better than #2. And it's a simple fact that someone is going to have a late career rise that is better than anyone else. It's not evidence of cheating. IMHO, Magnus or Chess.com need to pony up some hard evidence or STFU.

by maegul 4y ago

Orothrim 4y ago

Good design skills have zero correlation with good data analysis skills.

bombcar 4y ago
I might even go so far as to say it’s negative correlation. I’ve seen wonderful and factual data analysis presented in the most horrible default templates and hacked together screenshots.
maegul 4y ago
Honestly wonder how much of a HN crowd viewpoint that is. I'm not sure there's too much "correlation" between the two. Instead I'd say that if your job involves communication and data, then that necessitates decent visualisations and whatever "design skills" that entails. Honestly, you don't need "design skills" to make a decent graph ... just the wherewithal to think about what would make a graph clearer/easier to understand the same we expect of ourselves with writing.
- frumper 4y ago
  That wherewithal is the design skill. Some people have it, some don't. Some are good with data too, some not.
  - maegul 4y ago
    Ok ... then aren't we stretching the term "design skill" too thinly then, to the point of disrespecting the actual skillset? I suspect there'd be a tendency to presume a design process is more like solving a problem with a clear techniques for arriving at an objectively correct solution, when in reality, it can and often if just the iterative process of assessing-adjusting-repeat. Being better at these things to the point of sometimes reducing the iterative loop to 0 repeats is "design skill". But the essential base-line ingredient, IMO, is being willing to enter the iterative cycle at least to some extent.

treis 4y ago

IMHO unless line three of the table on page 5 is a typo it conclusively proves that their analysis is bunk. In every other line they think he cheated 100% of the games. But for line three it's only 12 out of 32. Of the 11 events in that table he had his 2nd highest rating in that event. That obviously shouldn't be possible. If he only cheated in 1/3 of the games in the event then his performance should have been much worse than events where he cheated 100% of the time. And I don't understand how they can say this: >However, while Hans has had a record-setting and remarkable rise in rating and strength, in our view there is a lack of concrete statistical evidence that he cheated in his game with Magnus or in any other over-the-board (“OTB”)—i.e., in-person—games. So what the hell? They pored over his OTB games, found no evidence of cheating, but then wrote 70 more pages? And even the circumstantial evidence is weak. They base a lot on Hans having unusually late improvements. But it's not like they're an order of magnitude better. He's slightly better than #2. And it's a simple fact that someone is going to have a late career rise that is better than anyone else. It's not evidence of cheating. IMHO, Magnus or Chess.com need to pony up some hard evidence or STFU.