4 ms·

The claim that it's statistically unlikely to get those random numbers isn't entirely clear. When asked how unlikely it is, the response is "extremely unlikely"

by azakai 9y ago

The claim that it's statistically unlikely to get those random numbers isn't entirely clear. When asked how unlikely it is, the response is "extremely unlikely", but it's important to quantify that. It's possible he's playing 20x more games than them, for example, and then we'd expect his best scores to contain more unlikely results. But it's hard to say if they would be as unlikely as what we see.

ordinaryperson 9y ago
> It's possible he's playing 20x more games than them Possible but unlikely b/c the person making this claim is the current No. 2 Donkey Kong player in the world Wes Copeland, not Joe Schmo off the street. Even if Mitchell's disputed scores were accepted they would still be far off Copeland's scores. So when the No. 2 player speaks it carries more weight than just some random dude in his basement.
- alexlrobertson 9y ago
  Wouldn’t a pro player be playing less games? They’re playing longer, higher scoring games not lots of average scoring games.
  - ldjb 9y ago
    Sort of. However, at that level, if you're in the middle of a game and aren't on track to getting a high score, you're probably going to abandon the game and try again from the beginning. Therefore there'll be a fairly high number of games, a sizeable proportion of which are abandoned. It's like when speedrunners attempt to complete games in the fastest time. If they make a mistake at any point in the game that will cost them a lot of time, they will most likely abandon the game and keep trying until they are able to complete the game without making any costly mistakes.
  - azakai 9y ago
    It depends how much time they put into it. No one checks how many games they play, so one player could play only on weekends while another might also play a few hours every evening.
bhickey 9y ago
This is easy to resolve. Just dump the ROM and look at the RNG.
- ordinaryperson 9y ago
  The full details are here: http://www.classicarcadegaming.com/forums/index.php?topic=4164.0 http://www.classicarcadegaming.com/forums/index.php?topic=41... But what Wes showed was that all the top gamers (with higher scores than Billy) average 13-15% of their total points from barrel smashes while Billy averaged almost 18%. Details: https://pbs.twimg.com/media/DVDyEHrX4AEnxP0.jpg:large https://pbs.twimg.com/media/DVDyEHrX4AEnxP0.jpg:large
  - deleted 9y ago
    [deleted]
  - bhickey 9y ago
    While that thread breaks down the idealized probabilities, it doesn't say anything about the RNG behavior itself. Some generators have bad seeds. It's conceivable that Billy got lucky and hit one of these. I sincerely doubt that's the case. With the source it'd we could definitively confirm or rule this out. Another possibility is that Billy figured out a hitherto unknown RNG manipulation trick. This is how a lot of TAS runs work. Again, I doubt this is the case but with the source it'd be easy to rule out.
  - jsnell 9y ago
    I don't get it. "Percentage of points coming from smashes" seems like a worthless metric for detecting cheating via favorable RNG seeds. The relevant metric would instead be average points per smash. And for that Mitchell's results are not particularly out of line with the other entries (the distribution is 464-511, with an average of 480; 491 doesn't look at all suspicious in that context). Wouldn't this rather be a sign of people making different strategic tradeoffs? Mitchell spent more time breaking barrels but at the cost of getting fewer bonus points for finishing the level in time.
viraptor 9y ago
Unless the rng is really broken and unstable, there's no reason to get a different result in any top run of the game. The longer you play, the closer you should be to the average.
- StavrosK 9y ago
  What? Of course there's a reason! It's because you only post top runs! That's like saying "there's no reason lottery winners will have hit more winning numbers than anyone else", but they're lottery winners exactly because they did!
  - azakai 9y ago
    Exactly. If a game has an element of chance, then a player's top scores will tend to be top scores in part because of luck helping there (same reason as why regression toward the mean [1] exists). In such a game if each player can choose how many times they play then they can gain an advantage simply by playing more. And their top games will look more and more statistically unlikely. That can be especially surprising if they only report those top games, which is the case here. What would be interesting to compute is how many more games does a player need to be playing in order to reasonably get the results in the article. If it's 2x, it's not proof of cheating. But if it's 1,000x then maybe it is, because who has that much free time? [1] https://en.wikipedia.org/wiki/Regression_toward_the_mean https://en.wikipedia.org/wiki/Regression_toward_the_mean
  - deleted 9y ago
    [deleted]
  - Yen 9y ago
    That's assuming that the random number generator is truly random, and not a deterministic pseudo-random-number-generator, which will _necessarily_ produce a sequence of numbers that satisfy certain statistical properties, over a long enough run. It's unlikely, but possible, to roll a 6-sided die 100 times in a row and get 100 6's. In basically every PRNG algorithm, there is _no_ reachable state which will result in 100 6's in a row.

by azakai 9y ago

ordinaryperson 9y ago

> It's possible he's playing 20x more games than them Possible but unlikely b/c the person making this claim is the current No. 2 Donkey Kong player in the world Wes Copeland, not Joe Schmo off the street. Even if Mitchell's disputed scores were accepted they would still be far off Copeland's scores. So when the No. 2 player speaks it carries more weight than just some random dude in his basement.

alexlrobertson 9y ago
Wouldn’t a pro player be playing less games? They’re playing longer, higher scoring games not lots of average scoring games.
- ldjb 9y ago
  Sort of. However, at that level, if you're in the middle of a game and aren't on track to getting a high score, you're probably going to abandon the game and try again from the beginning. Therefore there'll be a fairly high number of games, a sizeable proportion of which are abandoned. It's like when speedrunners attempt to complete games in the fastest time. If they make a mistake at any point in the game that will cost them a lot of time, they will most likely abandon the game and keep trying until they are able to complete the game without making any costly mistakes.
- azakai 9y ago
  It depends how much time they put into it. No one checks how many games they play, so one player could play only on weekends while another might also play a few hours every evening.

bhickey 9y ago

This is easy to resolve. Just dump the ROM and look at the RNG.

ordinaryperson 9y ago
The full details are here: http://www.classicarcadegaming.com/forums/index.php?topic=4164.0 http://www.classicarcadegaming.com/forums/index.php?topic=41... But what Wes showed was that all the top gamers (with higher scores than Billy) average 13-15% of their total points from barrel smashes while Billy averaged almost 18%. Details: https://pbs.twimg.com/media/DVDyEHrX4AEnxP0.jpg:large https://pbs.twimg.com/media/DVDyEHrX4AEnxP0.jpg:large
- deleted 9y ago
  [deleted]
- bhickey 9y ago
  While that thread breaks down the idealized probabilities, it doesn't say anything about the RNG behavior itself. Some generators have bad seeds. It's conceivable that Billy got lucky and hit one of these. I sincerely doubt that's the case. With the source it'd we could definitively confirm or rule this out. Another possibility is that Billy figured out a hitherto unknown RNG manipulation trick. This is how a lot of TAS runs work. Again, I doubt this is the case but with the source it'd be easy to rule out.
- jsnell 9y ago
  I don't get it. "Percentage of points coming from smashes" seems like a worthless metric for detecting cheating via favorable RNG seeds. The relevant metric would instead be average points per smash. And for that Mitchell's results are not particularly out of line with the other entries (the distribution is 464-511, with an average of 480; 491 doesn't look at all suspicious in that context). Wouldn't this rather be a sign of people making different strategic tradeoffs? Mitchell spent more time breaking barrels but at the cost of getting fewer bonus points for finishing the level in time.

viraptor 9y ago

Unless the rng is really broken and unstable, there's no reason to get a different result in any top run of the game. The longer you play, the closer you should be to the average.

StavrosK 9y ago
What? Of course there's a reason! It's because you only post top runs! That's like saying "there's no reason lottery winners will have hit more winning numbers than anyone else", but they're lottery winners exactly because they did!
- azakai 9y ago
  Exactly. If a game has an element of chance, then a player's top scores will tend to be top scores in part because of luck helping there (same reason as why regression toward the mean [1] exists). In such a game if each player can choose how many times they play then they can gain an advantage simply by playing more. And their top games will look more and more statistically unlikely. That can be especially surprising if they only report those top games, which is the case here. What would be interesting to compute is how many more games does a player need to be playing in order to reasonably get the results in the article. If it's 2x, it's not proof of cheating. But if it's 1,000x then maybe it is, because who has that much free time? [1] https://en.wikipedia.org/wiki/Regression_toward_the_mean https://en.wikipedia.org/wiki/Regression_toward_the_mean
- deleted 9y ago
  [deleted]
- Yen 9y ago
  That's assuming that the random number generator is truly random, and not a deterministic pseudo-random-number-generator, which will _necessarily_ produce a sequence of numbers that satisfy certain statistical properties, over a long enough run. It's unlikely, but possible, to roll a 6-sided die 100 times in a row and get 100 6's. In basically every PRNG algorithm, there is _no_ reachable state which will result in 100 6's in a row.