In order to judge whether these "many situations" are significant, we need to judge how big an effect the randomness of tile drawing is. To pick an extreme exa

In order to judge whether these "many situations" are significant, we need to judge how big an effect the randomness of tile drawing is. To pick an extreme example, let's say that the first player to get an "unlucky" rack gets stuck in loop that somehow keeps them at a disadvantage, one which perhaps snowballs. As an analogy, imagine a game of chess where each player rolls a die at the start of each turn, and if they roll a 6 they have to lose a pawn of their choice. It might be possible for a chess grandmaster to beat a chess computer under that rule set nearly 50% of the time, even with the human making sub-optimal moves in "many situations". Remember, in a perfectly random game, 50% of the time it doesn't matter if you make a mistake, because you were going to lose anyway. (I don't just mean there are psychological factors of players getting lazy when they know they're unlikely to win, I mean that these mistakes don't affect the outcome, so we're talking about very small statistical effects here). Anyway, your experience with Scrabble games probably gives you some good intuition for how big an effect luck could possibly have, and I'm happy to accept an assertion that 500 games would be enough to show a statistically measurable lead for a truly perfect player over both the top human and top AI players. It would still be nice to get some data on the effect of randomness, though, for example by measuring the difference in total score between an AI that gets a standard random draw of tiles each turn, and an AI that always draws some set of tiles in the n-th decile of expected score.