I highly doubt that IBM will dismantle the machine when they're done (yes, I know that's what they did with Deep Blue, but this is different). I'd bet that far

I highly doubt that IBM will dismantle the machine when they're done (yes, I know that's what they did with Deep Blue, but this is different). I'd bet that far more than two games will be played with Watson, over the next few years. Also, as pointed out in that thread, this is no different than a regular Jeopardy game. One game consists of dozens and dozens of questions. If Watson answers them all correctly and the humans get zero points, that is indeed statistically significant. If the score is close, then repetition in games will be necessary. The problem with the assumption in that thread is that one Jeopardy game is vastly different than the next. In reality, you could string together a bunch of games and call it one game. Or take one game and split it up into 50 "games". The point is, inside one game, there are enough different questions to definitely count as statistically significant.