4 ms·
Here's the meat: "They're not waiting for the light to come on," Welty said; rather, the human players try to time their buzzer presses so that they're coming i
by cma 16y ago
Here's the meat:
"They're not waiting for the light to come on," Welty said; rather, the human players try to time their buzzer presses so that they're coming in as close as possible to the light. Though Watson's reaction times are faster than a human, Welty noted that Watson has to wait for the light. Dr. Adam Lally, another member of Watson's team, noted that "Ken and Brad are really fast. They have to be."
What a deceptive little statement. If you set the "attempt answer" bit in Watson before the light comes on, and you have the humans attempt to press it as soon as allowable, they will never beat Watson.
Actually, they may beat him a few times per hundred-thousand attempts (assuming Watson gets the "light" and buzzes in via tens-of-microseconds-scale ethernet), but would have to be willing to tolerate what would normally be an unacceptable error rate (if you press early, you can't press again for ~2 seconds if I remember right).
- naveensundar 16y agoIf you set the "attempt answer" bit in Watson before the light comes on, and you have the humans attempt to press it as soon as allowable, they will never beat Watson. There were at least a couple of questions for which Watson had 90%+ confidence for the right answer and also way above its buzz threshold but could not buzz in.
- markszcz 16y agoHeres one with 65% confidence: http://www.youtube.com/watch?v=HR2_M8kL_3o&t=3m58s http://www.youtube.com/watch?v=HR2_M8kL_3o&t=3m58s 97% confidence: http://www.youtube.com/watch?v=HR2_M8kL_3o&t=4m26s http://www.youtube.com/watch?v=HR2_M8kL_3o&t=4m26s
- deleted 16y ago[deleted]
- cma 16y agoWatson doesn't decide instantaneously; did it pass its buzz threshold before or after the 'light' went on?
- gojomo 16y agoIndeed. And I suspect the light comes on as the result of some human process – a person offstage following Trebek's cadence – so it has an inherent variance. If that variance is above a certain threshold compared to Watson's light-triggered reaction time, then even a 'perfectly' accurate (on-average) prediction of the light-activation time could be dominated by Watson's precision, and rarely win any buzzes. (The lock-out for being a little too early – a mistake Watson can never make – makes it even harder.)
- ugh 16y agoWatson presses a mechanical buzzer, it’s not Ethernet. (I don’t know the reaction time on that and would like to know it. If read about anything between 1ms and 200ms.)
- modeless 16y agoThe humans can predict when the light will come on before it actually does based on when Trebek finishes reading the question. Watson can't, and that's what Welty meant. Also, Watson buzzes in via the same button the humans do, not electronically, so there is a window after the light comes on but before Watson can physically depress the button. That said, it was obvious last night that in practice Watson has a significant advantage in buzzing in. The supposed advantage of predicting when the light will come on doesn't cancel out Watson's reaction time advantage. I'm guessing the reason for that is there's a significant variation in the time between Trebek finishing reading the question and the light coming on (because the light is triggered by a human), making it hard to predict.