4 ms·
You're completely right, and I'm sick of people equating minimizing Watson's Jeopardy! success with minimizing the software/hardware accomplishment behind Watso
by baddox 16y ago
You're completely right, and I'm sick of people equating minimizing Watson's Jeopardy! success with minimizing the software/hardware accomplishment behind Watson. IBM has done something great here, but the buzzer dynamic renders Watson's Jeopardy! prowess unimpressive.
A good analogy, similar to your car vs. human analogy, is this: Hold a competition where two contestants much first pass some sort of Turing-like test. Upon passing that test, both contestants must sort one million 32-bit integers. The first contestant to finish both tasks wins.
In that hypothetical competition, it would certainly be impressive for a computer contestant to pass the Turing-like test. (The human contestant would probably have no trouble doing so.) But the sorting task would seal the door, and the computer would win every time it is able to pass the Turing-like test. The sorting task is known to be dominated by computers, just like buzzer reflexes are known to be dominated by computers.
- ugh 16y agoI don’t know what exactly makes Watson’s performance unimpressive. Being able to answer 84% of all questions correctly is impressive, reaction times or not.
- baddox 16y agoAs I've said many times, Watson being able to come up with correct responses that often is impressive. Watson winning Jeopardy! is what's not impressive.
- ugh 16y agoThat seems to be a contradiction. Being able to answer 84% of the questions seems to be about champion level.
- baddox 16y agoIt might be close to champion level, but I strongly suspect that it's not as good as Watson's two human contestants.
- ugh 16y agoI would be interesting to know how the human contestants would play on their own. We only see them in competitive matches, so we don’t really know how many of the questions they would and could answer. (I heard that contestants tend to push their buzzer as soon as they hear one of their competitors pushing the buzzer regardless of whether they know the answer, so you can’t even know from buzzer presses alone whether the would answer if there were no competitors.) 84% is, to my mind, very high. I’m skeptical of claims that even the best humans are better than that.