3 ms·

This makes we wish we could rate Watson's performance against the players in a way that didn't involve the buzzer. For instance, if Watson and each player were

by moskie 16y ago

This makes we wish we could rate Watson's performance against the players in a way that didn't involve the buzzer. For instance, if Watson and each player were able to give responses for every single clue in the game, who would have the highest percentage of correct responses and/or the highest score?

kenjackson 16y ago
Except, that's not Jeopardy. That's "Answer these Trivia Questions". A legitimate game, but not one that is likely to get much viewership. Everyone misses what should be knocked about Watson. The advantage isn't the buzzer per se. But rather the text message to get the question and the signal to indicate the question had been read. The way it should work in the future is that Watson reads the question from the screen, listens to Alex, and sees the signal from the light. Given the standard way Jeopardy text is written though, I suspect that all of these additional requirements won't change the outcome by much. Watson is just a better Jeopardy player than humans. Are there other trivia games where humans are better? Probably so. But people don't watch these other games in prime time.
- moskie 16y ago
  Well, right, that wouldn't be Jeopardy, but ultimately, I don't really care about Jeopardy, thus my question. What I (and most of us here, I think) would find interesting is how Watson's language processing abilities compares to that of (very smart) people. I don't really care how fast Watson can ring a buzzer. If there was a way to gauge that, I'd find it interesting.
  - kenjackson 16y ago
    I assume you mean "natural" language processing. For that trivia is the wrong test. It's better to just give it paragraphs and ask it questions about the paragraph. And of course have increasing levels of difficult. More like the SAT reading comprehension test. This would extract how it stores data brought in from structured/unstructed sources and the actual task of processing natural language. I suspect it will be a fair bit worse at processing natural language than even the average person.

by moskie 16y ago

kenjackson 16y ago

Except, that's not Jeopardy. That's "Answer these Trivia Questions". A legitimate game, but not one that is likely to get much viewership. Everyone misses what should be knocked about Watson. The advantage isn't the buzzer per se. But rather the text message to get the question and the signal to indicate the question had been read. The way it should work in the future is that Watson reads the question from the screen, listens to Alex, and sees the signal from the light. Given the standard way Jeopardy text is written though, I suspect that all of these additional requirements won't change the outcome by much. Watson is just a better Jeopardy player than humans. Are there other trivia games where humans are better? Probably so. But people don't watch these other games in prime time.

moskie 16y ago
Well, right, that wouldn't be Jeopardy, but ultimately, I don't really care about Jeopardy, thus my question. What I (and most of us here, I think) would find interesting is how Watson's language processing abilities compares to that of (very smart) people. I don't really care how fast Watson can ring a buzzer. If there was a way to gauge that, I'd find it interesting.
- kenjackson 16y ago
  I assume you mean "natural" language processing. For that trivia is the wrong test. It's better to just give it paragraphs and ask it questions about the paragraph. And of course have increasing levels of difficult. More like the SAT reading comprehension test. This would extract how it stores data brought in from structured/unstructed sources and the actual task of processing natural language. I suspect it will be a fair bit worse at processing natural language than even the average person.