3 ms·

I stopped taking this seriously when they compared the accuracy of the "ask the audience" and "phone a friend" bits on Who Wants to be a Millionaire without con

I stopped taking this seriously when they compared the accuracy of the "ask the audience" and "phone a friend" bits on Who Wants to be a Millionaire without considering that everyone used the ask the audience lifeline on an early question and saved the phone a friend for later questions. Without controlling for when they are used the comparison of accuracy is literally meaningless. The same stats show that, rightly or wrongly, the contestants didn't have as much faith in the audience for later stage, much harder questions. I guess I should give the benefit of the doubt and assume the writer didn't notice this and the error is accidental but that's only slightly better than just ignoring it because you wanted a sexy anecdote to support your point.

hammock 14y ago
You right that the two lifelines are used at differing stages in the game. I wonder though, what the accuracy rates look like after controlling for what stage of the game you are in. Maybe people SHOULD save their ask the audience for later in the game.
- T-hawk 14y ago
  Another variable is the subject matter of the question. Anecdotally, it feels that the audience almost invariably nails the answer by a wide margin on pop culture questions. On questions of a scientific or technical nature, the audience opinion is much less clear and more often actually wrong. Phone those questions to a domain knowledge expert. About the worst thing you can do, but contestants love doing this, is to burn the valuable audience lifeline early on a question when you already have a good sense of the answer but just want to "make sure". What a waste! Save that for a question when you have no clue, which WILL happen before you reach the million level.
  - nisse72 14y ago
    Something else they often do is say something like "I'm thinking it's B, but I'm not sure so I'll ask the audience". Well guess what the audience guesses after hearing that?
- freshhawk 14y ago
  My intuition is that the accuracy would drop a fair bit in later stages and anecdotes support this too. My intuition is that for ask the audience to be useful the number of people who know the answer has to be well above the noise or you don't get any wisdom from the crowd. So that means that enough people have to know the right answer AND the wrong answers should be distributed as evenly as possible. People don't often answer multiple choice trivia questions wrong in an evenly distributed way so this makes sense to me. It would be cool to see some actual data on this though, I thought I remembered seeing something back when the show was huge but it's a vague memory.
chmike 14y ago
If you stopped read at this point you missed the main idea of the essay. It is excellent.
- freshhawk 14y ago
  Didn't stop reading, just stopped taking it seriously. I agree with the conclusion that the wisdom of the crowd doesn't apply to creative endeavors, and the reasoning why. The framing of the vote as a veto was stating the idea in a nice way. But that means that all it is is a conclusion I already agree with prefaced by bad logic.

hammock 14y ago

You right that the two lifelines are used at differing stages in the game. I wonder though, what the accuracy rates look like after controlling for what stage of the game you are in. Maybe people SHOULD save their ask the audience for later in the game.

T-hawk 14y ago
Another variable is the subject matter of the question. Anecdotally, it feels that the audience almost invariably nails the answer by a wide margin on pop culture questions. On questions of a scientific or technical nature, the audience opinion is much less clear and more often actually wrong. Phone those questions to a domain knowledge expert. About the worst thing you can do, but contestants love doing this, is to burn the valuable audience lifeline early on a question when you already have a good sense of the answer but just want to "make sure". What a waste! Save that for a question when you have no clue, which WILL happen before you reach the million level.
- nisse72 14y ago
  Something else they often do is say something like "I'm thinking it's B, but I'm not sure so I'll ask the audience". Well guess what the audience guesses after hearing that?
freshhawk 14y ago
My intuition is that the accuracy would drop a fair bit in later stages and anecdotes support this too. My intuition is that for ask the audience to be useful the number of people who know the answer has to be well above the noise or you don't get any wisdom from the crowd. So that means that enough people have to know the right answer AND the wrong answers should be distributed as evenly as possible. People don't often answer multiple choice trivia questions wrong in an evenly distributed way so this makes sense to me. It would be cool to see some actual data on this though, I thought I remembered seeing something back when the show was huge but it's a vague memory.

chmike 14y ago

If you stopped read at this point you missed the main idea of the essay. It is excellent.

freshhawk 14y ago
Didn't stop reading, just stopped taking it seriously. I agree with the conclusion that the wisdom of the crowd doesn't apply to creative endeavors, and the reasoning why. The framing of the vote as a veto was stating the idea in a nice way. But that means that all it is is a conclusion I already agree with prefaced by bad logic.