5 ms·

Someone better than me at math should say how many votes we need to have a 95% accuracy.

nx 18y ago
We need the confidence thingy for this online community giving information about their age.
tokenadult 18y ago
The key issue is being good at statistics, not math. The data can't be relied on to make such an inference because they are not from a random sample of the relevant population. VOLUNTARY RESPONSE POLLS One professor of statistics, who is a co-author of a highly regarded AP statistics textbook, has tried to popularize the phrase that "voluntary response data are worthless" to go along with the phrase "correlation does not imply causation." Other statistics teachers are gradually picking up this phrase. [quote=Paul Velleman] -----Original Message----- From: Paul Velleman [SMTPfv2@cornell.edu] Sent: Wednesday, January 14, 1998 5:10 PM To: apstat-l@etc.bc.ca; Kim Robinson Cc: mmbalach@mtu.edu Subject: Re: qualtiative study Sorry Kim, but it just aint so. Voluntary response data are worthless. One excellent example is the books by Shere Hite. She collected many responses from biased lists with voluntary response and drew conclusions that are roundly contradicted by all responsible studies. She claimed to be doing only qualitative work, but what she got was just plain garbage. Another famous example is the Literary Digest "poll". All you learn from voluntary response is what is said by those who choose to respond. Unless the respondents are a substantially large fraction of the population, they are very likely to be a biased -- possibly a very biased -- subset. Anecdotes tell you nothing at all about the state of the world. They can't be "used only as a description" because they describe nothing but themselves.[/quote] http://mathforum.org/kb/thread.jspa?threadID=194473&tstart=36420 http://mathforum.org/kb/thread.jspa?threadID=194473&tsta... For more on the distinction between statistics and mathematics, see http://statland.org/MAAFIXED.PDF http://statland.org/MAAFIXED.PDF and http://repositories.cdlib.org/cgi/viewcontent.cgi?article=1002&context=uclastat/cts/tise http://repositories.cdlib.org/cgi/viewcontent.cgi?article=10...
- pg 18y ago
  Surely it's incorrect to make a blanket statement that voluntary response data is worthless. Worthless means there is zero information in the data, which would only be true in extreme cases.
  - Hexstream 18y ago
    No information is better than misinformation.
    - tomsaffell 18y ago
      True some of the time, but not all of the time.
    - ErrantX 18y ago
      I'd argue that entirely depends on the context :) it's a bit of a blanket statement.
  - jbert 18y ago
    It's effectively worthless because it needs to be qualified so heavily. Dropping the qualification (as will happen sometimes) makes it more like misinformation, which is worse than worthless, so it averages out to zero... "The median poll selection of a HN visitor who clicked on a poll which was on the front page on the following weekdays (and national holidays in the following countries:...) was..." etc. If the information content is sufficiently hard to extract usefully that it would be easier to redo the sample in a sensible way, then you could call the sample worthless. (A bit like uneconomic oil reserves).
    - IsaacSchlueter 18y ago
      So, doesn't that just mean that a statistically significant portion of the regular HN users would have to respond? Of course, you'd have to figure out how to define "regular HN users" and what would be a significant portion of them. If this poll were to get that much response, it would also tell you something interesting: that HN users respond to polls in statistically significant numbers :) So, voluntary response polls are not worthless if enough people voluntarily respond to them, but this is a tricky problem.
durana 18y ago
I couldn't find a figure on the number of registered HN users, so I used the recent number of daily unique IPs from http://ycombinator.com/newsnews.html http://ycombinator.com/newsnews.html. That number is 22,000. For a confidence level of 95% with a confidence interval of 4%, we'd need 584 votes. I'd imagine there's a lot of science to polling which this calculation probably ignores.
- tokenadult 18y ago
  For a confidence level of 95% with a confidence interval of 4%, we'd need 584 votes. I'd imagine there's a lot of science to polling which this calculation probably ignores. Yep, everything about being sure we have an unbiased sample, which is not likely for a question like this. What does your calculation say about the grouping of the data into categories?
  - durana 18y ago
    I don't think the calculation says anything about the grouping of data. Or does it? If you have an unbiased sample of 584 votes and 60% of the sample responded "25 - 30", could you say with 95% confidence that 60% (+/-4%) of the sampled population would select "25 - 30"?

nx 18y ago

We need the confidence thingy for this online community giving information about their age.

The key issue is being good at statistics, not math. The data can't be relied on to make such an inference because they are not from a random sample of the relevant population. VOLUNTARY RESPONSE POLLS One professor of statistics, who is a co-author of a highly regarded AP statistics textbook, has tried to popularize the phrase that "voluntary response data are worthless" to go along with the phrase "correlation does not imply causation." Other statistics teachers are gradually picking up this phrase. [quote=Paul Velleman] -----Original Message----- From: Paul Velleman [SMTPfv2@cornell.edu] Sent: Wednesday, January 14, 1998 5:10 PM To: apstat-l@etc.bc.ca; Kim Robinson Cc: mmbalach@mtu.edu Subject: Re: qualtiative study Sorry Kim, but it just aint so. Voluntary response data are worthless. One excellent example is the books by Shere Hite. She collected many responses from biased lists with voluntary response and drew conclusions that are roundly contradicted by all responsible studies. She claimed to be doing only qualitative work, but what she got was just plain garbage. Another famous example is the Literary Digest "poll". All you learn from voluntary response is what is said by those who choose to respond. Unless the respondents are a substantially large fraction of the population, they are very likely to be a biased -- possibly a very biased -- subset. Anecdotes tell you nothing at all about the state of the world. They can't be "used only as a description" because they describe nothing but themselves.[/quote] http://mathforum.org/kb/thread.jspa?threadID=194473&tstart=36420 http://mathforum.org/kb/thread.jspa?threadID=194473&tsta... For more on the distinction between statistics and mathematics, see http://statland.org/MAAFIXED.PDF http://statland.org/MAAFIXED.PDF and http://repositories.cdlib.org/cgi/viewcontent.cgi?article=1002&context=uclastat/cts/tise http://repositories.cdlib.org/cgi/viewcontent.cgi?article=10...

pg 18y ago
Surely it's incorrect to make a blanket statement that voluntary response data is worthless. Worthless means there is zero information in the data, which would only be true in extreme cases.
- Hexstream 18y ago
  No information is better than misinformation.
  - tomsaffell 18y ago
    True some of the time, but not all of the time.
  - ErrantX 18y ago
    I'd argue that entirely depends on the context :) it's a bit of a blanket statement.
- jbert 18y ago
  It's effectively worthless because it needs to be qualified so heavily. Dropping the qualification (as will happen sometimes) makes it more like misinformation, which is worse than worthless, so it averages out to zero... "The median poll selection of a HN visitor who clicked on a poll which was on the front page on the following weekdays (and national holidays in the following countries:...) was..." etc. If the information content is sufficiently hard to extract usefully that it would be easier to redo the sample in a sensible way, then you could call the sample worthless. (A bit like uneconomic oil reserves).
  - IsaacSchlueter 18y ago
    So, doesn't that just mean that a statistically significant portion of the regular HN users would have to respond? Of course, you'd have to figure out how to define "regular HN users" and what would be a significant portion of them. If this poll were to get that much response, it would also tell you something interesting: that HN users respond to polls in statistically significant numbers :) So, voluntary response polls are not worthless if enough people voluntarily respond to them, but this is a tricky problem.

durana 18y ago

I couldn't find a figure on the number of registered HN users, so I used the recent number of daily unique IPs from http://ycombinator.com/newsnews.html http://ycombinator.com/newsnews.html. That number is 22,000. For a confidence level of 95% with a confidence interval of 4%, we'd need 584 votes. I'd imagine there's a lot of science to polling which this calculation probably ignores.

tokenadult 18y ago
For a confidence level of 95% with a confidence interval of 4%, we'd need 584 votes. I'd imagine there's a lot of science to polling which this calculation probably ignores. Yep, everything about being sure we have an unbiased sample, which is not likely for a question like this. What does your calculation say about the grouping of the data into categories?
- durana 18y ago
  I don't think the calculation says anything about the grouping of data. Or does it? If you have an unbiased sample of 584 votes and 60% of the sample responded "25 - 30", could you say with 95% confidence that 60% (+/-4%) of the sampled population would select "25 - 30"?