4 ms·

If you posit that 50% of people like X, you survey 21 people and find that 0 like X, then the odds that you would get that result due to random sampling error a

If you posit that 50% of people like X, you survey 21 people and find that 0 like X, then the odds that you would get that result due to random sampling error assuming the original hypothesis were true is 1 in 2097152

danielvf 10y ago
However that doesn't take into account correlated sampling error. If you randomly sample college social science majors at a single school, you could easily get results like that, even if the rest of the population is actually split on the matter.
- dismantlethesun 10y ago
  That's true, but that's also not a random sampling so doesn't really detract from what the above poster is saying. Most social science experiments do not initially handle random samples. The famous ones like the Milgram experiments and others conducted by the same researcher do. If I have my facts correct, he used to mail out letters to people randomly chosen from the phonebook to ask them to participate in his study, then travel to their locales to study them. The Milgram experiment itself was redone multiple times, using different population segments---students, random people in the city, all women, all men, etc. As an anecdote of how poorly correlated a social experiment can be to real life---in my university, econ & social science students were required to take at least 10 experiments per semester. Often we would go there, realize the material tested was identical to what we had studied in class then behave like 'rational market actors' in order to get the best result (econ experiments used small sums of real money, so you'd be paid at the end based on performance). This isn't anything like how a random person would have behaved.
  - jdavis703 10y ago
    And I always behaved as an irrational market actor, because hey we exist in the real world too. I know in some of my finances I don't behave as a model would predict, for instance investing in companies that are doing something that is ethically interesting to me, despite the financials. Or doing the inverse if a company seems to be doing something wrong, even if it's perfectly legal and increasing the share price.
i__believe 10y ago
Question: how does one go about finding that number?
- AimHere 10y ago
  The commenter just described the probability of tossing a coin 21 times and all cointosses being heads, so (1/2)^21, or 1 in 2^21. It's a specific case of the cumulative binomial distribution which is what you might look up if you want to know about the case where the probability of people liking X in general isn't 50% but something else, or the number of samples isn't 21, or the number of sampled people liking X isn't 0.

danielvf 10y ago

However that doesn't take into account correlated sampling error. If you randomly sample college social science majors at a single school, you could easily get results like that, even if the rest of the population is actually split on the matter.

dismantlethesun 10y ago
That's true, but that's also not a random sampling so doesn't really detract from what the above poster is saying. Most social science experiments do not initially handle random samples. The famous ones like the Milgram experiments and others conducted by the same researcher do. If I have my facts correct, he used to mail out letters to people randomly chosen from the phonebook to ask them to participate in his study, then travel to their locales to study them. The Milgram experiment itself was redone multiple times, using different population segments---students, random people in the city, all women, all men, etc. As an anecdote of how poorly correlated a social experiment can be to real life---in my university, econ & social science students were required to take at least 10 experiments per semester. Often we would go there, realize the material tested was identical to what we had studied in class then behave like 'rational market actors' in order to get the best result (econ experiments used small sums of real money, so you'd be paid at the end based on performance). This isn't anything like how a random person would have behaved.
- jdavis703 10y ago
  And I always behaved as an irrational market actor, because hey we exist in the real world too. I know in some of my finances I don't behave as a model would predict, for instance investing in companies that are doing something that is ethically interesting to me, despite the financials. Or doing the inverse if a company seems to be doing something wrong, even if it's perfectly legal and increasing the share price.

i__believe 10y ago

Question: how does one go about finding that number?

AimHere 10y ago
The commenter just described the probability of tossing a coin 21 times and all cointosses being heads, so (1/2)^21, or 1 in 2^21. It's a specific case of the cumulative binomial distribution which is what you might look up if you want to know about the case where the probability of people liking X in general isn't 50% but something else, or the number of samples isn't 21, or the number of sampled people liking X isn't 0.