6 ms·

Frequentist has a prior also though. The uniform distribution. No. Experimental design affect frequentist conclusions in a way that is inconsistent with _ANY_

by btilly 7y ago

Frequentist has a prior also though. The uniform distribution. No. Experimental design affect frequentist conclusions in a way that is inconsistent with _ANY_ prior. Here is a real life example. My aunt and uncle had 7 children. 6 boys and one girl. Were they biased towards having one gender over another? If the null hypothesis is that they aren't, the p-value that you get is easily calculated as 16/2^7 = 1/8 = 0.125. (There is 1 arrangement of 7 girls, 7 of 6 girls and a boy, 7 of 6 boys and a girl, and 1 of 7 boys for 16 equally likely arrangements.) If I add the fact that they planned to have children until they had a boy and a girl, then that changes the p-value. In fact there are only 4 ways that their first 7 children can come out to give evidence this strong against the prior. So the p-value is now 4/2^7 = 1/32 = 0.03125. However a Bayesian looks at this and says that no matter what prior you pick, the knowledge that they planned to have children until they had both a boy and a girl does not affect your posterior conclusion. It literally has nowhere to go in the formula and can't make a difference. Therefore the frequentist's differing conclusions are not consistent with ANY prior, implicit or not.

pdonis 7y ago
> a Bayesian looks at this and says that no matter what prior you pick, the knowledge that they planned to have children until they had both a boy and a girl does not affect your posterior conclusion A Bayesian would say no such thing. A Bayesian would agree that the knowledge that they planned to have children until they had both a boy and a girl doesn't affect your prior: you still are picking how much probability mass you allocate to all of the possible odds of having a boy vs. a girl, and the couple's plans don't affect that. However, a Bayesian would also say that the knowledge that they planned to have children until they had both a boy and a girl significantly changes the likelihood ratio (or p-value, if you prefer to use that) associated with the observed data. And one of the advantages of Bayesianism is that it forces you to make that explicit as well. Notice, for example, that when you calculated the first p-value of 1/8, you implicitly assumed that the couple's plan was "have 7 children, no matter what gender each of them is". The sample space is therefore all possible arrangements of 7 children by gender, and the p-value is 1/8, as you say. But when you calculated the second p-value of 1/32, while you did change the count of arrangements, you failed to recognize that the sample space changed! Now the possibilities are not just all possible arrangements of 7 children (which is what you used), but all possible arrangements of up to 7 children (because the "stop condition" now is not when there are 7 children total, but when there is at least one child of each gender, and that could have happened at a number of children less than 7). So the correct p-value is not 4/2^7, but 4/(2^7 + 2^6 + 2^5 + 2^4 + 2^3 + 2^2) = 4/(2^8 - 2) = 2/127. A Bayesian, who has to calculate the p-value starting from the hypothesis, not the data, would not make that mistake. And Bayesianism does something else too: it forces you to recognize that the p-value is not actually the answer to the question you were asking! By the p-value criterion, at least with the typical threshold of 0.05, the null hypothesis (that your aunt and uncle are not biased towards having one gender) is rejected. But a Bayesian recognizes that the prior probability of the gender ratio, based on abundant previous evidence, is strongly peaked around 50-50, much more strongly peaked than data with a bias equivalent to a p-value of 2/127 can overcome. So the Bayesian is quite ready to accept that your aunt and uncle had no actual bias towards having boys, they just happened to be one of the statistical outliers that are to be expected given the huge number of humans who have children.
- idoubtit 7y ago
  > the sample space changed! Now the possibilities are not just all possible arrangements of 7 children, but all possible arrangements of up to 7 children [...] > the "stop condition" now is not when there are 7 children total Your answer makes no sense to me. If you consider the space of possibles combinations that can lead to having a boy and a girl, why do you stop at seven children. Why consider five boys and one girl but reject seven boys and one girl? Both of them are end cases that could be reached.
  - pdonis 7y ago
    Yes, I was posting in a rush and was being sloppy. Here's a more detailed calculation. The process involved is that the couple continues to have children until they have at least one of each gender. If we assume that at each birth there is a probability p of having a boy (as I noted in my response to btilly elsewhere, the Bayesian prior would actually be a distribution for p, not a point value, but I'll ignore that here for simplicity), then the process can be modeled as a branching tree something like this: Child #1: boy -> p; girl -> 1 - p Child #2: boy - boy -> p^2; boy - girl -> p(1 - p) : STOP; girl - boy -> (1 - p)p : STOP; girl - girl -> (1 - p)^2 So we have a probability of 2p(1 - p) of stopping at child 2. Child #3: boy - boy - boy -> p^3; boy - boy - girl -> p^2(1 - p) : STOP; girl - girl - boy -> p(1 - p)^2 : STOP; girl - girl - girl -> (1 - p)^3 So we have a probability of [1 - 2p(1 - p)] [p^2(1 - p) + p(1 - p)^2] of stopping at child 3 (the first factor comes from the probability that we didn't stop at child 2 above). By a similar process we can carry out the tree for as many children as we want. For the case p = 1/2, which was the case I was considering, all of these expressions for the probability of stopping at child #N (for N > 1) simplify to 1 / 2^(N - 1). So the probability of stopping at or before child #N is the sum of those probabilities from 2 to N; for N = 7 that is 1/2 + 1/4 + 1/8 + 1/16 + 1/32 + 1/64 = 63/64. That is close enough to 1 that I ignored cases with more than 7 children; but for a more exact calculation you could add an extra 1/64 to the denominator used to calculate the likelihood (or p-value) of the specific case that was actually observed, to allow for the cases with more than 7 children.
    - btilly 7y ago

by btilly 7y ago

pdonis 7y ago

> a Bayesian looks at this and says that no matter what prior you pick, the knowledge that they planned to have children until they had both a boy and a girl does not affect your posterior conclusion A Bayesian would say no such thing. A Bayesian would agree that the knowledge that they planned to have children until they had both a boy and a girl doesn't affect your prior: you still are picking how much probability mass you allocate to all of the possible odds of having a boy vs. a girl, and the couple's plans don't affect that. However, a Bayesian would also say that the knowledge that they planned to have children until they had both a boy and a girl significantly changes the likelihood ratio (or p-value, if you prefer to use that) associated with the observed data. And one of the advantages of Bayesianism is that it forces you to make that explicit as well. Notice, for example, that when you calculated the first p-value of 1/8, you implicitly assumed that the couple's plan was "have 7 children, no matter what gender each of them is". The sample space is therefore all possible arrangements of 7 children by gender, and the p-value is 1/8, as you say. But when you calculated the second p-value of 1/32, while you did change the count of arrangements, you failed to recognize that the sample space changed! Now the possibilities are not just all possible arrangements of 7 children (which is what you used), but all possible arrangements of up to 7 children (because the "stop condition" now is not when there are 7 children total, but when there is at least one child of each gender, and that could have happened at a number of children less than 7). So the correct p-value is not 4/2^7, but 4/(2^7 + 2^6 + 2^5 + 2^4 + 2^3 + 2^2) = 4/(2^8 - 2) = 2/127. A Bayesian, who has to calculate the p-value starting from the hypothesis, not the data, would not make that mistake. And Bayesianism does something else too: it forces you to recognize that the p-value is not actually the answer to the question you were asking! By the p-value criterion, at least with the typical threshold of 0.05, the null hypothesis (that your aunt and uncle are not biased towards having one gender) is rejected. But a Bayesian recognizes that the prior probability of the gender ratio, based on abundant previous evidence, is strongly peaked around 50-50, much more strongly peaked than data with a bias equivalent to a p-value of 2/127 can overcome. So the Bayesian is quite ready to accept that your aunt and uncle had no actual bias towards having boys, they just happened to be one of the statistical outliers that are to be expected given the huge number of humans who have children.

idoubtit 7y ago
> the sample space changed! Now the possibilities are not just all possible arrangements of 7 children, but all possible arrangements of up to 7 children [...] > the "stop condition" now is not when there are 7 children total Your answer makes no sense to me. If you consider the space of possibles combinations that can lead to having a boy and a girl, why do you stop at seven children. Why consider five boys and one girl but reject seven boys and one girl? Both of them are end cases that could be reached.
- pdonis 7y ago
  Yes, I was posting in a rush and was being sloppy. Here's a more detailed calculation. The process involved is that the couple continues to have children until they have at least one of each gender. If we assume that at each birth there is a probability p of having a boy (as I noted in my response to btilly elsewhere, the Bayesian prior would actually be a distribution for p, not a point value, but I'll ignore that here for simplicity), then the process can be modeled as a branching tree something like this: Child #1: boy -> p; girl -> 1 - p Child #2: boy - boy -> p^2; boy - girl -> p(1 - p) : STOP; girl - boy -> (1 - p)p : STOP; girl - girl -> (1 - p)^2 So we have a probability of 2p(1 - p) of stopping at child 2. Child #3: boy - boy - boy -> p^3; boy - boy - girl -> p^2(1 - p) : STOP; girl - girl - boy -> p(1 - p)^2 : STOP; girl - girl - girl -> (1 - p)^3 So we have a probability of [1 - 2p(1 - p)] [p^2(1 - p) + p(1 - p)^2] of stopping at child 3 (the first factor comes from the probability that we didn't stop at child 2 above). By a similar process we can carry out the tree for as many children as we want. For the case p = 1/2, which was the case I was considering, all of these expressions for the probability of stopping at child #N (for N > 1) simplify to 1 / 2^(N - 1). So the probability of stopping at or before child #N is the sum of those probabilities from 2 to N; for N = 7 that is 1/2 + 1/4 + 1/8 + 1/16 + 1/32 + 1/64 = 63/64. That is close enough to 1 that I ignored cases with more than 7 children; but for a more exact calculation you could add an extra 1/64 to the denominator used to calculate the likelihood (or p-value) of the specific case that was actually observed, to allow for the cases with more than 7 children.
  - btilly 7y ago