3 ms·

See the math here: https://news.ycombinator.com/item?id=11965961 https://news.ycombinator.com/item?id=11965961 You can play with numbers to see exactly how im

See the math here: https://news.ycombinator.com/item?id=11965961 https://news.ycombinator.com/item?id=11965961 You can play with numbers to see exactly how important the base rate is. If you believe your screening process is 99.9999% perfect, then most likely base rates are nearly irrelevant. I don't think my screening process is that good.

hackinthebochs 10y ago
The math isn't what I'm questioning. The math doesn't tell you about the quality of your prior, just how to factor it in. But if your prior is crud, then you end up making worse decisions as a result. By using the prior admissions test scores you are assuming that those factors are relevant to a population that has graduated and passed your own screening. This is not an "uninformed prior", you are explicitly modelling the belief of the relevance of entrance scores. Modelling no belief would be a uniform prior, that is no bias in the population.
- yummyfajitas 10y ago
  By using the prior admissions test scores you are assuming that those factors are relevant to a population that has graduated and passed your own screening. Unless applying your screening to a population somehow magically eliminates all variation that previously existed across the populations, that's a perfectly valid assumption. The argument that if the prior is wrong, it hurts you is of course true. Similarly a uniform prior in a non-uniform situation will hurt you - see this example for an illustration of how ignoring base rates can spread HIV. https://www.chrisstucchio.com/blog/2016/why_gays_cant_donate_blood.html https://www.chrisstucchio.com/blog/2016/why_gays_cant_donate...
  - hackinthebochs 10y ago
    The difference in the HIV case is that we have an actual population sample that justifies our chosen base rate. That is to say, this isn't a "prior belief", this is a measured statistic. It's applicability is precisely because the population of individuals getting an HIV test is random from the general population, from which we have a measured statistic. But the use of the base rate in this case does not justify the usage of informative priors in every case. >Unless applying your screening to a population somehow magically eliminates all variation that previously existed across the populations But it might. The "population" of graduated + passed screening bears no a priori relationship to the population of entering freshmen precisely because this is a non-random sample, orders of magnitude smaller than the original population. Furthermore, there is further evidence that there is no relationship precisely because of the studies that show that entrance exams bare little correlation to college success past the first year.

hackinthebochs 10y ago

The math isn't what I'm questioning. The math doesn't tell you about the quality of your prior, just how to factor it in. But if your prior is crud, then you end up making worse decisions as a result. By using the prior admissions test scores you are assuming that those factors are relevant to a population that has graduated and passed your own screening. This is not an "uninformed prior", you are explicitly modelling the belief of the relevance of entrance scores. Modelling no belief would be a uniform prior, that is no bias in the population.

yummyfajitas 10y ago
By using the prior admissions test scores you are assuming that those factors are relevant to a population that has graduated and passed your own screening. Unless applying your screening to a population somehow magically eliminates all variation that previously existed across the populations, that's a perfectly valid assumption. The argument that if the prior is wrong, it hurts you is of course true. Similarly a uniform prior in a non-uniform situation will hurt you - see this example for an illustration of how ignoring base rates can spread HIV. https://www.chrisstucchio.com/blog/2016/why_gays_cant_donate_blood.html https://www.chrisstucchio.com/blog/2016/why_gays_cant_donate...
- hackinthebochs 10y ago
  The difference in the HIV case is that we have an actual population sample that justifies our chosen base rate. That is to say, this isn't a "prior belief", this is a measured statistic. It's applicability is precisely because the population of individuals getting an HIV test is random from the general population, from which we have a measured statistic. But the use of the base rate in this case does not justify the usage of informative priors in every case. >Unless applying your screening to a population somehow magically eliminates all variation that previously existed across the populations But it might. The "population" of graduated + passed screening bears no a priori relationship to the population of entering freshmen precisely because this is a non-random sample, orders of magnitude smaller than the original population. Furthermore, there is further evidence that there is no relationship precisely because of the studies that show that entrance exams bare little correlation to college success past the first year.