2 ms·
(Note: Second paragraph in grandparent was edit, original reply in parent may have been to first paragraph only.) Well, keeping in mind that experimenters sho
by Eliezer 16y ago
(Note: Second paragraph in grandparent was edit, original reply in parent may have been to first paragraph only.)
Well, keeping in mind that experimenters should just report likelihood ratios, I'd have estimated a pretty low prior probability that one of two identical pages produced more signups than the other.
But I suppose that after seeing a likelihood ratio of 1000/1 favoring the hypothesis "each of these pages has a separate fixed conversion rate" over "both of these pages have the same fixed conversion rate", using Laplace's Rule of Succession (i.e. a flat prior between 0 and 1 for the conversion rate) in all cases, I'd think that something was more likely than not going on.
- carbocation 16y agoFor academic journals, I think that what you propose is totally reasonable. I'll try to give a small taste for those not familiar: For a new HIV test, I want to know the likelihood ratios, not other parameters which tend to be driven by the background HIV prevalence in the researcher's specific catchment area. My catchment area's HIV prevalence will differ, so other aspects of the test that are dominated by prevalence will not be the same in my population. However, the likelihood ratio will be. The point that I didn't finish making before getting kicked out of the coffee shop was that, at some point, people will want to compute a posterior, so they'll need to figure out a prior. Your suggestion above is helpful. Edit - Also, you're right in that I saw just the 1st paragraph when I initially replied.