4 ms·
A statistician could talk me out of this, but I've always been puzzled by the use of "prior." Nothing in Bayes's Theorem says that the prior has to be establish
by analog31 8y ago
A statistician could talk me out of this, but I've always been puzzled by the use of "prior." Nothing in Bayes's Theorem says that the prior has to be established before anything else. It makes Bayesian statistics seem like a matter of letting your expectations influence your results.
Instead, an interpretation that seems more favorable to me, is that you consider all of the information at your disposal, that can be brought to bear on a problem, and this could include known constraints on likelihoods. Bayes's Theorem becomes a tool for working through problems where a single statistic can't be readily used to analyze an entire data set, e.g., when data come from disparate sources and can't be readily combined.
- lgas 8y agoI'm not a statistician but as I understand it, in Bayesian terms, when you get some new piece of information you must update your beliefs to incorporate the new information. So you take your belief prior to the new information and incorporate the new information to get your updated belief.
- analog31 8y agoThat's cool. It sounds like how we all learned to think since time immoral, so I can hardly dispute it.
- hooloovoo_zoo 8y agoIt sounds like you're talking about just using the maximum likelihood estimator (with a special likelihood) which is similar to using a flat (improper) prior. One disadvantage of doing so is that the results aren't invariant to reparameterization of the problem. The latter problem can be fixed by using Jeffreys' priors. However, the MLE is very effective in its own right.
- sjg007 8y agoA prior could be a population level statistic which is then updated by a likelihood to reveal a posterior probability. Take for example a medical test. We know that on average that 1/10 people have some disease X. We have a test that tests for X (with some false positive rate).. Using that we can calculate the posterior probability that you actually have disease X. If you relied just on the test you don't discount the population level statistics which might be relevant. You can also use the tests FP rate in the same way as your prior... e.g. the people who test positive who don't have the disease etc...
- sl8r 8y agoPart of the issue is that you can't not assume a prior; it's unavoidable. The Bayesian POV just makes this assumption clear / explicit, while many frequentist methods (if naively applied) amount to choosing a uniform prior. E.g., imagine you have an e-commerce site that has, historically, had a 2% conversion rate (landing page to purchase). Now you run an A/B test with two variants, a control (A) and a treatment (B). Both buckets get 10,000 landings, of which A converts 200 of them and B converts 250. How can you tell if A is better than B? Cutting to the chase, the frequentist approach (if applied naively) would be to model B as some distribution centered around 2.5%, for example N(2.5%, 0.15%) or Beta(251, 9751). A Bayesian would say that this assumes a uniform prior - but that this is probably a bad prior because it ignores what we know about the historical conversion rate of 2.0%. Said another way, the above amounts to saying (before we run the test) that we think it's just as likely for B to have a conversion rate of 2.5% as it is to have a conversion rate of 100%. Clearly we don't actually believe this.
- adwn 8y ago> Cutting to the chase, the frequentist approach (if applied naively) would be to model B as some distribution centered around 2.5% Why would that be wrong? > A Bayesian would say that this assumes a uniform prior - but that this is probably a bad prior because it ignores what we know about the historical conversion rate of 2.0%. What would a Bayesian conclude instead?
- sl8r 8y ago> Why would that be wrong? The issue is that modeling B with a distro centered around 2.5% ignores what we know about the historical conversion rate (2.0%) and the control bucket's conversion rate (also 2.0%). If our goal is to make the best estimate for the future that we can, we should take this data into account when evaluating B. As a thought experiment, imagine that you have A at 2.0% and B at 2.5% conversion for Week 1, with a historical conversion rate of 2.0%. Someone says they'll pay you $100 if you correctly guess what B's conversion rate will be next week, either (i) in the range 2.0% to 2.5%, or (ii) in the range 2.5% to 3.0%. I'd prefer to bet on (i) than on (ii). > What would a Bayesian conclude instead? One simple approach would just be to start with a more informative prior, like Beta(2+1,100-2+1) instead of Beta(1,1). This would pull bucket B's posterior distribution closer to 2.0%. Another approach is to use a hierarchical model [1], which will fit the individual buckets' priors for you. [1] Here's something I wrote on this a couple years ago, more focused on solving multiple comparisons problems but with the same proposed solution: http://normal-extensions.com/2014/07/16/ab-testing-hierarchical-model/ http://normal-extensions.com/2014/07/16/ab-testing-hierarchi...
- valgor 8y ago>It makes Bayesian statistics seem like a matter of letting your expectations influence your results. The smart approach is to have a set of a prior expectations, run through the data, and see what comes out the other side. Essentially, through a large amount of experimentations, you can narrow down what assumptions were indeed correct.