5 ms·
Negative binomial / binomial experiments: frequentist or Bayesian?
- contravariant 9y agoSurely it isn't that surprising that the outcomes of two different experiments can end up providing the exact same information? In fact this case can be inferred from the fact that the Beta distribution is the conjugate prior distribution for both the binomial and negative binomial distribution. There are many more distributions that have the Beta distribution as conjugate prior to one of the variables (in fact it's pretty much all distributions with a factor p or (1-p) somewhere).
- tel 9y agoIt is often considered surprising and controversial. In particular, if the experiment performed included "controversial" stopping criteria then you could imagine the same data arising but not trusting each result identically. Arguments exist for each side.
- RA_Fisher 9y agoYep, I should add that portion. You're spot on. This was surprising to me because I've shown both models contain the same information but identical Frequentist hypothesis tests will show different p-values. This unshown part is what suprised me. The next post! :)
- contravariant 9y agoSo I guess the problem here is that to calculate the probability that the likelihood exceeds a certain bound can't be calculated solely from the likelihood (+prior) itself. Yet interestingly, the likelihood + prior is enough to make statements about the distribution of the parameters you're looking at. So you can say something about the certainty that your parameter exceeds a certain bound. I suppose it really depends on your application which you'd want to use.
- tel 9y agoThat's a Bayesian perspective, but the heart of the Likelihood Principle is that non-Bayesians/non-Likelihoodists can find reason to believe that the Likelihood doesn't really contain all of the relevant information for making inferences.
- nonbel 9y ago> "all of the relevant information for making inferences" I think this terminology is way too strong. Obviously the context matters. For example, the way the data is collected (whether or not the data collector is a known liar/p-hacker, the sensor is known to malfunction at certain temperatures, etc) should affect inference.
- tel 9y agoRight, and the Likelihoodist/Bayesian would put those details into the model. Real world concerns get handled by each side in acceptable ways, but very special details about how sampling and posterior distributions differ are what make this tricky.
- RA_Fisher 9y agoThanks, I didn't realize it was related to conjugacy!
- gerdesj 9y agoApril 1st is three months away.
- nerdponx 9y ago> Frequentist: yes. I don't follow.
- RA_Fisher 9y agoI ought to directly show this, but Frequentist hypothesis tests will produce different p-values depending on which model is chosen.
- RA_Fisher 9y agoSee this document with the calculation: https://www2.isye.gatech.edu/~brani/isyebayes/bank/handout2.pdf https://www2.isye.gatech.edu/~brani/isyebayes/bank/handout2....
- gerdesj 9y agoI read that pdf and it is not exactly rigorous. Eg page 4: "Should depend only on the outcome observed and not on any other outcome we might have observed and thus sharply contrasts with the method of likelihood inference from the Neyman-Pearson, or more generally from a frequentist, approach. In particular, questions of unbiasedness, minimum variance and risk, consistency, the whole apparatus of confidence intervals, significance levels, and power of tests, etc., violate the conditionality principle." and "Here is yet another scenario that will not impress a conditionalist:" You don't write a paper to trash a POV. You put your own thesis across. That thing is bollocks.
- tel 9y agoFrequentist analyses will depend upon assumptions of the sampling process which often include specific stopping conditions. This means that the _reason_ an experiment ends is an important part of data. That exactly defies the likelihood principle since it means that our outcome is dependent on information not observed.
- RA_Fisher 9y agoYes! I feel like I'm finally getting to this crux in the difference between the Bayesian / information theoretic methods and Frequentism.
- nonbel 9y agoWhat is the purpose of the call to prod here? negative_binomial_likelihood <- function(p) { prod(dnbinom(40, 7, p)) } binomial_likelihood <- function(p) { prod(dbinom(7, 47, p)) }
- RA_Fisher 9y agoNice eye! In this case it's a no op and the function has no effect because I've used the `rowwise()` call such that the data frame is evaluated one row at a time x 1. The `prod` call is an artifact of me previously using the function in a vectorized manner to calculate the model's likelihood.[1] This isn't the common way one might see. More often one would work with the log likelihood [2]. The reason being that the product of the density can be converted into sums avoiding overflow errors. The likelihood is __very cool_. I liken it to a grand generalization of the needle of a record player wrt to information (entropy). [1] https://en.wikipedia.org/wiki/Maximum_likelihood_estimation#Principles https://en.wikipedia.org/wiki/Maximum_likelihood_estimation#... [2] https://en.wikipedia.org/wiki/Likelihood_function#Log-likelihood https://en.wikipedia.org/wiki/Likelihood_function#Log-likeli...
- nonbel 9y agoIn R you can just plug in a vector of probabilities directly to get the likelihood. Ie: p = seq(.01, .99, by = .01) y1 = dbinom(7, 47, p) y2 = dnbinom(40, 7, p) It doesn't matter much for this use case but that will also be much faster. It does simplify the code quite a bit too though.
- RA_Fisher 9y agoThank you for pushing me to see that, indeed I should take advantage of the vectorized aspect of this code!