3 ms·

Not disputing the paper or the scientific method applied here, but how is "bayesian factor" different than p-values?

robterrin 9y ago
This might answer your question: http://www.stat.columbia.edu/~gelman/research/published/avoiding.pdf http://www.stat.columbia.edu/~gelman/research/published/avoi...
- nurettin 9y ago
  I don't see "Bayesian factor" in there.
Asdfbla 9y ago
Well, for one thing it doesn't carry that subconscious bias of interpreting something below 0.05 as guaranteed significant as people tend to do with p-values, but is a softer metric that reflects your belief in one or the other hypothesis, given the data. I think that in itself makes it easier to evaluate and reason about. The fact that you can incorporate (and reflect on) a priori assumptions about the likelihood of one or the other model is another advantage imho.
tmalsburg2 9y ago
In a nutshell: If the p-value is below some conventional threshold (in psychology it's usually 0.05), you can conclude that the null-hypothesis (no effect) is likely wrong and that something else must be the case. However, if p is above the threshold you can't conclude anything, because that means that there is either no effect or that the data was not sufficient to show the effect. So the p-value method (formally known as null-hypothesis significance testing, NHST) can only ever be used to reject the null hypothesis but not to support it. This is where Bayes factors are different. In that approach, you formulate not just one hypothesis but two. Then you test how well both hypotheses are supported by the data. Three conclusions are possible: Hypothesis A receives more support, the alternative hypothesis receives more support, there is not enough evidence to conclude anything with sufficient certainty. Presented this way, the Bayes factor approach appears to be strictly superior to NHST. But there are also some complications that make Bayes factors tricky (related to how you select and formulate the hypotheses) and some statisticians go as far as to say that Bayes factors are obviously non-sense (e.g. Andrew Gelman). The truth is probably that they are neither a panacea nor non-sense.
- nurettin 9y ago
  That makes a lot of sense, but can't you model two null hypotheses using the same data in some clever way?

This might answer your question: http://www.stat.columbia.edu/~gelman/research/published/avoiding.pdf http://www.stat.columbia.edu/~gelman/research/published/avoi...

nurettin 9y ago
I don't see "Bayesian factor" in there.

Asdfbla 9y ago

Well, for one thing it doesn't carry that subconscious bias of interpreting something below 0.05 as guaranteed significant as people tend to do with p-values, but is a softer metric that reflects your belief in one or the other hypothesis, given the data. I think that in itself makes it easier to evaluate and reason about. The fact that you can incorporate (and reflect on) a priori assumptions about the likelihood of one or the other model is another advantage imho.

tmalsburg2 9y ago

In a nutshell: If the p-value is below some conventional threshold (in psychology it's usually 0.05), you can conclude that the null-hypothesis (no effect) is likely wrong and that something else must be the case. However, if p is above the threshold you can't conclude anything, because that means that there is either no effect or that the data was not sufficient to show the effect. So the p-value method (formally known as null-hypothesis significance testing, NHST) can only ever be used to reject the null hypothesis but not to support it. This is where Bayes factors are different. In that approach, you formulate not just one hypothesis but two. Then you test how well both hypotheses are supported by the data. Three conclusions are possible: Hypothesis A receives more support, the alternative hypothesis receives more support, there is not enough evidence to conclude anything with sufficient certainty. Presented this way, the Bayes factor approach appears to be strictly superior to NHST. But there are also some complications that make Bayes factors tricky (related to how you select and formulate the hypotheses) and some statisticians go as far as to say that Bayes factors are obviously non-sense (e.g. Andrew Gelman). The truth is probably that they are neither a panacea nor non-sense.

nurettin 9y ago
That makes a lot of sense, but can't you model two null hypotheses using the same data in some clever way?