4 ms·
Bayesian methods are not easy to use, I agree. But that's because they're trying to answer much more meaningful (but harder) questions than frequentist ones, wh
by civilized 3y ago
Bayesian methods are not easy to use, I agree. But that's because they're trying to answer much more meaningful (but harder) questions than frequentist ones, which researchers should be trying to do. You can't ignore Bayesian epistemology just by not using Bayesian methods. The underlying considerations in the Bayesian framework will inevitably become relevant to how you interpret your data, whether or not you use a formal Bayesian method.
The thing is, formally, frequentist methods like Null Hypothesis Significance Testing don't tell you what you really want to know. If you get a significant p-value, that means the data you observed wouldn't often happen by chance (within your model of the null). This doesn't actually tell you if your particular hypothesis should be favored. That requires other considerations, including ones that Simonsohn is negative about in this article.
For example, Simonsohn's conclusion says:
> To use Bayes factors to test hypotheses: you need to be OK with the following two things:
> 1. Accepting the null when “the alternative” you consider, and reject, does not represent the theory of interest.
> 2. Rejecting a theory after observing an outcome that the theory predicts.
He implies that these should be points against Bayes factors. But #2 is something you actually should do sometimes. Demonstrably. If the data suggests a wildly implausible effect size that doesn't show up consistently in other analyses, that should be a point against your theory and in favor of some more mundane explanation, like noisy data from an underpowered study [1].
Not using Bayesian methods is understandable if you don't feel comfortable with the very heavy demands they can make on your statistics acumen. But if you're, say, a social scientist incentivized to get "sexy" results and you refuse to engage with Bayesian epistemology at all, your career will almost certainly just be a contribution of more noise publications to the replication crisis.
[1] http://www.stat.columbia.edu/~gelman/presentations/ziff.pdf http://www.stat.columbia.edu/~gelman/presentations/ziff.pdf
- kkoncevicius 3y ago> Bayesian methods are not easy to use, I agree. But that's because they're trying to answer much more meaningful (but harder) questions than frequentist ones I think your view about the difference between Frequentist and Bayesian methods is wrong. There is this rant I like from Larry Wasserman [1] on the subject: My opinions have shifted a bit. [...] Bayes-Frequentist debate still matters. And people — including many statisticians — are still confused about the distinction. I thought the basic Bayes-Frequentist debate was behind us. A year and a half of blogging (as well as reading other blogs) convinced me I was wrong here too. And this still does matter. My emphasis on high-dimensional models is germane, however. In our world of high-dimensional, complex models I can’t see how anyone can interpret the output of a Bayesian analysis in any meaningful way. I wish people were clearer about what Bayes is/is not and what frequentist inference is/is not. Bayes is the analysis of subjective beliefs but provides no frequency guarantees. Frequentist inference is about making procedures that have frequency guarantees but makes no pretense of representing anyone’s beliefs. In the high dimensional world, you have to choose: objective frequency guarantees or subjective beliefs. Choose whichever you prefer, but you can’t have both. I don’t care which one people pick; I just wish they would be clear about what they are giving up when they make their choice. [...] Of course, one can embrace objective Bayesian inference. If this means “Bayesian procedures with good frequentist properties” then I am all for it. But this is just frequentist inference in Bayesian clothing." [1]: https://errorstatistics.com/2013/12/27/deconstructing-larry-wasserman/#comment-18941 https://errorstatistics.com/2013/12/27/deconstructing-larry-...
- kgwgk 3y ago> But this is just frequentist inference in Bayesian clothing. Or maybe this is "just" frequentist inference done right - on a Bayesian foundation.
- civilized 3y agoHaving a specific opinion isn't inherently a bias, and it's rather uninspiring to put effort into a substantial comment and then get a reply which is essentially nothing but a baseless accusation of bias followed by a long block quote of unclear relevance (it's extremely far from true that all statistical analyses worth consideration are in high dimensions). Review the Hacker News guidelines: > Edit out swipes. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html EDIT: much appreciated.
- kkoncevicius 3y agoHm, that line doesn't seem bad to me. But I can certainly edit it if you think it's harsh in some way. Which would you prefer?: 1. "I think your view about the difference between Frequentist and Bayesian methods is biased" 2. "I think your view about the difference between Frequentist and Bayesian methods is wrong". 3. Any other suggestion?
- civilized 3y agoI don't find it a helpful comment either way, but "wrong" would at least shed the implication of measuring unfairness from an objective standpoint.
- kkoncevicius 3y agoA Bayesian proponent wanting an objective standpoint ... Sorry, could not resist :) But I edited the comment. Sorry if it ruined your mood. I shared it with good intentions, not to pick a fight.
- 3y ago