3 ms·
Because hessenwolf and I are philosophically opposed here, I'll give my take. Balance your impressions between us as you choose. --- Frequentist statistics at
by tel 16y ago
Because hessenwolf and I are philosophically opposed here, I'll give my take. Balance your impressions between us as you choose.
---
Frequentist statistics attempts to answer the question "How will my experimentation appear knowing that there is some hidden, unknown truth to the world which generates it?" The methods then proceed to use a variety of clever arguments to show that seeing a certain experimental result (considering all possible experimental results) constrains the possible underlying reality and gives you a good guess at to what it is (and allows you to estimate how much it might vary).
Bayesian statistics asks the very different question "How does this observation I'm making affect my current knowledge of the world?" It is pretty difficult to look at the methods without seeing an interpretive nod toward the process of learning. To do this update step, Bayesians consider the relative likelihood of all possible underlying realities given that they've seen said experiment.
It's not clear to me that these two methods are at all asking the same question. In particular, they each consider (marginalize, integrate) vastly different properties and their results have different interpretations. However, since both of them fit into the space of quantifying the effect of observation on the parameters of a model of the world they end up in constant conflict.
Moreover, it's easy to construct Bayesian arguments which correspond to exactly the same algorithms as some Frequentist arguments. Bayesians argue then that their path to reach that algorithm is more interpretable and clear, especially to non-mathematician. This method collision serves to further conflate the two methods as enemies.
---
tl;dr: Bayesian statistics is an average over possible realities, Frequent averages over possible experimental outcomes. It's not clear that these are comparable at all, but since they often try to answer the same questions we compare them anyway.
- hessenwolf 16y agoNope. I agree with what you are saying; and would like my research presented as summary statistics under different models (of which the p-value is one) for sciency stuff, and expected utility values under different utility functions and priors for decision making. I think the priors actually really become a moot point after you tack on the utility function.
- tel 16y agoI think that choice of model (even nonparametric or empirical distributions) and choice of priors are linked. Both are assumptions based on prior knowledge and analytical approach. Both are overwhelmed by the data in a fertile experiment. Utility functions are a different beast though. They don't have an update procedure and can wildly affect your decision. I'm also convinced they're the best tool we've got so far, so I take it as an illustration that making informed decisions is just hard. Presentation of summary statistics is fine. I prefer presentation of full, untransformed, unpruned data as well when feasible. It's, of course, often not feasible. I also demand justification for why you think those summary statistics are meaningful and under what kinds of situations they would fail to capture the conclusion presented. Not saying that this isn't done in a frequentist setting, but I think it's harder.
- hessenwolf 16y agoYou demand, eh? Honestly, really, truly, honestly, i never bothered with p values as a statistician except for two cases. The first is when performing a test for somebody else to go into a standard article format. The second is when automating reports on complex data. P values are for people who you do not trust to make decisions. Graphs and arrays of summary statistics fro. Several differentodels are for statisticians. Also, i disagree that model choice will be overwhelmed by the data.
- tel 16y agoHah, I should really write these with more care. I'd feel entitled to demand, but more meekly expect that there's a bit more trust and convention in scientific publication. Though that can be taken too far. You're right that model choice can still break your analysis given large amounts of data. I was thinking more in terms of a whole inquiry where large amounts of data will help you to locate a model that extracts the maximal information from your observations. If we're able to keep experimenting forever, we pretty much assume we'll eventually get highly accurate maps of the world. The primary difference was in utility functions where no matter how long you experiment they remain exogenous and static.
- 16y ago