2 ms·
The root challenge of all modern frequentist statistics is "does the selection criteria for the sample actually allow me to generalize the results". Experiments
by blamestross 4y ago
The root challenge of all modern frequentist statistics is "does the selection criteria for the sample actually allow me to generalize the results". Experiments and post-hoc analysis are really sensitive to selection biases and researchers (and much worse the media) tend to overgeneralize the results of a study. Its particularly bad in medicine. My personal bugbear is that you can only participate in a study for an autoimmune disorder if you only have the 1 disorder of focus, but real people with autoimmune disorders tend to have more than one (diagnosed or not). So studies on treatment efficacy basically can't actually generalize to the populations they intend to treat because they select against them.
- kkoncevicius 4y ago> The root challenge of all modern frequentist statistics is "does the selection criteria for the sample actually allow me to generalize the results". Why did you single out frequentists here? This is a challenge for all inference, prediction, classification, and any other "forecasting".
- blamestross 4y agoAll those things are either frequentist or baysian statistical models. Baysian models are much harder (often impossible) to use but at least pay attention to intersectionality.
- jmalicki 4y agoFrequentists define this as their challenge (and in very specific ways). Some other subfields doing similar things don't.