3 ms·
I agree in principle but oftentimes the research done by grad-students is part of a project that is spearheaded by a career researcher. I question the notion t
by omginternets 11y ago
I agree in principle but oftentimes the research done by grad-students is part of a project that is spearheaded by a career researcher. I question the notion that MS and PhD work is filling journals with crap.
Instead, I think there are three problems:
1. The inherent problem of inference. When you set p < .05, then 5% of studies will yield false positives. This can be mitigated by a variety of statistical approaches, but it's an irreducible problem on a fundamental level.
2. Certain (sub)disciplines of science aren't very scientific. In particular, they suffer from a high incidence of unfalsifiable claims and hand-wavy definitions. A good example, I think, is the shame vs guilt literature.
3. Certain (sub)disciplines of science are highly politicized. I point my finger at anything involving "identity" (in the sense of "identity politics"), "diversity" (sex, age, ethnicity, sexual orientation, etc) or "bias".
Regarding point 3, take a look at the faculty and students of any university you'd like and take stock of how many Black people are working on racism, women on sexism, LGBT on sexual orientation, etc. As a cognitive neuroscientist, I don't want to claim that people can't study themselves, but in light of the serious reproducibility issues that social psychology is facing, I think this correlation is anything but trivial.
I strongly suspect that the "hard psych vs social psych" dichotomy can be further broken down, and that the sub-disciplines listed in point 3 account for a disproportionate quantity of false-positive results, on accounts of flawed methodology. I suspect that much of this research is actually an attempt to validate political opinion.
I may be wrong about all of this, but at least the core claim testable with a handful of linear regressions.
- analog31 11y agoAll good stuff. Thanks. I'm just saying, don't make publication a requirement for graduation. I'm not a statistician, but as I understand it, a problem with the p value is that the standard calculation hinges on assumptions that are not necessarily met by the data, and if those assumptions were correctly accounted for, p would be higher. If p < 0.05, and in practice 50% of published results are bunk, then regardless of the underlying math, a practical rule of thumb seems to be: Multiply p by 10.
- omginternets 11y agoThe practical rule of thumb for a p-value is that it's nothing more than the rate of expected false-positives. p=.05 means you can expect 5% of false positives. If more than 5% of your studies are failing to replicate, then something else is involved, but it's not necessarily fraudulent. Post-hoc hypotheses are usually the biggest culprit, but it's insidious to the point that researchers often don't realize they're doing it.