5 ms·
Politics and education are certainly factors, but I would argue that they are minor. The bigger issue is the natural incentive structures that academia has crea
by mumblemumble 4y ago
Politics and education are certainly factors, but I would argue that they are minor. The bigger issue is the natural incentive structures that academia has created. If you want to have a career, you get both attention and funding in return for headline-grabbing results. This creates two huge second-order effects: first, everyone's incentivized to look for big impressive new effects. Second, focusing on confirmatory work is a great way to tank your career. And those create a third order effect: people who cut corners thrive, and there's not much to keep that in check because people who cautiously vet their work and others' get stuck in adjunct professor hell until they finally give up and exit the field.
Politics just adds some spin onto that. Education, I just don't think that's even a possible solution. The math involved in some of these fancier statistical techniques can be devilishly hard to get right, and it's often simply asking too much to expect someone to be an expert at both that and molecular biology. You could argue that that means people should look for competent co-authors to cover that side of things. But, see above - current incentive structures don't really favor the researcher who says, "I know! Let's bring on some nerd whose only job is to find reasons why we shouldn't publish!"
- pvaldes 4y agoIf we assign grant points to people that flood the precious journal space with fake articles claiming over-hyped promises, maybe we should give grant points also to the people able to spot the lies on those kind of articles. After all they are serving to the advance of science also (and saving thousands of expensive hours of research to other). If you want to promote it, just create incentives. Otherwise, If the cheater wins in a 95% of the times and the verifier does not have anything to gain (and will damage their own research by the time invested on this) after a while only cheaters remain [1]. At middle term, all research tend towards a sort of expensive shamanism in well illuminated white rooms. [1] (And of course the non cheaters will be actively blocked from this point).
- MPSimmons 4y agoLike a bug bounty for academic publications?
- pvaldes 4y agoYes, something like that
- pvaldes 4y agoMy proposal would be: If there was a clear, proven, intention of cheating and not just an error (we all commit stupid errors sometimes), the letter or communication of the people disclosing it will receive the same citation rank that the fake article had accumulated until that date. Good for the journals, good for the research.
- WalterBright 4y ago> Politics and education are certainly factors, but I would argue that they are minor Are they? If your proposed project does not align with current politics, no funding for you.
- mike_hearn 4y agoFrom reading papers of various different types in the past few years it really depends a lot on the exact field in question. Nutrition? Would be surprising if there's much political distortion there but there's lots of really bad methodologies, partly because it's hard to get good data (people are bad at reliably writing down what they eat). Social bot/misinformation research? Basically just politics masquerading as science. Methodologies are invariably bogus and conclusions are clearly determined before any 'research' is done. Epidemiology? A mix of bad incentives and ideological distortion. The focus on policy advice isn't really scientific, and their policies somehow always end up requiring massive state intervention in people's private lives. This isn't something that falls naturally out of the science (which is unreliable anyway) but more the fact that they don't even consider any other possibilities. The above three are pretty bad fields. There are plenty of better ones. Computer science is mostly OK. AI and cryptography are slightly dubious. Stuff like databases or computer graphics are fine. So IMO it doesn't make sense to talk about research or academia in general, or it does but only when discussing the universal incentive problems. Some fields just seem naturally resistant to political game playing because they don't have much/any impact on social policy to begin with. Obviously I mean politics as in democratic politics here, not the more mundane everyday departmental politics.
- cryptonector 4y agoAcademia is pretty corrupt or corruptible (well, it's humans after all). If CS research isn't corrupted at the moment, it will be if it being corrupted becomes useful.
- mumblemumble 4y agoI don't know about hard CS, and I don't know about that word "corrupt" because it implies malice and I think that Hanlon's Razor is probably a useful heuristic here. But, in machine learning, I've see plenty of the kind of thing I'm talking about. Papers often introduce new algorithms or techniques whose improvement over the competition can plausibly be attributed to excessively tuned hyperparameters that overfit the data on one or more benchmarks. It's really not all that far removed from p-hacking in my opinion.