3 ms·
I think a lot of comments here are vastly oversimplifying the problem (including this one). People have lost confidence in academic findings due to a wide varie
by godelski 5y ago
I think a lot of comments here are vastly oversimplifying the problem (including this one). People have lost confidence in academic findings due to a wide variety of reasons. There's many people to blame here. There's academics, journals, news/reporting, politicians, and more. Let's look at a few (far from exhaustive and still pretty shallow analysis).
Academics: Incentives are frequently to publish fast and often. This encourages work with less detail, higher uncertainty/error, and less impactful work. You simply cannot produce as high quality work in 3 months as you can in 1 year (or 3) or research. Often we want PhD students to produce 5 first author papers in top tier journals before graduating. That's 5-6 years to produce just as many top tier papers (realistically 3-4 given your first 2 years are more class based). It is possible to put out a high quality and high impact paper every year, yes. But this is also a very difficult task.
Journals: There's no need to explain this one to anyone who has gone through the review process. Everyone I know that has attempted to publish papers has had several papers where they question what the reviewer actually read because it definitely wasn't their paper. Or where reviewers are vastly under-qualified to review the topic at hand. These are common occurrences. (Note: this more often causes false negatives rather than false positives due to incentive alignments) This creates legitimate complaints and can often be misunderstood from people outside academia.
Reporting: We all have seen this. Things like "eating chocolate is healthy" and other things that are obviously wrong. We've seen universities oversell their research. We see news channels oversell or vastly misunderstand research (despite having easy access to experts and that authors are frequently happy to discuss their research). The public also doesn't understand things like confidence levels. They don't understand that we have a different confidence level in anthropocentric climate change vs how much healthier a glass of wine a day is. This isn't their fault though. We should hold the reporters (academic or news sources) accountable since their are structured to be the science communicators. Instead they are politicizing and adding opinions rather than explaining.
Politics: We all know this one. Everything is politicized. Not just that, but to the point where things are "left" or "right". If the left supports one thing then the right opposes it and vise versa. This is far worse than both sides accepting the issue and coming at it from different angles (which I've seen this happen and both angles also not be in line with the actual research). Covid is a great example: people on the left over-estimate the dangers and people on the right vastly underestimate them (these are not equally biased positions and I'm not claiming they are, just noting that both have errors, not the magnitude of the errors). Climate change is another example, where right denies and left only focuses on pop/hip topics like tree planting or banning straws and not larger issues. (I'll note that in my anecdotal observations it seems that the magnitude of error is frequently smaller on the left).
So we have a system where we aren't encouraged to do high quality research (not meaning it doesn't happen, there are a lot of researchers), research (of any kind) is difficult to get properly peer reviewed, it is poorly explained to laymen, and results become tribal. Why does the public lose confidence in academic findings? You tell me.