4 ms·

> "Going by these numbers, roughly one in 1,000 papers gets retracted [..] that something more like one in 50 papers has results which are unreliable because of

• ska 4y ago
  > The only "value" they provide is.. As far as I can see this is mostly an incentives problem. Bureaucratic control of academic hiring has ended up emphasizing the short term measurable (e.g. #of pages published, so-called impact factors, etc.) over the long term, with pretty predictable results. Medical research in particular is fraught with another set of problems; the default clinical pathway gives a weak at best grounding in science, and even the MD/PhD programs have been gamed to some degree. There are definite counterexamples (lots!) but there are also a lot of clinicians with incentive to produce research but little skill in it and even less time available...
  • opportune 4y ago
    I think zooming further out, the incentive for academia is to churn out degrees + get grants. Getting a PhD is supposed to require doing something that nobody has done before. Most people getting PhDs/in academia are not actually good enough to do this (it’s really hard! There’s not a ton of low hanging fruit, and you’re “competing” with many others), which is why we end up with tons of garbage papers nobody will ever care about. Medical research may be performed by MDs but the incentives are still basically the same: papers are resume builders. It looks better when applying to a fellowship/job to have a nice publication history. Obviously the best case is to have worked on some really groundbreaking stuff - still really hard - but the next best case is to have a ton of meh publications, since that beats having a few publications, or no publications. In medical programs the grant thing is a lot bigger too because there is, rightfully, tons of money to throw to that area. You need grants as an academic to progress. You won’t keep getting grants if you take grants and then don’t publish anything, so even if you have nothing good come out of it, you need to publish something. That incentivizes fraud in the worst case and noise in the best. The better-best case would be if academia were more open to accepting null results
    • xpe 4y ago
      I wish the extent to which a paper is well written (honest, clear, appropriate for the audience) was enough. Sure, I want scientist to "have a nose" for finding interesting results, but I don't like the framing that a solid piece of work is any less because it happened to not demonstrate a useful result. I'll add that I would like to see a LOT more work that synthesizes and analyzes other work; i.e. literature reviews and meta-analyses.
      • opportune 4y ago
        That’s what most of those papers I am calling garbage are: they are not necessarily wrong but they are not interesting. A meta analysis is cheap and easy to do for even an undergrad, and doesn’t require any special insight or foresight. I think the bigger problem is we have too many people trying to chase after the highest tier of academic achievement relative to what that tier “should” be or was in the past. It’s benchmarked on novelty, but most people doing research are never going to produce any worthwhile novel results - in some cases it’s just bad luck but in most I think it is just lack of aptitude. Research is not supposed to just be a resume checkmark, and a PhD isn’t just supposed to be some structured degree program where you can get “on rails” and churn out papers to get a degree proving you have above-average intelligence. But that’s what it is, and it generates tons of noise, while cheapening the value of a PhD. If I were emperor of the world I’d split research into two tiers where one is more focused on basic science: investigative studies, verifying results, writing papers with solid structure, applying stats. This would be what most people get, and could soak up demand for people getting PhDs with no intention of becoming academics (ie to immigrate, to qualify for some kind of job). And then a second tier of research for the wickedly skilled researchers who are producing novel results and really moving the field forward. Right now that second group, in most science disciplines, is who progresses in academia anyway. But tons of people are “doing research” and getting advanced degrees based on research in a way that is more like a “basic scientist training program” rather than “moving forward a field of study.”
        • xpe 4y ago
          > A meta analysis is cheap and easy to do for even an undergrad, and doesn’t require any special insight or foresight. This is too harsh, too narrow, and too judgmental. This only mentions the skill required, but neglects to account for the benefits to the body of knowledge (literature). I reject the idea that scientific papers impact should be judged on a curve according to how difficult the work was: 1. This is premised on a false notion of scarcity. 2. Breakthroughs can seem obvious to a lucky person in the right place and time. 3. Others may ascribe "genius-level" status to such lucky people, but this elevation is often unwarranted and not useful. 4. The time delay before impact and benefit is uncertain. Often work takes time to be synthesized, appreciated, applied, and so on. 5. What is {important, relevant, useful} can change over time. Having only a narrow definition misses the point.
• nradov 4y ago
  Some of that 80% of papers are also valuable to politicians and bureaucrats pushing biased narratives on the public. No matter what position they want to take they can cherry pick some low-quality research to justify it with a veneer of "science".
• throwawaysleep 4y ago
  How are we defining “useless” here? Papers that are wrong and fraudulent? Or is it broader and any paper we currently can’t do anything with or yields a negative result?
  • johnnymorgan 4y ago
    Did anyone use the paper for anything? If not, it's useless, ie it lacks any utility