3 ms·
> Would you care to explain why the suggested actions would be "somewhere between ridiculous and useless?" Not OP, but the proposed "solutions" not only add mo
by jpeloquin 6y ago
> Would you care to explain why the suggested actions would be "somewhere between ridiculous and useless?"
Not OP, but the proposed "solutions" not only add more work items to the ever-growing checklist (as mentioned by meow1032), but to be useful everyone must spend even more time checking everyone else's work items:
Solution 1, requiring data sharing and preregistration, greatly increases the work of peer review, perhaps by an order of magnitude. Someone needs to check that the data produces the published results and that the final analysis plan matched the preregistration. That is hard, time-consuming volunteer work, with no reward incentive. Current peer review currently trusts the authors did what they said they did, correctly, and it still takes 4-12 hours to review an article. Most reviewers cut corners. If no one does the work to check the open data or preregistrations, "open science" will be merely performative, with no quality improvement.
Solution 2, changing hiring policies to "look beyond publication and citation numbers", is pretty much what hiring committees already do. But with ~ 200 applicants per job opening the depth of examination per applicant is somewhat limited. As in solution 1, lack of time for deep checks is a problem. Applicants who are well-networked with good pre-existing reputations (i.e, who are plugged into the web of trust) get hired; everyone else doesn't. From a perspective of research quality, this may be a good thing.
Solution 3, funders fund boring / rigorous research, could improve matters in theory. But with only enough money to fund ~ 1 in 10 proposals, projected impact will always be an overwhelming concern. Proposals will include a "research integrity" section or similar and nothing substantive will change.
Solution 4, scientists "vote with their feet" (stop participating in the dysfunctional parts of the system), is a call for people to come up with their own solutions or support other proposed solutions, not a solution in its own right. Ironically, it is perhaps the most useful because it pushes back on the idea that poor quality science is inevitable under the current structure. "Perverse incentives" must not become a generally accepted excuse to sacrifice scientific integrity for the benefit of one's own career. Science is meant to discover new information. Without a culture of integrity, that information will always be suspect, regardless of what top-down interventions are attempted.
An effective intervention must reduce workload or at least break even, not increase it. Or increase the resources available. Otherwise people will be forced (actually forced, not just incentivized; there are only so many work hours each day) to cut even more corners elsewhere to make up for lost time.