4 ms·
Yes, there is a time lag problem. However, instant distribution has been around for a long time (in the case of arXiv.org, since 1991). It's widely accepted in
by simonster 14y ago
Yes, there is a time lag problem. However, instant distribution has been around for a long time (in the case of arXiv.org, since 1991). It's widely accepted in the physics community, but it hasn't gained much traction in most other scientific disciplines. I think there are two reasons for this: the chicken and egg problem, and the peer review problem.
The chicken and egg problem is that no one in these disciplines publishes unreviewed manuscripts because no one reads them. The corollary here is that if you do something interesting and someone happens to read it, take your idea, and publish first, as far as credit goes, you're fucked. This happens with any form of public presentation of ideas, not all that often but often enough that every scientist knows someone who it has happened to. If you just sank a year of your life into a project, you want to make damn sure you're going to get credit for it. At present, instant distribution is too risky. If the profile of instant distribution can rise to the point where a manuscript will be sufficiently widely read to be acknowledged as the source of an idea, scientists in less competitive areas may be more open to it.
The bigger issue is, I think, that scientists actually appreciate peer review. Peer review ensures both quality and fairness in research. If I read a paper in a high-impact journal, I generally believe can trust the results regardless of who wrote it. By contrast, any reputation-based metrics will be strongly colored by the reputation of the lab from which the paper originates. (I have a hunch that this is already true for citation metrics.) Replacing peer review with reputation-based metrics may mean research gets out there faster, but it may also mean that a lot of valuable research gets ignored. This still sucks, and it may suck more. Turning a paper into a startup that may succeed or may fail depending on how well a scientist can market his or her findings would absolutely suck ass. IMHO, scientific funding is already too concentrated in the hands of established labs, and these labs are often too large to make effective use of their personnel. Reputation-based metrics would only contribute to this problem. They would also lead to confusion in the popular press, which is already somewhat incapable of triaging important and unimportant scientific results. This is a much bigger deal in biomedical science than in theoretical physics, because the former has direct bearing on individuals' lives.
On top of this, citation metrics are simply not peer review. In his previous article, Richard Price pointed out that researchers need to spend a lot of time performing peer review. This is absolutely the way it should be. Researchers should spend hours poring over new papers, suggest ways of improving them to the authors, and ultimately ensure that whatever makes it into press is as high quality as possible. IMHO, the easiest way to get quality research out faster is to encourage journals to set shorter peer review deadlines and encourage researchers to meet them, not to throw away the entire system.
OTOH, I think open sharing of data sets among researchers will massively enhance scientific progress, and has a reasonable chance of happening because the push is coming from funding agencies, not startups. As a scientist, the idea of being able to ask my own questions with other people's data gets me far more excited than being able to read their papers before release.
- RichardPrice 14y agoI totally agree with you about data-sharing. I wanted to spend more time on that in the article, but didn't because I didn't want to make the article longer. I think the ability to share and ask questions about data really has enormous potential to drive science forward. The fact that enormous amounts of scientific data remains private to the lab, and not shared, is really a big loss to science. It's going to be very exciting as that data starts getting shared more. The key to making that happen is disrupting the credit system. Right now scientists aren't incentivized to curate and share their data, so they don't put in the work to do it. You can't put data-sets on your resume, much like you can't put blog posts, or anything that is not a paper. As soon as scientists start getting credit for sharing data-sets, I think we'll start to see it happen. Similar points apply, as you mention, to instant distribution. Instant distribution will happen more as scientists start getting credit for scientific ideas that they distribute instantly. You are already seeing some disruption to the credit system. In the last 5-10 years, since citation counts have been made publicly available by Google Scholar, citation counts have started to play a much larger role in resource allocation decisions, e.g. decisions by hiring committees and grant committees. I did my PhD at Oxford in philosophy from 2001-2007, and remained involved with some of the hiring decisions at the Oxford philosophy department until 2011, and it's been very interesting to watch the increased influence, over those years, of citation counts in hiring decisions. Citation counts aren't perfect, but they are another signal. Hiring committees, I have experienced, are desperate for more signals that they can take into account when comparing candidates. Comparing candidates is a tough job. As with any signal, to wield it properly, you need to know its pros and cons. Fundamentally what the community is looking for here is a variety of signals that show how much a highly respected chunk of the scientific community has interacted with a piece of your content, and found it useful. To get data-sets, and other media, to attract scientific credit, we need to develop metrics that demonstrate the traction that those pieces of media are getting in highly respected parts of the scientific community. I think those metrics will get developed, and that new metrics will play an enormous role in allowing different kinds of media to be shared, and everything to be shared faster.