8 ms·

Why publish science in peer-reviewed journals?

55 points by ig1 15y ago

nolite 15y ago
It gets you funding
- danieldk 15y ago
  This. I guess at least a fraction of scientists are open towards a change in reviewing. However, during funding applications and interviews, the number of publications in journals is very important. Hence, if you want to make any chance in science, you submit to journals.
- Estragon 15y ago
  Oh, yeah. I want to publish in PLoSOne, but my more established collaborators (the ones who have arranged the NIH grants) absolutely need to publish in Nature, etc.
eru 15y ago
Using this system, you can still gather a bunch of web-published articles, print them out together, and call the results "Nature". Just in case you miss the current system.
bendmorris 15y ago
While I'm in favor of opening up journals and possibly putting out some nonprofit alternatives, and while I recognize that the peer review process is definitely not without its problems, be careful not to discount it completely. I cringe when I hear phrases like "value to a community" and "collective opinion" used to refer to scientific articles. The average reader is not qualified to judge the merits of an article, hence the peer review process. Some great examples: the age of the earth, evolution, global climate change; three areas where public opinion is divided but experts overwhelmingly agree. The value of science is based on its accuracy (which is best determined by experts) not its appeal to the masses.
- hotpockets 15y ago
  I was assuming that the article was suggesting a system that is similar to reddit, but with important differences. For instance, requiring real names and public voting records. Or, you could reduce a person's vote importance if they are voting on an article outside their area of expertise. There are probably other things you could do to.
  - radarsat1 15y ago
    > Or, you could reduce a person's vote importance if they are voting on an article outside their area of expertise. I see an immediate problem there. There are magnitudes more people _outside_ any given area of expertise than inside. Even small votes add up.
    - juiceandjuice 15y ago
      Ladies and gentleman, I present to you the logarithmic scale.
  - rlpb 15y ago
    Apart from the lack of anonymity, I think that the ideals of such a system are very similar to the ideas of any social voting system (such as Stack Overflow, Slashdot, reddit, HN). The goal is to simply have a rating system that works, right? The trouble is that we still haven't managed to do this well in the technology world. And some of us believe that we need to go back to some level of curation. I'd love to see a system as described that actually works. But until we (the tech world) manage to figure out a system that works, I'm not convinced that scientific publishing will fare any differently.
    - Joakal 15y ago
      The technology world already has it; We have Wikipedia (Edit history), Advogato (Paths), StackOverflow (Rep), etc that encourage personal profiles. You are more valued as a loyal member than one that just started. Unlike Reddit, Slashdot and HN that do not encourage personal profiles. Do you even remember my name? What about other HNers besides pg? Why do you read their posts anyway? They work because it's a form of almost pure democracy being applied and have the same problems of it eg tyranny of the majority. What the scientific model can do is a multitude of above; institution API confirming credentials (Referral), history (paths, did this person do other reviews?), etc. Cheap and still effective peer-review. I'm not sure which kind of government type applies to my suggestion above, but there are many forms of government they can apply. More: https://secure.wikimedia.org/wikipedia/en/wiki/List_of_forms_of_government https://secure.wikimedia.org/wikipedia/en/wiki/List_of_forms...
cmonsen 15y ago
Not to mention that publication bias leads to incorrect results. See http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/ http://www.theatlantic.com/magazine/archive/2010/11/lies-dam... or the PLoS paper: http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0050201 http://www.plosmedicine.org/article/info:doi/10.1371/journal.... I am most hopeful for Google Scholar or Mendeley (mendeley.org) to open this up, but who knows?
flocial 15y ago
The article doesn't touch on the fact that all this scholarship, usually funded in some way with taxes, is locked up behind pay walls unless you want to pay $10-$40 per article (that's probably a bigger obstacle). However, ArXiv works because at least with harder sciences you have proofs and empirical results that are more clearly defined. It would be harder to judge papers in economics, humanities, etc. (there's already a lot of variance in quality of scholarship even in the current published model, I don't see how it would flourish under an open model aside from becoming more like the blogs we read here - no offense). There have been many experiments where people have passed on jargon-laden papers more fiction than research off to "respected" journals. If you can solve the quality/trust problem and make it acceptable for governments and other funding bodies, then we might move forward. The thing about the current model is that they distribute pdfs of all things when I think the raw text, bibliography files, and even data should be distributed with the article and available at no cost.
- stcredzero 15y ago
  I wouldn't mind a two tiered system, with a curated membership and paywalls, and the riffraff/unwashed masses in a lower tier. Exclude the latter from making comments, but give out the knowledge for free.
- starwed 15y ago
  The article doesn't touch on the fact that all this scholarship, usually funded in some way with taxes, is locked up behind pay walls unless you want to pay $10-$40 per article (that's probably a bigger obstacle). However, ArXiv works because at least with harder sciences you have proofs and empirical results that are more clearly defined. Most of the stuff on arxiv is also published in peer reviewed journals. That's why they are called preprints.
  - flocial 15y ago
    But not necessarily. There are some landmark papers like Grigori Perelman's solution of the Poincaré Conjecture that only exist on ArXiv.
- 15y ago
vacri 15y ago
I have one single solitary paper that has been published in a peer reviewed journal, so my experience is present but limited. What I gained out of the peer review process was an article that had grey areas filled in and statistical methods applied better. The quality of the reviewed article was much better than the quality of the submitted article. The reviewers were reading the article with an even-handed critical eye; they weren't just looking for support for their pet opinions. There's also an aspect of criticism - criticism is expected from reviewers, but random dude A on the internet is frequently going to get something along the lines of "who are you to criticise my methods", or any request for more info declined. "Please provide more info" will work if you need to get a paper published, but won't work if you have already published your paper on scribd - where's the motivation to do rework (beyond honour of course, but scientists are human and that does not always apply) And then there's the point that popularity doesn't equal 'correct'. Here on HN I've seen time and time again that a throwaway popular comment gets upvotes, and a well-reasoned but unpopular comment gets downvotes. Peer-reviewed articles should be about scientific rigor, not visceral feeling. Consensus of the masses is orthogonal to the needs of scientific rigor. The main article describes very, very little in the way of quality assurance.
- anghyflawn 15y ago
  I agree. I've had two articles reviewed by now (not in a science field, but still), and only one reviewer was relatively unhelpful (they thought the paper was too dense and insufficiently well presented to do a detailed review); all the others, even whey they could not recommend acceptance, went out of their way to suggest various ways of making the paper(s) better. People are in general much more likely to bother giving negative feedback than to give positive feedback, much less defend other people's efforts. I'm afraid such a system would be no different: would a ton of "likes" offset the negativity of a couple of arseholes wreaking havoc in the comments to make it worth it?
- Estragon 15y ago
  Yes, the popularity of useless, emotive comments here really bugs me. There is an inverse correlation between the points I get for a comment and the amount of time and expertise I put into it. In the scientific case, though, "popularity" would be measured by the extent of citation. (Really, it already is.) That is a considerably higher bar than "I think I'll click on that arrow." For a paper to be cited, it has to make enough of an impact that people remember it and its ideas shape their thinking.
jgamman 15y ago
and no mention of mendely? http://www.mendeley.com/ http://www.mendeley.com/ the pain point is that people are focused on the publication - you need to focus on the people. i don't care what you think about a paper, but i want to know what Feynman has published and who he keeps an eye on. professional scientists know who to keep an eye on - the rest is noise. science is the oldest social network there is...
thret 15y ago
The issues he discusses seem to be addressed by phygg, essentially a combination of arvix and digg for physics: http://www.phygg.com/phygg/ http://www.phygg.com/phygg/ I see no reason why this shouldn't work for arvix as a whole.
- chalst 15y ago
  Most recent "paper" on Phygg, fram July 2nd, begins Red Bull Hats: Wholesale from factory, limited time! Get MLB Hats, NFL Hats, New Era Hats for your favorite teams. It's gathered net upvotes. Most papers there have been automatically cross-posted from Arxiv. I don't think we can take Phygg to be a proven model. There's a wider issue of how new gatekeepers can be established for such a new model.
wickedchicken 15y ago
I see a pretty significant issue with "one-click voting" for a scientific paper. Reading papers is often a tough exercise and one that involves rereading, thinking, and rereading again. How many times have you read a dense webpage, reread it over the course of a week, then clicked the upvote on HN? Never. You glance at the webpage, think "that's neat" and click away. This isn't to say that the journal-of-reviewers model is perfect, but I am wary of casually tossing it away in favor of "upvotes." Upvotes have massive semantic ambiguity: does it mean the article is accurate? Does it mean the article is funny? Is it maybe wrong, but important to think about? Does it merely match the reader's interest? How do you express something such as "we believe your underlying methodology is correct but aren't certain your analyses are rigorous enough to prevent X." A downvote?
- copper 15y ago
  A better model would be something like what the Real-World Haskell book used when it was under development: allow comments (anonymous or otherwise) on each paragraph, and refine the content based on those comments. Of course, this does have the twin problems that it doesn't keep results private, and makes credit assignment a real pain :) Even with all the problems it has, I actually prefer peer-review right now. Some of the stuff that gets self-published on arXiv is of sketchy quality, and there's just no quick way of figuring that out by just reading the abstract if a 'known' person hasn't co-authored it.
  - _delirium 15y ago
    In physics at least, a substantial proportion of peer-review goes via arXiv first anyway, so it's not really an either-or. In many cases, the real peer-review happens when people see a new paper in their area on arXiv, they send in comments or challenges, etc. to the authors, and in the best case, by the time a journal article is submitted for formal peer review, many of the relevant peers have already reviewed and helped improve it. Some mathematicians have also gotten pretty active in using blogs and wikis for peer review and even actively doing research. I don't expect journals to go away, but I wouldn't be too surprised if there are some areas of the sciences where they become more of a formal archival service: where papers go to be filed on shelves once the real action/review/dissemination has already happened.
a3_nm 15y ago
My experience (in computer science, ymmv) is that a vast majority of researchers would agree that, with the Internet, the profits made by journals are excessive relative to what they do (ie. what they actually do, not what they outsource to other researchers). The reason why there are still there isn't because we're convinced that we need them (as the article implied), but because of inertia. This is the criterion by which most researchers judge each other and themselves, and it survives because few people have the guts to refuse it: everyone knows it's evil, but it's hard to walk away. People are lured by prestige, and by the tangible consequences of prestige (like funding). We do not need a killer app to get rid of the journals, we need a good number of motivated and prominent researchers to opt out and bootstrap something else.
BasDirks 15y ago
"peer review is costly (in terms of time and money)" This is an objection why? Of course science costs time and money. "However, journals do perform a service of sorts: they filter papers, ostensibly based on interest to a community, and also ostensibly ensure that the details of an analysis are correct and clear enough to be replicated (though in practice, I doubt these filters work as intended)." Nobody cares about your doubts. Back it up.
- anghyflawn 15y ago
  >Of course science costs time and money. One problem is that peer review is essentially free for the journals, and people have to take time for it out of the other things they have to do (like teaching, or scrambling to get Yet Another Publication before the tenure review comes up, or doing the reporting to 550 external agencies). People are expected to uphold the highest standard of rigour essentially in their free time. In that sense, the current peer review leads to waste, by decreasing the efficiency of the investments originally made into other things.
NY_Entrepreneur 15y ago
The author's ideas are wacko and won't work. I've published two technical papers in peer-reviewed journals where I was the only author, published several more papers in peer-reviewed journals where I had co-authors, and wrote a Ph.D. dissertation that was reviewed as "an original contribution to knowledge worthy of publication". The papers I published were reviewed essentially on the usual criteria of "new, correct, and significant". Here is one place the author gets way off track: For a technical research paper, e.g., new, correct, and significant, reading it is difficult, and judging if it is new, correct, and significant is more difficult. In an experimental science, judging if the work is reproducible adds more difficulty. It's TOUGH work. The work is so tough that for some huge majority of papers that are published, nearly the only careful reading at least for some years will be during the review process. Maybe later the paper will get referenced, carefully read by many people, etc. So, the idea of the article of this thread that commonly for papers that just appear on-line there can be 'votes' from many competent readings is just uninformed, misinformed, ignorant, just plain wrong, brain-dead nonsense. Here's more: Now, when an author A sends a paper to journal X, there is a good chance that the paper is actually in or close to the specialty S of journal X. So, the reviewers of journal X have a good chance of being among the best people to review the paper. If journal X does publish the paper, then author A, the paper, and people interested in specialty S get a lot of help because journal X has said that the paper is new, correct, and significant for specialty S. STILL, even with all this help for specialty S, the paper will likely be carefully and competently read at least for some years essentially only by the reviewers of journal X. Since it is so difficult even with the help of journal X in specialty S to get such papers read, for a paper just posted on-line there is no hope at all. Maybe in some experimental sciences reading a paper is relatively easy; however, judging if the experimental work is reproducible promises to be difficult. Maybe not in experimental science but in essentially all of the rest, there is a fundamental difficulty about research that is new, correct, and significant: Almost necessarily nearly no one will understand the paper easily or understand the paper at all. Else, given basic academic competitiveness, the results in the paper would long since have been old instead of new. So, the research and review processes have to work always right at the edge of what can currently be understood at all. For meeting this fundamental difficulty, the present journal system helps, and just putting papers on the Internet is hopeless.
rnadna 15y ago
Yesterday I got an email from a journal asking me to submit a paper. They needed it within 2 weeks. And it would be published 2 weeks thereafter. After I spewed coffee through my nose, laughing, I clicked the "spam" button. Publication in real journals is slow; reviewers are asked to send comments within a month, but they often take more than that. Publication in real journals is difficult; the best-ranked journals reject 90 percent of submissions. Publication in real journals is flexible; complicated decisions about how to transform unpublishable into publishable are made in each step along a long, often meandering and complicated, path. The simple answer to the question posed ("why publish in peer-reviewed journals") is that, for many fields, not doing so ends a career. But if the question is, as it seems by comments here and in many conversations I've had with students, "what value do peer-reviewed journals have" the answer is that, for many fields, doing so improves the science for the individual paper, and for the larger corpus.

55 points by ig1 15y ago

nolite 15y ago

It gets you funding

danieldk 15y ago
This. I guess at least a fraction of scientists are open towards a change in reviewing. However, during funding applications and interviews, the number of publications in journals is very important. Hence, if you want to make any chance in science, you submit to journals.
Estragon 15y ago
Oh, yeah. I want to publish in PLoSOne, but my more established collaborators (the ones who have arranged the NIH grants) absolutely need to publish in Nature, etc.

eru 15y ago

Using this system, you can still gather a bunch of web-published articles, print them out together, and call the results "Nature". Just in case you miss the current system.

bendmorris 15y ago

While I'm in favor of opening up journals and possibly putting out some nonprofit alternatives, and while I recognize that the peer review process is definitely not without its problems, be careful not to discount it completely. I cringe when I hear phrases like "value to a community" and "collective opinion" used to refer to scientific articles. The average reader is not qualified to judge the merits of an article, hence the peer review process. Some great examples: the age of the earth, evolution, global climate change; three areas where public opinion is divided but experts overwhelmingly agree. The value of science is based on its accuracy (which is best determined by experts) not its appeal to the masses.

hotpockets 15y ago
I was assuming that the article was suggesting a system that is similar to reddit, but with important differences. For instance, requiring real names and public voting records. Or, you could reduce a person's vote importance if they are voting on an article outside their area of expertise. There are probably other things you could do to.
- radarsat1 15y ago
  > Or, you could reduce a person's vote importance if they are voting on an article outside their area of expertise. I see an immediate problem there. There are magnitudes more people _outside_ any given area of expertise than inside. Even small votes add up.
  - juiceandjuice 15y ago
    Ladies and gentleman, I present to you the logarithmic scale.
- rlpb 15y ago
  Apart from the lack of anonymity, I think that the ideals of such a system are very similar to the ideas of any social voting system (such as Stack Overflow, Slashdot, reddit, HN). The goal is to simply have a rating system that works, right? The trouble is that we still haven't managed to do this well in the technology world. And some of us believe that we need to go back to some level of curation. I'd love to see a system as described that actually works. But until we (the tech world) manage to figure out a system that works, I'm not convinced that scientific publishing will fare any differently.
  - Joakal 15y ago
    The technology world already has it; We have Wikipedia (Edit history), Advogato (Paths), StackOverflow (Rep), etc that encourage personal profiles. You are more valued as a loyal member than one that just started. Unlike Reddit, Slashdot and HN that do not encourage personal profiles. Do you even remember my name? What about other HNers besides pg? Why do you read their posts anyway? They work because it's a form of almost pure democracy being applied and have the same problems of it eg tyranny of the majority. What the scientific model can do is a multitude of above; institution API confirming credentials (Referral), history (paths, did this person do other reviews?), etc. Cheap and still effective peer-review. I'm not sure which kind of government type applies to my suggestion above, but there are many forms of government they can apply. More: https://secure.wikimedia.org/wikipedia/en/wiki/List_of_forms_of_government https://secure.wikimedia.org/wikipedia/en/wiki/List_of_forms...

cmonsen 15y ago

Not to mention that publication bias leads to incorrect results. See http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/ http://www.theatlantic.com/magazine/archive/2010/11/lies-dam... or the PLoS paper: http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0050201 http://www.plosmedicine.org/article/info:doi/10.1371/journal.... I am most hopeful for Google Scholar or Mendeley (mendeley.org) to open this up, but who knows?

flocial 15y ago

The article doesn't touch on the fact that all this scholarship, usually funded in some way with taxes, is locked up behind pay walls unless you want to pay $10-$40 per article (that's probably a bigger obstacle). However, ArXiv works because at least with harder sciences you have proofs and empirical results that are more clearly defined. It would be harder to judge papers in economics, humanities, etc. (there's already a lot of variance in quality of scholarship even in the current published model, I don't see how it would flourish under an open model aside from becoming more like the blogs we read here - no offense). There have been many experiments where people have passed on jargon-laden papers more fiction than research off to "respected" journals. If you can solve the quality/trust problem and make it acceptable for governments and other funding bodies, then we might move forward. The thing about the current model is that they distribute pdfs of all things when I think the raw text, bibliography files, and even data should be distributed with the article and available at no cost.

stcredzero 15y ago
I wouldn't mind a two tiered system, with a curated membership and paywalls, and the riffraff/unwashed masses in a lower tier. Exclude the latter from making comments, but give out the knowledge for free.
starwed 15y ago
The article doesn't touch on the fact that all this scholarship, usually funded in some way with taxes, is locked up behind pay walls unless you want to pay $10-$40 per article (that's probably a bigger obstacle). However, ArXiv works because at least with harder sciences you have proofs and empirical results that are more clearly defined. Most of the stuff on arxiv is also published in peer reviewed journals. That's why they are called preprints.
- flocial 15y ago
  But not necessarily. There are some landmark papers like Grigori Perelman's solution of the Poincaré Conjecture that only exist on ArXiv.
15y ago

vacri 15y ago

I have one single solitary paper that has been published in a peer reviewed journal, so my experience is present but limited. What I gained out of the peer review process was an article that had grey areas filled in and statistical methods applied better. The quality of the reviewed article was much better than the quality of the submitted article. The reviewers were reading the article with an even-handed critical eye; they weren't just looking for support for their pet opinions. There's also an aspect of criticism - criticism is expected from reviewers, but random dude A on the internet is frequently going to get something along the lines of "who are you to criticise my methods", or any request for more info declined. "Please provide more info" will work if you need to get a paper published, but won't work if you have already published your paper on scribd - where's the motivation to do rework (beyond honour of course, but scientists are human and that does not always apply) And then there's the point that popularity doesn't equal 'correct'. Here on HN I've seen time and time again that a throwaway popular comment gets upvotes, and a well-reasoned but unpopular comment gets downvotes. Peer-reviewed articles should be about scientific rigor, not visceral feeling. Consensus of the masses is orthogonal to the needs of scientific rigor. The main article describes very, very little in the way of quality assurance.

anghyflawn 15y ago
I agree. I've had two articles reviewed by now (not in a science field, but still), and only one reviewer was relatively unhelpful (they thought the paper was too dense and insufficiently well presented to do a detailed review); all the others, even whey they could not recommend acceptance, went out of their way to suggest various ways of making the paper(s) better. People are in general much more likely to bother giving negative feedback than to give positive feedback, much less defend other people's efforts. I'm afraid such a system would be no different: would a ton of "likes" offset the negativity of a couple of arseholes wreaking havoc in the comments to make it worth it?
Estragon 15y ago
Yes, the popularity of useless, emotive comments here really bugs me. There is an inverse correlation between the points I get for a comment and the amount of time and expertise I put into it. In the scientific case, though, "popularity" would be measured by the extent of citation. (Really, it already is.) That is a considerably higher bar than "I think I'll click on that arrow." For a paper to be cited, it has to make enough of an impact that people remember it and its ideas shape their thinking.

jgamman 15y ago

and no mention of mendely? http://www.mendeley.com/ http://www.mendeley.com/ the pain point is that people are focused on the publication - you need to focus on the people. i don't care what you think about a paper, but i want to know what Feynman has published and who he keeps an eye on. professional scientists know who to keep an eye on - the rest is noise. science is the oldest social network there is...

thret 15y ago

The issues he discusses seem to be addressed by phygg, essentially a combination of arvix and digg for physics: http://www.phygg.com/phygg/ http://www.phygg.com/phygg/ I see no reason why this shouldn't work for arvix as a whole.

chalst 15y ago
Most recent "paper" on Phygg, fram July 2nd, begins Red Bull Hats: Wholesale from factory, limited time! Get MLB Hats, NFL Hats, New Era Hats for your favorite teams. It's gathered net upvotes. Most papers there have been automatically cross-posted from Arxiv. I don't think we can take Phygg to be a proven model. There's a wider issue of how new gatekeepers can be established for such a new model.

wickedchicken 15y ago

I see a pretty significant issue with "one-click voting" for a scientific paper. Reading papers is often a tough exercise and one that involves rereading, thinking, and rereading again. How many times have you read a dense webpage, reread it over the course of a week, then clicked the upvote on HN? Never. You glance at the webpage, think "that's neat" and click away. This isn't to say that the journal-of-reviewers model is perfect, but I am wary of casually tossing it away in favor of "upvotes." Upvotes have massive semantic ambiguity: does it mean the article is accurate? Does it mean the article is funny? Is it maybe wrong, but important to think about? Does it merely match the reader's interest? How do you express something such as "we believe your underlying methodology is correct but aren't certain your analyses are rigorous enough to prevent X." A downvote?

copper 15y ago
A better model would be something like what the Real-World Haskell book used when it was under development: allow comments (anonymous or otherwise) on each paragraph, and refine the content based on those comments. Of course, this does have the twin problems that it doesn't keep results private, and makes credit assignment a real pain :) Even with all the problems it has, I actually prefer peer-review right now. Some of the stuff that gets self-published on arXiv is of sketchy quality, and there's just no quick way of figuring that out by just reading the abstract if a 'known' person hasn't co-authored it.
- _delirium 15y ago
  In physics at least, a substantial proportion of peer-review goes via arXiv first anyway, so it's not really an either-or. In many cases, the real peer-review happens when people see a new paper in their area on arXiv, they send in comments or challenges, etc. to the authors, and in the best case, by the time a journal article is submitted for formal peer review, many of the relevant peers have already reviewed and helped improve it. Some mathematicians have also gotten pretty active in using blogs and wikis for peer review and even actively doing research. I don't expect journals to go away, but I wouldn't be too surprised if there are some areas of the sciences where they become more of a formal archival service: where papers go to be filed on shelves once the real action/review/dissemination has already happened.

a3_nm 15y ago

My experience (in computer science, ymmv) is that a vast majority of researchers would agree that, with the Internet, the profits made by journals are excessive relative to what they do (ie. what they actually do, not what they outsource to other researchers). The reason why there are still there isn't because we're convinced that we need them (as the article implied), but because of inertia. This is the criterion by which most researchers judge each other and themselves, and it survives because few people have the guts to refuse it: everyone knows it's evil, but it's hard to walk away. People are lured by prestige, and by the tangible consequences of prestige (like funding). We do not need a killer app to get rid of the journals, we need a good number of motivated and prominent researchers to opt out and bootstrap something else.

BasDirks 15y ago

"peer review is costly (in terms of time and money)" This is an objection why? Of course science costs time and money. "However, journals do perform a service of sorts: they filter papers, ostensibly based on interest to a community, and also ostensibly ensure that the details of an analysis are correct and clear enough to be replicated (though in practice, I doubt these filters work as intended)." Nobody cares about your doubts. Back it up.

anghyflawn 15y ago
>Of course science costs time and money. One problem is that peer review is essentially free for the journals, and people have to take time for it out of the other things they have to do (like teaching, or scrambling to get Yet Another Publication before the tenure review comes up, or doing the reporting to 550 external agencies). People are expected to uphold the highest standard of rigour essentially in their free time. In that sense, the current peer review leads to waste, by decreasing the efficiency of the investments originally made into other things.

NY_Entrepreneur 15y ago

The author's ideas are wacko and won't work. I've published two technical papers in peer-reviewed journals where I was the only author, published several more papers in peer-reviewed journals where I had co-authors, and wrote a Ph.D. dissertation that was reviewed as "an original contribution to knowledge worthy of publication". The papers I published were reviewed essentially on the usual criteria of "new, correct, and significant". Here is one place the author gets way off track: For a technical research paper, e.g., new, correct, and significant, reading it is difficult, and judging if it is new, correct, and significant is more difficult. In an experimental science, judging if the work is reproducible adds more difficulty. It's TOUGH work. The work is so tough that for some huge majority of papers that are published, nearly the only careful reading at least for some years will be during the review process. Maybe later the paper will get referenced, carefully read by many people, etc. So, the idea of the article of this thread that commonly for papers that just appear on-line there can be 'votes' from many competent readings is just uninformed, misinformed, ignorant, just plain wrong, brain-dead nonsense. Here's more: Now, when an author A sends a paper to journal X, there is a good chance that the paper is actually in or close to the specialty S of journal X. So, the reviewers of journal X have a good chance of being among the best people to review the paper. If journal X does publish the paper, then author A, the paper, and people interested in specialty S get a lot of help because journal X has said that the paper is new, correct, and significant for specialty S. STILL, even with all this help for specialty S, the paper will likely be carefully and competently read at least for some years essentially only by the reviewers of journal X. Since it is so difficult even with the help of journal X in specialty S to get such papers read, for a paper just posted on-line there is no hope at all. Maybe in some experimental sciences reading a paper is relatively easy; however, judging if the experimental work is reproducible promises to be difficult. Maybe not in experimental science but in essentially all of the rest, there is a fundamental difficulty about research that is new, correct, and significant: Almost necessarily nearly no one will understand the paper easily or understand the paper at all. Else, given basic academic competitiveness, the results in the paper would long since have been old instead of new. So, the research and review processes have to work always right at the edge of what can currently be understood at all. For meeting this fundamental difficulty, the present journal system helps, and just putting papers on the Internet is hopeless.

rnadna 15y ago

Yesterday I got an email from a journal asking me to submit a paper. They needed it within 2 weeks. And it would be published 2 weeks thereafter. After I spewed coffee through my nose, laughing, I clicked the "spam" button. Publication in real journals is slow; reviewers are asked to send comments within a month, but they often take more than that. Publication in real journals is difficult; the best-ranked journals reject 90 percent of submissions. Publication in real journals is flexible; complicated decisions about how to transform unpublishable into publishable are made in each step along a long, often meandering and complicated, path. The simple answer to the question posed ("why publish in peer-reviewed journals") is that, for many fields, not doing so ends a career. But if the question is, as it seems by comments here and in many conversations I've had with students, "what value do peer-reviewed journals have" the answer is that, for many fields, doing so improves the science for the individual paper, and for the larger corpus.