5 ms·

> It's to abolish the failed peer review system. And replace with...?

jsmith45 7y ago
In many ways the real peer review comes after publication, when more people have seen the work, and can react to it. In some fields it is not terribly rare for flaws in the work get discovered at that point, and rebuttals get written and circulated (or possibly even published in the journal.) The prepublication peer review is not always a terribly high bar to pass. It does help to weed out some of the crap, but lots still get published, especially in less competitive journals.
chr1 7y ago
Publish your article with accompanying data and programs on sci-hub (github for science). Tell your colleagues about it if you normally hold seminars before publishing papers, or post to mailing list. As interested people read your paper they will post comments, open issues, or star your repository. If it gets many stars Nature will include a link to it in their monthly list of important works, which would help even more people to find your repository.
- cozzyd 7y ago
  So that Nature will be full of papers from researchers who hire the most clickfarmers to star their work?
  - chr1 7y ago
    Nature would not assign the same value to every star. Star from someone who works in a university and has large impact factor in the related field will have much higher value than a star from newly created account without any published work.
    - rusticpenn 7y ago
      That is then a circular problem. Most people who have high impact right now, know how to play the publishing game.
- meej 7y ago
  > Publish your article with accompanying data and programs on sci-hub (github for science). How does one publish on Sci-Hub? I would suggest sites like https://zenodo.org/ https://zenodo.org/ or https://osf.io/ https://osf.io/ for self-publishing research products. Or one's own institution's repository, if they have one (many do, see http://v2.sherpa.ac.uk/opendoar/ http://v2.sherpa.ac.uk/opendoar/).
  - matwood 7y ago
    Publish the paper and put the protocols on something like https://www.protocols.io https://www.protocols.io so the results can be reproduced.
- jononor 7y ago
  There are no repositories or publishing functions on sci-hub. It is only a kind of proxy for already published works.
  - chr1 7y ago
    I did not mean the existing scihub site, but simply something that would be github for science. It seemed to me amusing to reference scihub in the name, but unfortunately looks like it is confusing, and diverts attention from the content of my comment, which is that publishing paper should be similar to publishing a repository on github, and reviewing should be like reviewing pull requests.
    - beojan 7y ago
      That sounds like arXiv.
      - chr1 7y ago
        arXiv doesn't provide a way to review, comment, or rate papers which is needed to replace journals by a better process.
- brazzy 7y ago
  Turn science into an unmitigated popularity contest, how could that possibly go wrong?!?
  - chr1 7y ago
    Stack overflow is a "popularity contest" and nothing has gone wrong with it so far, and science have always been popularity contest, i am just suggesting to give it better tools to measure the popularity.
BurningFrog 7y ago
The way things were done before the current system took over (in the 1980s, I believe). Not sure what it was exactly. I heard rumors that the journal often did its own review. Either way, if it was good enough for Einstein and Feynman, it might work for others. The important thing is that the people whose careers might be upended by revolutionary findings can't stop or steal the ideas.
- Cyph0n 7y ago
  So you want to replace the current peer review system with one you don’t really understand? Have you tried looking into why the system was changed in the first place? The fact that it worked for Einstein doesn’t make it a better system. Do you think Einstein wouldn’t have gotten his revolutionary work published with the current system?
  - petschge 7y ago
    Btw: The old system was the editor reading and reviewing all submissions himself. The new system of only a quick glance by the editor and then further scrutiny by a peer was instituted for two reasons: 1.) to many submissions for the editor to read all and 2.) increasing specialization. So going back to the old system would either lead to very cursory and potentially incompetent reviews or the increase in the number of editors to a point where all current senior reviewers are now editors. The former is completely intractable and the latter is still worse then the current status quo, especially since it removes the practice of having two or more reviews that specialize at different aspects (e.g. having on reviewer focus on the numerical side of the simulation and another on the statistical side of the comparison with observation data).
    - BurningFrog 7y ago
      > So going back to the old system would either lead to very cursory and potentially incompetent reviews... Yeah, this is side I'd pick. I'd rather have many mediocre submissions published fast than the current system. Let the "real" review happens after after the results are public. I see how reasonable people can pick the other side.
JMTQp8lwXL 7y ago
The only reason the review process is politicized is because academic institutions compete for research money. Preventing your peer institutions from publishing helps you get money for your own. You need to change how research is funded. Follow the money.
- BurningFrog 7y ago
  It's not the only reason. There is also the glory of being the genius leader in your field, and all the prizes, titles and lovers that come with it. > Preventing your peer institutions from publishing helps you get money for your own. You need to change how research is funded. Surely removing people's opportunity to veto their competitors publications is a more direct way!
- thaumasiotes 7y ago
  > The only reason the review process is politicized is because academic institutions compete for research money. Well, publication is also how researchers get promoted by people who can't evaluate their research. There's a within-department need outside of just attracting grants.
- slavik81 7y ago
  Politics have been part of the peer review process for a very, very long time. It's not all about the money, either: > Fourier was interested in heat propagation, and presented a paper in 1807 to the Institut de France on the use of sinusoids to represent temperature distributions. The paper contained the controversial claim that any continuous periodic signal could be represented as the sum of properly chosen sinusoidal waves. Among the reviewers were two of history's most famous mathematicians, Joseph Louis Lagrange (1736-1813), and Pierre Simon de Laplace (1749-1827). > While Laplace and the other reviewers voted to publish the paper, Lagrange adamantly protested. For nearly 50 years, Lagrange had insisted that such an approach could not be used to represent signals with corners, i.e., discontinuous slopes, such as in square waves. The Institut de France bowed to the prestige of Lagrange, and rejected Fourier's work. It was only after Lagrange died that the paper was finally published, some 15 years later. https://www.dspguide.com/ch8/1.htm https://www.dspguide.com/ch8/1.htm

In many ways the real peer review comes after publication, when more people have seen the work, and can react to it. In some fields it is not terribly rare for flaws in the work get discovered at that point, and rebuttals get written and circulated (or possibly even published in the journal.) The prepublication peer review is not always a terribly high bar to pass. It does help to weed out some of the crap, but lots still get published, especially in less competitive journals.

chr1 7y ago

Publish your article with accompanying data and programs on sci-hub (github for science). Tell your colleagues about it if you normally hold seminars before publishing papers, or post to mailing list. As interested people read your paper they will post comments, open issues, or star your repository. If it gets many stars Nature will include a link to it in their monthly list of important works, which would help even more people to find your repository.

cozzyd 7y ago
So that Nature will be full of papers from researchers who hire the most clickfarmers to star their work?
- chr1 7y ago
  Nature would not assign the same value to every star. Star from someone who works in a university and has large impact factor in the related field will have much higher value than a star from newly created account without any published work.
  - rusticpenn 7y ago
    That is then a circular problem. Most people who have high impact right now, know how to play the publishing game.
meej 7y ago
> Publish your article with accompanying data and programs on sci-hub (github for science). How does one publish on Sci-Hub? I would suggest sites like https://zenodo.org/ https://zenodo.org/ or https://osf.io/ https://osf.io/ for self-publishing research products. Or one's own institution's repository, if they have one (many do, see http://v2.sherpa.ac.uk/opendoar/ http://v2.sherpa.ac.uk/opendoar/).
- matwood 7y ago
  Publish the paper and put the protocols on something like https://www.protocols.io https://www.protocols.io so the results can be reproduced.
jononor 7y ago
There are no repositories or publishing functions on sci-hub. It is only a kind of proxy for already published works.
- chr1 7y ago
  I did not mean the existing scihub site, but simply something that would be github for science. It seemed to me amusing to reference scihub in the name, but unfortunately looks like it is confusing, and diverts attention from the content of my comment, which is that publishing paper should be similar to publishing a repository on github, and reviewing should be like reviewing pull requests.
  - beojan 7y ago
    That sounds like arXiv.
    - chr1 7y ago
      arXiv doesn't provide a way to review, comment, or rate papers which is needed to replace journals by a better process.
brazzy 7y ago
Turn science into an unmitigated popularity contest, how could that possibly go wrong?!?
- chr1 7y ago
  Stack overflow is a "popularity contest" and nothing has gone wrong with it so far, and science have always been popularity contest, i am just suggesting to give it better tools to measure the popularity.

BurningFrog 7y ago

The way things were done before the current system took over (in the 1980s, I believe). Not sure what it was exactly. I heard rumors that the journal often did its own review. Either way, if it was good enough for Einstein and Feynman, it might work for others. The important thing is that the people whose careers might be upended by revolutionary findings can't stop or steal the ideas.

Cyph0n 7y ago
So you want to replace the current peer review system with one you don’t really understand? Have you tried looking into why the system was changed in the first place? The fact that it worked for Einstein doesn’t make it a better system. Do you think Einstein wouldn’t have gotten his revolutionary work published with the current system?
- petschge 7y ago
  Btw: The old system was the editor reading and reviewing all submissions himself. The new system of only a quick glance by the editor and then further scrutiny by a peer was instituted for two reasons: 1.) to many submissions for the editor to read all and 2.) increasing specialization. So going back to the old system would either lead to very cursory and potentially incompetent reviews or the increase in the number of editors to a point where all current senior reviewers are now editors. The former is completely intractable and the latter is still worse then the current status quo, especially since it removes the practice of having two or more reviews that specialize at different aspects (e.g. having on reviewer focus on the numerical side of the simulation and another on the statistical side of the comparison with observation data).
  - BurningFrog 7y ago
    > So going back to the old system would either lead to very cursory and potentially incompetent reviews... Yeah, this is side I'd pick. I'd rather have many mediocre submissions published fast than the current system. Let the "real" review happens after after the results are public. I see how reasonable people can pick the other side.

JMTQp8lwXL 7y ago

The only reason the review process is politicized is because academic institutions compete for research money. Preventing your peer institutions from publishing helps you get money for your own. You need to change how research is funded. Follow the money.

BurningFrog 7y ago
It's not the only reason. There is also the glory of being the genius leader in your field, and all the prizes, titles and lovers that come with it. > Preventing your peer institutions from publishing helps you get money for your own. You need to change how research is funded. Surely removing people's opportunity to veto their competitors publications is a more direct way!
thaumasiotes 7y ago
> The only reason the review process is politicized is because academic institutions compete for research money. Well, publication is also how researchers get promoted by people who can't evaluate their research. There's a within-department need outside of just attracting grants.
slavik81 7y ago
Politics have been part of the peer review process for a very, very long time. It's not all about the money, either: > Fourier was interested in heat propagation, and presented a paper in 1807 to the Institut de France on the use of sinusoids to represent temperature distributions. The paper contained the controversial claim that any continuous periodic signal could be represented as the sum of properly chosen sinusoidal waves. Among the reviewers were two of history's most famous mathematicians, Joseph Louis Lagrange (1736-1813), and Pierre Simon de Laplace (1749-1827). > While Laplace and the other reviewers voted to publish the paper, Lagrange adamantly protested. For nearly 50 years, Lagrange had insisted that such an approach could not be used to represent signals with corners, i.e., discontinuous slopes, such as in square waves. The Institut de France bowed to the prestige of Lagrange, and rejected Fourier's work. It was only after Lagrange died that the paper was finally published, some 15 years later. https://www.dspguide.com/ch8/1.htm https://www.dspguide.com/ch8/1.htm