5 ms·

Is there any proof peer review has increased scientific productivity? It is relatively new, and I don't see any reason prima facie why it would work better than

Is there any proof peer review has increased scientific productivity? It is relatively new, and I don't see any reason prima facie why it would work better than an editorial teaming willing to delegate when necessary. Anecdotally, I see lots of academics complain about very important results being unable to make it through peer review. Ralph Merkel's early groundbreaking cryptography work was suppressed for years by unthinking peer review. From Merklel: >My first paper (and, in fact, the first paper) on public key cryptography was submitted in 1974 and initially rejected by an unknown "cryptography expert" because it was "...not in the main stream of present cryptography thinking...."

sokoloff 7y ago
Isn’t peer review at most journals done by “an editorial team delegating as necessary”? I married into the field, didn’t work in it, but that’s what I’ve observed from the side.
- tptacek 7y ago
  The ACM and Usenix peer review work I did involved rotating teams of volunteers and very little guidance from the publication/venue.
  - PeterisP 7y ago
    Well yes, but the review power (both nominally and in practice) does lie with the publication/venue editor[ial team], it's just that they have decided to delegate almost all that power to rotating teams of volunteers, because they can, and because they want to for obvious practical and financial reasons. A system with an "editorial teaming willing to delegate when necessary" simply becomes the current system as the editorial team is willing and able to delegate, and it's neccessary pretty much all the time because of resource constraints - because being an editor generally also is an unpaid volunteer position done in addition to your normal work in academia.
lonelappde 7y ago
You are thinking of journal review, which hijacked peer review until arxiv and the internet in general rescued it.
michaelt 7y ago
Is there any proof peer review has increased scientific productivity? It is relatively new According to [1] "The council minutes of 1 March 1665 made provisions for the tract to be revised by members of the council of the Royal Society, providing the framework for peer review to eventually develop, becoming fully systematic as a process by the 1830s." Peer review is a relatively new procedure in the same way that California is a relatively new state :-p [1] https://en.wikipedia.org/wiki/Philosophical_Transactions_of_the_Royal_Society#Origins https://en.wikipedia.org/wiki/Philosophical_Transactions_of_...
- nextos 7y ago
  However, some famous papers like Watson & Crick's DNA model were published without peer review.
  - asdff 7y ago
    You can publish today in PNAS without review if you are a member of the national academy of sciences.
    - jhbadger 7y ago
      Not exactly any more. It's true that it used to be that a member could just contribute their own manuscripts which would be published as is without review. But after some unfortunate cases of questionable stuff getting published, even the contributed track has to be peer reviewed these days. However members still have leverage in that they can choose the reviewers, which obviously could be abused.
- BurningFrog 7y ago
  The concept is certainly old, but it hasn't been a systematic part of the "universal scientific process" for more than a few decades. Most revolutionary scientific discoveries were made without it.
- rrss 7y ago
  "Peer review" nowadays refers almost exclusively to prepublication review by people other than the editor. This was still relatively rare at the beginning of the 20th century. If you name a famous publication prior to 1900, it almost certainly was published without peer review.

sokoloff 7y ago

Isn’t peer review at most journals done by “an editorial team delegating as necessary”? I married into the field, didn’t work in it, but that’s what I’ve observed from the side.

tptacek 7y ago
The ACM and Usenix peer review work I did involved rotating teams of volunteers and very little guidance from the publication/venue.
- PeterisP 7y ago
  Well yes, but the review power (both nominally and in practice) does lie with the publication/venue editor[ial team], it's just that they have decided to delegate almost all that power to rotating teams of volunteers, because they can, and because they want to for obvious practical and financial reasons. A system with an "editorial teaming willing to delegate when necessary" simply becomes the current system as the editorial team is willing and able to delegate, and it's neccessary pretty much all the time because of resource constraints - because being an editor generally also is an unpaid volunteer position done in addition to your normal work in academia.

lonelappde 7y ago

You are thinking of journal review, which hijacked peer review until arxiv and the internet in general rescued it.

michaelt 7y ago

Is there any proof peer review has increased scientific productivity? It is relatively new According to [1] "The council minutes of 1 March 1665 made provisions for the tract to be revised by members of the council of the Royal Society, providing the framework for peer review to eventually develop, becoming fully systematic as a process by the 1830s." Peer review is a relatively new procedure in the same way that California is a relatively new state :-p [1] https://en.wikipedia.org/wiki/Philosophical_Transactions_of_the_Royal_Society#Origins https://en.wikipedia.org/wiki/Philosophical_Transactions_of_...

nextos 7y ago
However, some famous papers like Watson & Crick's DNA model were published without peer review.
- asdff 7y ago
  You can publish today in PNAS without review if you are a member of the national academy of sciences.
  - jhbadger 7y ago
    Not exactly any more. It's true that it used to be that a member could just contribute their own manuscripts which would be published as is without review. But after some unfortunate cases of questionable stuff getting published, even the contributed track has to be peer reviewed these days. However members still have leverage in that they can choose the reviewers, which obviously could be abused.
BurningFrog 7y ago
The concept is certainly old, but it hasn't been a systematic part of the "universal scientific process" for more than a few decades. Most revolutionary scientific discoveries were made without it.
rrss 7y ago
"Peer review" nowadays refers almost exclusively to prepublication review by people other than the editor. This was still relatively rare at the beginning of the 20th century. If you name a famous publication prior to 1900, it almost certainly was published without peer review.