7 ms·

How would you incentivize peer review with your approach?

codelord 9y ago
Do you think Springer pays for reviewing? No publisher does. It's all voluntary work by the same people who write the papers. It's the modern day slavery.
- philipodonnell 9y ago
  The parent comment implied that reviewers would be anonymized with this Github-based approach unless the PR was approved, which seems like it would lead to more acceptances so that the reviewers could get credit.
jfaucett 9y ago
First, I'd question that you really need to incentivize it. People - myself included - are doing research in a topic because they are passionate about it and most are more than happy to look at new research and ideas about things they are passionate about. Will you really need incentivization to get people to interact with the topic they have devoted their life towards studying? Let's say however that you need incentivization, I suppose one approach would be at a legislative level where you require all researchers publish on a decentralized open access platform where every citizen - they are the one's funding the research after all - can freely and uninhibitedly access the information.
- philipodonnell 9y ago
  > Will you really need incentivization to get people to interact with the topic they have devoted their life towards studying? Perhaps not. But people keep telling me in this thread that reviewers work because the journals expect them to or they will get future papers rejected/lose conference access, not for the love of the topic or because they get paid or otherwise any kind of special visibility for it. I wonder whether an approach that promised only publicity for having done reviews or relying on altruism would really produce the quality of reviews necessary. > I suppose one approach would be at a legislative level where you require all researchers publish on a decentralized open access platform where every citizen - they are the one's funding the research after all - can freely and uninhibitedly access the information. That's not really incentivizing the _reviewers_, though, right?
cat199 9y ago
not parent, but research association conference presentations might help to centralize announcements/review efforts but also keep things under researcher purview..
sevensor 9y ago
The same way we do now? There's a strong cultural expectation that one will agree to review, or have one's graduate students review, a reasonable number of papers. If you're a member of the Foo committee of the Society for Bar, you're not going to refuse to pull your weight as a reviewer for the Society for Bar's International Journal of Foo unless you want to get the cold shoulder at your next conference. It's not like anybody gets paid for this today.
chr1 9y ago
By providing opportunity to find errors in research of people you don't like! More seriously though, peer review is usually free or almost free, reviewers do not get anything near to the ridiculous amount of the money that "open access" journals demand. Besides, having articles on github, with all the data, so that any reader can check the results, and comment, would be much more powerful than the type of peer review we have today.
deleted 9y ago
[deleted]
RangerScience 9y ago
Clearly, blockchain! But no seriously, this seems like an actual technology / problem fit, although you'd need a decent amount of tooling, product design, etc for it to work - but some of that's already being explored by the "open source + blockchain" endeavors. In basic form, I imagine you'd basically turn the transaction fees into review fees. (The product design and tooling comes from how you prevent abuses to that basic format, I think)
- philipodonnell 9y ago
  I actually think this is a pretty good use case, since it fits the decentralized trust aspect that many common blockchain use cases lack. > some of that's already being explored by the "open source + blockchain" endeavors. Any links?
  - fnordsensei 9y ago
    I work for an Open Access publisher/research discovery technology company, and we literally have this on the drawing board. We're small, but well connected in the world of publishing and research. I for one would be also be interested in links. One thing we don't want to do is reinvent the actual blockchain technology, so we're also interested in finding partnerships in terms of platform technology.
    - philipodonnell 9y ago
      A proof of stake smart contract would be perfect for this. The currency is reputation earned by writing well-done reviews and 'staked' behind reviews they believe are well-done. Papers need a certain number of well-done reviews with the stakes on both sides accumulating behind approve/deny. You have to pay with the currency to have your paper reviewed and the currencies are distributed to the reviewers to allow monetization via exchanges with a small amount withdrawn as a siphon to support the continued development of the network and marketing/public relations.
      - fnordsensei 9y ago
jononor 9y ago
Reviewers get their names on the paper too. Which means if the paper was bad, well, they can be held responsible for it too.
- roel_v 9y ago
  In what field is that? I've never seen this happen, nor would I want to be on any paper I merely reviewed.
  - jokla 9y ago
    I think this was meant as a suggestion, I've also never seen this happen. However, I am wondering why and so may I ask why you do not want to be mentioned as reviewer on a paper, maybe even with your review?

Do you think Springer pays for reviewing? No publisher does. It's all voluntary work by the same people who write the papers. It's the modern day slavery.

philipodonnell 9y ago
The parent comment implied that reviewers would be anonymized with this Github-based approach unless the PR was approved, which seems like it would lead to more acceptances so that the reviewers could get credit.

jfaucett 9y ago

First, I'd question that you really need to incentivize it. People - myself included - are doing research in a topic because they are passionate about it and most are more than happy to look at new research and ideas about things they are passionate about. Will you really need incentivization to get people to interact with the topic they have devoted their life towards studying? Let's say however that you need incentivization, I suppose one approach would be at a legislative level where you require all researchers publish on a decentralized open access platform where every citizen - they are the one's funding the research after all - can freely and uninhibitedly access the information.

philipodonnell 9y ago
> Will you really need incentivization to get people to interact with the topic they have devoted their life towards studying? Perhaps not. But people keep telling me in this thread that reviewers work because the journals expect them to or they will get future papers rejected/lose conference access, not for the love of the topic or because they get paid or otherwise any kind of special visibility for it. I wonder whether an approach that promised only publicity for having done reviews or relying on altruism would really produce the quality of reviews necessary. > I suppose one approach would be at a legislative level where you require all researchers publish on a decentralized open access platform where every citizen - they are the one's funding the research after all - can freely and uninhibitedly access the information. That's not really incentivizing the _reviewers_, though, right?

cat199 9y ago

not parent, but research association conference presentations might help to centralize announcements/review efforts but also keep things under researcher purview..

sevensor 9y ago

The same way we do now? There's a strong cultural expectation that one will agree to review, or have one's graduate students review, a reasonable number of papers. If you're a member of the Foo committee of the Society for Bar, you're not going to refuse to pull your weight as a reviewer for the Society for Bar's International Journal of Foo unless you want to get the cold shoulder at your next conference. It's not like anybody gets paid for this today.

chr1 9y ago

By providing opportunity to find errors in research of people you don't like! More seriously though, peer review is usually free or almost free, reviewers do not get anything near to the ridiculous amount of the money that "open access" journals demand. Besides, having articles on github, with all the data, so that any reader can check the results, and comment, would be much more powerful than the type of peer review we have today.

deleted 9y ago

[deleted]

RangerScience 9y ago

Clearly, blockchain! But no seriously, this seems like an actual technology / problem fit, although you'd need a decent amount of tooling, product design, etc for it to work - but some of that's already being explored by the "open source + blockchain" endeavors. In basic form, I imagine you'd basically turn the transaction fees into review fees. (The product design and tooling comes from how you prevent abuses to that basic format, I think)

philipodonnell 9y ago
I actually think this is a pretty good use case, since it fits the decentralized trust aspect that many common blockchain use cases lack. > some of that's already being explored by the "open source + blockchain" endeavors. Any links?
- fnordsensei 9y ago
  I work for an Open Access publisher/research discovery technology company, and we literally have this on the drawing board. We're small, but well connected in the world of publishing and research. I for one would be also be interested in links. One thing we don't want to do is reinvent the actual blockchain technology, so we're also interested in finding partnerships in terms of platform technology.
  - philipodonnell 9y ago
    A proof of stake smart contract would be perfect for this. The currency is reputation earned by writing well-done reviews and 'staked' behind reviews they believe are well-done. Papers need a certain number of well-done reviews with the stakes on both sides accumulating behind approve/deny. You have to pay with the currency to have your paper reviewed and the currencies are distributed to the reviewers to allow monetization via exchanges with a small amount withdrawn as a siphon to support the continued development of the network and marketing/public relations.
    - fnordsensei 9y ago

jononor 9y ago

Reviewers get their names on the paper too. Which means if the paper was bad, well, they can be held responsible for it too.

roel_v 9y ago
In what field is that? I've never seen this happen, nor would I want to be on any paper I merely reviewed.
- jokla 9y ago
  I think this was meant as a suggestion, I've also never seen this happen. However, I am wondering why and so may I ask why you do not want to be mentioned as reviewer on a paper, maybe even with your review?