7 ms·

Should journals pay peer reviewes?

40 points by dariosalvi78 6y ago

zackkatz 6y ago
There’s, ironically, a typo in the title. Should be `reviewers`.
prepend 6y ago
I’d rather just reduce the number of for profit journals. Getting paid $450 seems to be against open access as open journals can’t afford this. Also the motivations are different when pay is involved. I will donate time to peer review, but $450 is not enough to pay me. So I would be more likely to review as a volunteer than an employee.
- RBerenguel 6y ago
  I want to contend that to most non-tenured professors (outside of the States at least?) 450$, even if not enough as pay either would be a nice extra from time to time.
  - michaelmior 6y ago
    As a non-tenured professor within the US, I don't really mind reviewing for free. But $450 per review would indeed be a nice extra :)
  - prepend 6y ago
    I like money and $450 is a nice amount (hey, free PlayStation). But my employer doesn’t allow outside payments so I couldn’t accept. And while I like money, I fee a duty to review a few papers a year, even if they are not good, or boring, or whatever. If I was being paid, I would be more likely to skip those papers because I no longer feel a duty. This may be a similar motivation described in an experiment that showed how late pickups went up when the daycare introduced late fees. [0] I think that many in academia are underpaid and should have more money. I just think that adding review fees are not the way. [0] https://freakonomics.com/2013/10/23/what-makes-people-do-what-they-do/ https://freakonomics.com/2013/10/23/what-makes-people-do-wha...
    - RBerenguel 6y ago
      In my (well, former) field you can only review papers if you are affiliated with a research organisation, so peer reviewers are necessarily in academia.
- ta988 6y ago
  I agree we don't need the publishers anymore. We have preprint services (arxiv and others), open review systems (pubpeer), sharing platforms (osf, zenodo, dataverse). You have Free software to handle all the manuscript lifecycle (there are a few cant remember the names). We keep the publishers because they still have some prestige and they are blackmailing university libraries. Now there is Scihub to go atound that, but of course there are ethical and legal reasons associated with its use so in my opinion thats not a long term alternative.
ktpsns 6y ago
Well, I much more like the question: Should authors pay for their publications? There is a common pun: As a scientist, you pay two times for an article: The one time you publish, and the other time when you subscribe to read your published article. And this is only to access the article as a PDF in the web. There is no print copy or so involved in that payment. Notorious XKCD: https://xkcd.com/2085/ https://xkcd.com/2085/
- michaelmior 6y ago
  This depends on the field. I have never paid to publish and I doubt I ever would. (This does not count however the conference expenses typically required of an author to present their paper. But attending a conference has significant benefit to me personally, so I don't really see it as a direct cost of publication.)
- ta988 6y ago
  You usually have three models: - pay for publish and read: big tabloid journals, large medical maf... associations. Cost to publish is usually below $1000, cost to read is $5-$20 a page - pay for read, free to publish: most journals from the big publishers. Cost is between $5 to $20 a page. - pay to publish, free to read: the model of most Open Access journals. Some big publishers are proposing that as an option as well. Cost of that option is usually $3000 to $5000 per article.
sokoloff 6y ago
Looking at the real-estate market and its assortment of minimal-value-add/maximum-value-extraction players (appraisals, title insurance firms, 75% of inspectors) as an analog, I suspect that directly paying for reviews will not likely improve the situation. Perhaps a recognition by institution or university model would work, or softening the dollars (they are credits against APCs or something) might work.
- ta988 6y ago
  Many universities take reviewing "into account " for the tenure-track process and performance reviews. but it counts really little compared to publications. Data sharing or management of ressources (databases, forums...) is completely ignored usually.
dash2 6y ago
I've seen some journals do this. The problem TBH is that the amounts paid aren't enough to be an incentive. Typically £50-100. A good review takes a day to do (maybe more). And academics are probably paid £100/day or more anyway, and aren't very money-motivated. I think a better approach would be publicity. Academics care about reputation a lot. If good reviewers were acknowledged by the journal, that would be a big incentive, something you could put on your CV.
- dairyleia 6y ago
  This is exactly why they like Publons, it lets them take credit for their reviews.
  - michaelmior 6y ago
    Publons is a nice way of keeping track and I use it myself, but I think institutions are generally willing to take faculty at their word with respect to reviews we have completed. (Although my statement here is really just based on anecdotal evidence, so I could be wrong.
- foobarbecue 6y ago
  Great point. Offering to pay a small amount could actually reduce the number of people willing to review, since they are currently motivated by a sense of duty. Reminds me of the Freakonomics chapter with the day care: https://freakonomics.com/2013/10/23/what-makes-people-do-what-they-do/ https://freakonomics.com/2013/10/23/what-makes-people-do-wha...
DanielleMolloy 6y ago
Journals should not shed out tiny payments for peer reviews by overworked faculty, they should hire teams of reviewers with diverse specialisations full time. By decent quality assurance on the massive amount of papers each year publishers could actually do something valuable for science.
- michaelmior 6y ago
  I don't necessarily disagree with paying reviewers, but I think there is some value to having people who are actively working on problems in the same area being the ones doing the reviewing. Yes, it's additional work that we aren't paid for, but personally I enjoy it. I do agree though that for some it can create undue pressure so there may be something in the middle there that works well.
  - DanielleMolloy 6y ago
    I understand this point, yet to correctly assess whether a typical paper in my field - for instance - can be trusted you would need a modern neural networks researcher, a statistician, a cognitive scientist, a neuroscientist and perhaps a physicist. All would have to know about common fallacies in their respective fields and should optimally discuss with another. Faculty reviewers are not only overworked, they have accumulated biases, can be overconfident about their expertise and in the worst case have malicious incentive to prohibit publication of your results. I’m certain that a team of specialists who review as their full time job would do a better job than at least half of the reviewers I encountered (especially those at high-profile AI conferences). I enjoy reviewing too and am doing my best to be constructive, but I’m not sufficiently qualified for most of the offers I receive.
    - michaelmior 6y ago
      I suppose a lot of this is dependent on the field. I think anyone who has had their work peer reviewed has many stories of reviews from people who seem to have completely misunderstood the work in question. Certainly there can be accumulated biases and overconfidence among faculty reviewers, but I don't see how that would be less prevalent among paid reviewers.
gww 6y ago
If the journal is paying a reviewer would there be more pressure in terms of turn around time? Or more pressure to accept contentious papers because the journal expects them to be highly cited or wants to pander to their authors? I also imagine there would be people out there who would become serial reviewers with the goal of making as much money as possible. I imagine the quality of their reviews wouldn't be great either.
- michaelmior 6y ago
  For turnaround time, I would probably say yes. Right now it's typically to have several months to review a journal paper. And there's really not much a journal can do if a reviewer is late other than not ask them to review again. Even if reviewers were paid, I suspect it would be difficult from a budgetary perspective to difficult to have serial reviewing be lucrative with the possible exception of adjunct faculty who unfortunately are typically very underpaid.
  - ta988 6y ago
    I'm curious to know in what field you get months for reviewing. In chemistry almost everything is 2-3 weeks, lower for open access journals.
    - gww 6y ago
      In the life sciences and bioinformatics I have had to wait 2-4 months for first and second round reviews.
      - ta988 6y ago
        Wait time is different. It is due to the editor overwhelmed, and the difficulty to find reviewers and also their typesetting made in foreign country with cheaper labor.
    - michaelmior 6y ago
      Computer science for me. It depends on the publication. Some certainly have shorter turnarounds, but several months is not uncommon.
Mediterraneo10 6y ago
No, please. There are still fields where journals are both open-access and not pay-to-publish. This is because they are published by the same non-profit learned societies they always have been for many decades, and overhead costs are low. Bring in a culture of paying peer reviewers, and suddenly authors (or the learned society) will be under more pressure to get more grant money to pay the new publication costs. Also, as someone who works in academia on an occasional basis, taking years off from funded positions at a time, I appreciate being able to write the occasional article on my own time and get it published without paying hundreds and hundreds of euro out of my own pocket -- fields with pay-to-publish traditions make it hard for non-permanent faculty to contribute.
- jariel 6y ago
  This is insightful, thanks, but there's a little problem here: The cost is borne one way or the other, the $500 is a matter of comp, not cost. If someone is going to review your work, that's an act of very material labour. That they are not really compensated for it, at all, is inconsistent with the basic premise of labour and comp. You would not be a researcher if you weren't comped in some way, so the absence and/or misalignment of comp is the issue. It's not free to publish and have 2 reviewers (!!) there is a very material cost that you're just not paying. Allocating at $1K cost for publication more accurately reflects the labour involved in checking it out (at least a little bit). It'd seem to me a lot of folks might want to take up the $500/review as a job to earn some reasonable income, they might even get good at it. It's entirely plausible that one of the 'dysfunctions' of science is 'not enough review' and it's simply because there is no incentive structure for it at all. We pay people for literally every other function in life. Doctors are paid to 'save lives' even though there is a moral obligation. The notion that we don't pay people for academic work seems odd. Perhaps journals could identify the papers with the most promise and offer payment for review, or something like that.
  - Mediterraneo10 6y ago
    Unpaid peer reviewing is certainly not incentive-free. For example, oftentimes one enjoys peer reviewing because the author cites some publications that are useful or important, but which one wasn't previously aware of. Sometimes the author makes some observation tangential to his/her main argument which proves an inspiration or key to one's own research. These sort of things may be more common in the humanities versus hard STEM, but in my own field peer review is not generally seen as drudgery or exploitation.
pulse7 6y ago
It is written somewhere: "Those who work deserve their pay!"
jmakov 6y ago
Sholud you pay a programmer for code reviews?
- michaelmior 6y ago
  If that programmer doesn't work for your company and you want them to do it in a timely manner, then yes, I think it's reasonable to do so. For open access publications, you could argue that reviewing is similar to working on an open source project, but open access is still the exception, not the norm.
jrochkind1 6y ago
So, if the problem is that scientists lack motivation to peer review, I think having peer review work recognized as important in tenure-and-promotion would probably be far more motivating than whatever hourly rate they'd end up paying. Article authors are generally not paid! Yet there is no lack of motivation to write articles. How to motivate to do peer-review well instead of perfunctorily is another question.
- ta988 6y ago
  Article authors often pay as well. Either directly because the journal is a science-tabloid or because the authors want to make their work accessible to all. Or indirectly through the indirect cost the universities are taking from grant agencies to pay for their confidential agreements with publishers. Try to do a FOIA of a state university to get that information you will see how hard they will fight the fact that they get scammed. Authors write because this is how you justify that you are really working, be it for your university, department or your grant agency. Also you want others to build upon your work, but academics would publish a lot less if it was not for the huge pressure from funders and employers (increasing numbers is all that count for many of them).
  - jrochkind1 6y ago
    Right. Peer review is also essential to the production of research articles. And is done largely by the same people who are writing articles; but they are incentivized non-monetarily to write articles, but not as much to review. To restate my point perhaps more clearly: If reviewing were rewarded/expected as part of one's role at a university, department, or grant agency similarly to authorship, that would do far more to incentivize it than paying a relatively modest amount for a review.
- michaelmior 6y ago
  I would argue that article authors generally are paid indirectly by the institution which employs them. Most scientific publications are authored by academics or members of research labs who are specifically paid to publish. Of course, pay is not directly tied to each publication, but that is what most authors are paid for. At least in my institution, contributing to the peer review process is also an expected part of the process to achieve tenure. But the incentive is certainly far less than the incentive to publish.
deleted 6y ago
[deleted]
mgr86 6y ago
Can you share some of these non profits? I work at one who used to publish a journal for decades, but Sage has handled it in more recent memory. We make our salaries through grants and a library product. But I am not aware of others doing similar work. Partially out of ignorance.
rocqua 6y ago
How about making reviews available under the name of the reviewer. This means good reviewers can get respect of their reviewees, and reviewers are incentivized to write with slightly more consideration. Similarly, universities should value good reviewers more. Which probably means reviewers need to get more prestige, so universities get prestige for having good reviewers.
- ta988 6y ago
  This is something that is pushed by some people. With many journals I review for however, I am able to see otger reviewers reviews and I suppose they would be ashamed to have their names to be associated with them (extremely short and with generoc sentences that looks like they didnt even read the paper or just plainly agressive). I've heard many arguments against open review, but the most common is that people are afraid of retaliations if they criticize a work. I understand that people have retaliated to me after comments on PubPeer. But I still think we should do open reviews, that doesnt mean that journals that make people pay for publishing or reading should pay reviewers. We work for free get 0 reward... It is a bit like asking a designer to work for you, and tell them that it is to help the world of design and thank you no you can't sign...
karl11 6y ago
Yes. And their names should be attached to anything they review (including the papers they reject) so that they have some skin in the game.
- foobarbecue 6y ago
  Hunh, this is a great idea. Rejected articles should be published as rejected, and all reviews should be published. I guess this is a bit like the open review model, like The Cryosphere uses, except I think they hide papers after the discussion period if they aren't accepted.
- michaelmior 6y ago
  I think the problem with this is that it can encourage retaliation. If I know you rejected my paper, I might be incline to reject yours. Now, a nobody like me probably can't get away with doing that to someone with an established reputation. But the other way around is certainly possible.
AcerbicZero 6y ago
Can’t we jus my have a GitHub for science and get this over with? It seems silly to maintain this system that may have been functional when it started, but is now clearly behind the technology curve.
- michaelmior 6y ago
  How would that address the core question being asked which is whether work spent reviewing should be paid? In any case, while there are problems with peer review, I don't think it's related to technology.
TheOtherHobbes 6y ago
I don't think the process would be less effective if papers were published without peer review, but were then questioned and challenged in public debate under editorial moderation. Peer review isn't a single-goal process. It exists to introduce citations of existing work if they aren't already referenced, to question assumptions, and to find obvious errors. (Among others.) All of these could be done in peer debate. It's not as if the process is genuinely anonymous anyway. Of course it pretends to be, but it isn't. Peer review dates from the days when information travelled as fast as a horse, and it made sense to include a review stage for basic quality control. I'm not convinced it's a good model now, when information exchange is rather quicker. The real problem is that journal publication is about career status, not about quality of work. Peer review is almost a side issue. The scientific publishing industry is not just financially parasitic, it also introduces unnecessary delays in idea exchange. Public peer debate would eliminate some of that friction, but it would also have public consequences for reputation - and that might well be more effective for quality control than pseudo-anonymity.
- michaelmior 6y ago
  At least in my field, I expect this would be a total disaster. One of the major advantages of peer review is that knowing the top tier publication venues, I can look to these to keep up to date on the latest work knowing that what I find is likely to be high quality. In the absence of peer-reviewed publication venues, I see two options: 1. Read everything I can find that is published and try to sort through it all myself to sort out what is good quality. I think in aggregate, this is likely to take more of researchers' time than contributing to the peer review process. 2. Focus only on publications from top quality institutions or researchers whose work I am already familiar with. This has the advantage of saving me time. But it also has the massive disadvantage that I am likely to miss quality work which comes from lesser known institutions or authors. I don't think either of those two options is attractive. In my field, the delay of idea exchange is not really a problem. If people want to share their ideas, they can often publish a preprint in parallel with submission to a peer-reviewed journal. (Although some journals effectively prohibit this since it has the potential to break the anonymity of the review process.) Yes, some of the purposes of peer review could be accomplished in the method of "peer debate" you mention. But I think such a process would be much less effective. Of course, that's not to say I don't think there are things broken about peer review that could use fixing :)
jltsiren 6y ago
Like many other recent international hires at US universities, I'm on H-1B visa. If paying for reviews was the norm, I believe it would be illegal for me to review papers, even for free. I'm not sure what to think about this. Peer review is an important way of contributing to the community, and every academic is expected to participate in it. Commercial publishers that make profits from free academic labor are parasites, and they should pay for the services they use. Yet immigration rules often make it difficult to earn any side income, so paying for reviews would exclude many (often less established) academics.
- michaelmior 6y ago
  That is a good point and one I hadn't considered despite being an H-1B holder myself. I agree with your argument that paying for reviews under current immigration laws would be highly problematic for visa holders. It would either mean we would be expected to do it for free when everyone else gets paid, which would be unfair and potentially still not legal. Or we would be unable to review which I think is an important part of establishing oneself within the community.
taylodl 6y ago
Is there a way this can be made double-blind, where the Journal doesn't know the reviewer and the reviewer doesn't know the Journal? Can something akin to Mechanical Turk be utilized? Just trying to think of ways to have paid reviews without incurring all the negatives so typically associated with that arrangement.

40 points by dariosalvi78 6y ago

There’s, ironically, a typo in the title. Should be `reviewers`.

I’d rather just reduce the number of for profit journals. Getting paid $450 seems to be against open access as open journals can’t afford this. Also the motivations are different when pay is involved. I will donate time to peer review, but $450 is not enough to pay me. So I would be more likely to review as a volunteer than an employee.

RBerenguel 6y ago
I want to contend that to most non-tenured professors (outside of the States at least?) 450$, even if not enough as pay either would be a nice extra from time to time.
- michaelmior 6y ago
  As a non-tenured professor within the US, I don't really mind reviewing for free. But $450 per review would indeed be a nice extra :)
- prepend 6y ago
  I like money and $450 is a nice amount (hey, free PlayStation). But my employer doesn’t allow outside payments so I couldn’t accept. And while I like money, I fee a duty to review a few papers a year, even if they are not good, or boring, or whatever. If I was being paid, I would be more likely to skip those papers because I no longer feel a duty. This may be a similar motivation described in an experiment that showed how late pickups went up when the daycare introduced late fees. [0] I think that many in academia are underpaid and should have more money. I just think that adding review fees are not the way. [0] https://freakonomics.com/2013/10/23/what-makes-people-do-what-they-do/ https://freakonomics.com/2013/10/23/what-makes-people-do-wha...
  - RBerenguel 6y ago
    In my (well, former) field you can only review papers if you are affiliated with a research organisation, so peer reviewers are necessarily in academia.
ta988 6y ago
I agree we don't need the publishers anymore. We have preprint services (arxiv and others), open review systems (pubpeer), sharing platforms (osf, zenodo, dataverse). You have Free software to handle all the manuscript lifecycle (there are a few cant remember the names). We keep the publishers because they still have some prestige and they are blackmailing university libraries. Now there is Scihub to go atound that, but of course there are ethical and legal reasons associated with its use so in my opinion thats not a long term alternative.

ktpsns 6y ago

Well, I much more like the question: Should authors pay for their publications? There is a common pun: As a scientist, you pay two times for an article: The one time you publish, and the other time when you subscribe to read your published article. And this is only to access the article as a PDF in the web. There is no print copy or so involved in that payment. Notorious XKCD: https://xkcd.com/2085/ https://xkcd.com/2085/

michaelmior 6y ago
This depends on the field. I have never paid to publish and I doubt I ever would. (This does not count however the conference expenses typically required of an author to present their paper. But attending a conference has significant benefit to me personally, so I don't really see it as a direct cost of publication.)
ta988 6y ago
You usually have three models: - pay for publish and read: big tabloid journals, large medical maf... associations. Cost to publish is usually below $1000, cost to read is $5-$20 a page - pay for read, free to publish: most journals from the big publishers. Cost is between $5 to $20 a page. - pay to publish, free to read: the model of most Open Access journals. Some big publishers are proposing that as an option as well. Cost of that option is usually $3000 to $5000 per article.

sokoloff 6y ago

Looking at the real-estate market and its assortment of minimal-value-add/maximum-value-extraction players (appraisals, title insurance firms, 75% of inspectors) as an analog, I suspect that directly paying for reviews will not likely improve the situation. Perhaps a recognition by institution or university model would work, or softening the dollars (they are credits against APCs or something) might work.

ta988 6y ago
Many universities take reviewing "into account " for the tenure-track process and performance reviews. but it counts really little compared to publications. Data sharing or management of ressources (databases, forums...) is completely ignored usually.

dash2 6y ago

I've seen some journals do this. The problem TBH is that the amounts paid aren't enough to be an incentive. Typically £50-100. A good review takes a day to do (maybe more). And academics are probably paid £100/day or more anyway, and aren't very money-motivated. I think a better approach would be publicity. Academics care about reputation a lot. If good reviewers were acknowledged by the journal, that would be a big incentive, something you could put on your CV.

dairyleia 6y ago
This is exactly why they like Publons, it lets them take credit for their reviews.
- michaelmior 6y ago
  Publons is a nice way of keeping track and I use it myself, but I think institutions are generally willing to take faculty at their word with respect to reviews we have completed. (Although my statement here is really just based on anecdotal evidence, so I could be wrong.
foobarbecue 6y ago
Great point. Offering to pay a small amount could actually reduce the number of people willing to review, since they are currently motivated by a sense of duty. Reminds me of the Freakonomics chapter with the day care: https://freakonomics.com/2013/10/23/what-makes-people-do-what-they-do/ https://freakonomics.com/2013/10/23/what-makes-people-do-wha...

DanielleMolloy 6y ago

Journals should not shed out tiny payments for peer reviews by overworked faculty, they should hire teams of reviewers with diverse specialisations full time. By decent quality assurance on the massive amount of papers each year publishers could actually do something valuable for science.

michaelmior 6y ago
I don't necessarily disagree with paying reviewers, but I think there is some value to having people who are actively working on problems in the same area being the ones doing the reviewing. Yes, it's additional work that we aren't paid for, but personally I enjoy it. I do agree though that for some it can create undue pressure so there may be something in the middle there that works well.
- DanielleMolloy 6y ago
  I understand this point, yet to correctly assess whether a typical paper in my field - for instance - can be trusted you would need a modern neural networks researcher, a statistician, a cognitive scientist, a neuroscientist and perhaps a physicist. All would have to know about common fallacies in their respective fields and should optimally discuss with another. Faculty reviewers are not only overworked, they have accumulated biases, can be overconfident about their expertise and in the worst case have malicious incentive to prohibit publication of your results. I’m certain that a team of specialists who review as their full time job would do a better job than at least half of the reviewers I encountered (especially those at high-profile AI conferences). I enjoy reviewing too and am doing my best to be constructive, but I’m not sufficiently qualified for most of the offers I receive.
  - michaelmior 6y ago
    I suppose a lot of this is dependent on the field. I think anyone who has had their work peer reviewed has many stories of reviews from people who seem to have completely misunderstood the work in question. Certainly there can be accumulated biases and overconfidence among faculty reviewers, but I don't see how that would be less prevalent among paid reviewers.

gww 6y ago

If the journal is paying a reviewer would there be more pressure in terms of turn around time? Or more pressure to accept contentious papers because the journal expects them to be highly cited or wants to pander to their authors? I also imagine there would be people out there who would become serial reviewers with the goal of making as much money as possible. I imagine the quality of their reviews wouldn't be great either.

michaelmior 6y ago
For turnaround time, I would probably say yes. Right now it's typically to have several months to review a journal paper. And there's really not much a journal can do if a reviewer is late other than not ask them to review again. Even if reviewers were paid, I suspect it would be difficult from a budgetary perspective to difficult to have serial reviewing be lucrative with the possible exception of adjunct faculty who unfortunately are typically very underpaid.
- ta988 6y ago
  I'm curious to know in what field you get months for reviewing. In chemistry almost everything is 2-3 weeks, lower for open access journals.
  - gww 6y ago
    In the life sciences and bioinformatics I have had to wait 2-4 months for first and second round reviews.
    - ta988 6y ago
      Wait time is different. It is due to the editor overwhelmed, and the difficulty to find reviewers and also their typesetting made in foreign country with cheaper labor.
  - michaelmior 6y ago
    Computer science for me. It depends on the publication. Some certainly have shorter turnarounds, but several months is not uncommon.

Mediterraneo10 6y ago

No, please. There are still fields where journals are both open-access and not pay-to-publish. This is because they are published by the same non-profit learned societies they always have been for many decades, and overhead costs are low. Bring in a culture of paying peer reviewers, and suddenly authors (or the learned society) will be under more pressure to get more grant money to pay the new publication costs. Also, as someone who works in academia on an occasional basis, taking years off from funded positions at a time, I appreciate being able to write the occasional article on my own time and get it published without paying hundreds and hundreds of euro out of my own pocket -- fields with pay-to-publish traditions make it hard for non-permanent faculty to contribute.

jariel 6y ago
This is insightful, thanks, but there's a little problem here: The cost is borne one way or the other, the $500 is a matter of comp, not cost. If someone is going to review your work, that's an act of very material labour. That they are not really compensated for it, at all, is inconsistent with the basic premise of labour and comp. You would not be a researcher if you weren't comped in some way, so the absence and/or misalignment of comp is the issue. It's not free to publish and have 2 reviewers (!!) there is a very material cost that you're just not paying. Allocating at $1K cost for publication more accurately reflects the labour involved in checking it out (at least a little bit). It'd seem to me a lot of folks might want to take up the $500/review as a job to earn some reasonable income, they might even get good at it. It's entirely plausible that one of the 'dysfunctions' of science is 'not enough review' and it's simply because there is no incentive structure for it at all. We pay people for literally every other function in life. Doctors are paid to 'save lives' even though there is a moral obligation. The notion that we don't pay people for academic work seems odd. Perhaps journals could identify the papers with the most promise and offer payment for review, or something like that.
- Mediterraneo10 6y ago
  Unpaid peer reviewing is certainly not incentive-free. For example, oftentimes one enjoys peer reviewing because the author cites some publications that are useful or important, but which one wasn't previously aware of. Sometimes the author makes some observation tangential to his/her main argument which proves an inspiration or key to one's own research. These sort of things may be more common in the humanities versus hard STEM, but in my own field peer review is not generally seen as drudgery or exploitation.

pulse7 6y ago

It is written somewhere: "Those who work deserve their pay!"

jmakov 6y ago

Sholud you pay a programmer for code reviews?

michaelmior 6y ago
If that programmer doesn't work for your company and you want them to do it in a timely manner, then yes, I think it's reasonable to do so. For open access publications, you could argue that reviewing is similar to working on an open source project, but open access is still the exception, not the norm.

jrochkind1 6y ago

So, if the problem is that scientists lack motivation to peer review, I think having peer review work recognized as important in tenure-and-promotion would probably be far more motivating than whatever hourly rate they'd end up paying. Article authors are generally not paid! Yet there is no lack of motivation to write articles. How to motivate to do peer-review well instead of perfunctorily is another question.

ta988 6y ago
Article authors often pay as well. Either directly because the journal is a science-tabloid or because the authors want to make their work accessible to all. Or indirectly through the indirect cost the universities are taking from grant agencies to pay for their confidential agreements with publishers. Try to do a FOIA of a state university to get that information you will see how hard they will fight the fact that they get scammed. Authors write because this is how you justify that you are really working, be it for your university, department or your grant agency. Also you want others to build upon your work, but academics would publish a lot less if it was not for the huge pressure from funders and employers (increasing numbers is all that count for many of them).
- jrochkind1 6y ago
  Right. Peer review is also essential to the production of research articles. And is done largely by the same people who are writing articles; but they are incentivized non-monetarily to write articles, but not as much to review. To restate my point perhaps more clearly: If reviewing were rewarded/expected as part of one's role at a university, department, or grant agency similarly to authorship, that would do far more to incentivize it than paying a relatively modest amount for a review.
michaelmior 6y ago
I would argue that article authors generally are paid indirectly by the institution which employs them. Most scientific publications are authored by academics or members of research labs who are specifically paid to publish. Of course, pay is not directly tied to each publication, but that is what most authors are paid for. At least in my institution, contributing to the peer review process is also an expected part of the process to achieve tenure. But the incentive is certainly far less than the incentive to publish.

deleted 6y ago

[deleted]

mgr86 6y ago

Can you share some of these non profits? I work at one who used to publish a journal for decades, but Sage has handled it in more recent memory. We make our salaries through grants and a library product. But I am not aware of others doing similar work. Partially out of ignorance.

rocqua 6y ago

How about making reviews available under the name of the reviewer. This means good reviewers can get respect of their reviewees, and reviewers are incentivized to write with slightly more consideration. Similarly, universities should value good reviewers more. Which probably means reviewers need to get more prestige, so universities get prestige for having good reviewers.

ta988 6y ago
This is something that is pushed by some people. With many journals I review for however, I am able to see otger reviewers reviews and I suppose they would be ashamed to have their names to be associated with them (extremely short and with generoc sentences that looks like they didnt even read the paper or just plainly agressive). I've heard many arguments against open review, but the most common is that people are afraid of retaliations if they criticize a work. I understand that people have retaliated to me after comments on PubPeer. But I still think we should do open reviews, that doesnt mean that journals that make people pay for publishing or reading should pay reviewers. We work for free get 0 reward... It is a bit like asking a designer to work for you, and tell them that it is to help the world of design and thank you no you can't sign...

karl11 6y ago

Yes. And their names should be attached to anything they review (including the papers they reject) so that they have some skin in the game.

foobarbecue 6y ago
Hunh, this is a great idea. Rejected articles should be published as rejected, and all reviews should be published. I guess this is a bit like the open review model, like The Cryosphere uses, except I think they hide papers after the discussion period if they aren't accepted.
michaelmior 6y ago
I think the problem with this is that it can encourage retaliation. If I know you rejected my paper, I might be incline to reject yours. Now, a nobody like me probably can't get away with doing that to someone with an established reputation. But the other way around is certainly possible.

AcerbicZero 6y ago

Can’t we jus my have a GitHub for science and get this over with? It seems silly to maintain this system that may have been functional when it started, but is now clearly behind the technology curve.

michaelmior 6y ago
How would that address the core question being asked which is whether work spent reviewing should be paid? In any case, while there are problems with peer review, I don't think it's related to technology.

TheOtherHobbes 6y ago

I don't think the process would be less effective if papers were published without peer review, but were then questioned and challenged in public debate under editorial moderation. Peer review isn't a single-goal process. It exists to introduce citations of existing work if they aren't already referenced, to question assumptions, and to find obvious errors. (Among others.) All of these could be done in peer debate. It's not as if the process is genuinely anonymous anyway. Of course it pretends to be, but it isn't. Peer review dates from the days when information travelled as fast as a horse, and it made sense to include a review stage for basic quality control. I'm not convinced it's a good model now, when information exchange is rather quicker. The real problem is that journal publication is about career status, not about quality of work. Peer review is almost a side issue. The scientific publishing industry is not just financially parasitic, it also introduces unnecessary delays in idea exchange. Public peer debate would eliminate some of that friction, but it would also have public consequences for reputation - and that might well be more effective for quality control than pseudo-anonymity.

michaelmior 6y ago
At least in my field, I expect this would be a total disaster. One of the major advantages of peer review is that knowing the top tier publication venues, I can look to these to keep up to date on the latest work knowing that what I find is likely to be high quality. In the absence of peer-reviewed publication venues, I see two options: 1. Read everything I can find that is published and try to sort through it all myself to sort out what is good quality. I think in aggregate, this is likely to take more of researchers' time than contributing to the peer review process. 2. Focus only on publications from top quality institutions or researchers whose work I am already familiar with. This has the advantage of saving me time. But it also has the massive disadvantage that I am likely to miss quality work which comes from lesser known institutions or authors. I don't think either of those two options is attractive. In my field, the delay of idea exchange is not really a problem. If people want to share their ideas, they can often publish a preprint in parallel with submission to a peer-reviewed journal. (Although some journals effectively prohibit this since it has the potential to break the anonymity of the review process.) Yes, some of the purposes of peer review could be accomplished in the method of "peer debate" you mention. But I think such a process would be much less effective. Of course, that's not to say I don't think there are things broken about peer review that could use fixing :)

jltsiren 6y ago

Like many other recent international hires at US universities, I'm on H-1B visa. If paying for reviews was the norm, I believe it would be illegal for me to review papers, even for free. I'm not sure what to think about this. Peer review is an important way of contributing to the community, and every academic is expected to participate in it. Commercial publishers that make profits from free academic labor are parasites, and they should pay for the services they use. Yet immigration rules often make it difficult to earn any side income, so paying for reviews would exclude many (often less established) academics.

michaelmior 6y ago
That is a good point and one I hadn't considered despite being an H-1B holder myself. I agree with your argument that paying for reviews under current immigration laws would be highly problematic for visa holders. It would either mean we would be expected to do it for free when everyone else gets paid, which would be unfair and potentially still not legal. Or we would be unable to review which I think is an important part of establishing oneself within the community.

taylodl 6y ago

Is there a way this can be made double-blind, where the Journal doesn't know the reviewer and the reviewer doesn't know the Journal? Can something akin to Mechanical Turk be utilized? Just trying to think of ways to have paid reviews without incurring all the negatives so typically associated with that arrangement.