6 ms·

So many of these discussions take it for granted that publishers don't do anything of value. This blog post, by an open access advocate, was for me very illumin

by czr80 11y ago

So many of these discussions take it for granted that publishers don't do anything of value. This blog post, by an open access advocate, was for me very illuminating on what value they actually add (http://cameronneylon.net/blog/polecon-of-oa-publishing-i-what-is-it-publishers-do-anyway/ http://cameronneylon.net/blog/polecon-of-oa-publishing-i-wha...) Quoting from that link: "One of the big challenges is discussing the costs and value added in managing peer review is that researchers who engage in this conversation tend to be amongst the best editors and referees. Professional publishers on the other hand tend to focus on the (relatively small number of) contributors, who are, not to put too fine a point on it, awful. Good academic editors tend to select good referees who do good work, and when they encounter a bad referee they discount it and move on. Professional staff spend the majority of their time dealing with editors who have gone AWOL, referees who are late in responding, or who turn out to be inappropriate either in what they have written or their conflicts of interest, or increasingly who don’t even exist! .... Much of the irritation you see from publishers when talking about why managing peer review is more than “sending a few emails” relates to this gap in perception. The irony is that the problems are largely invisible to the broader community because publishers keep them under wraps, hidden away so that they don’t bother the community." I'm sure there must be some way to achieve this and also make the content freely available. But whatever system replaces the current one has to deal with these issues too.

return0 11y ago
> I'm sure there must be some way to achieve this and also make the content freely available. But whatever system replaces the current one has to deal with these issues too. Post-publication peer review could solve all of these (and the access issue too!). I'm still not convinced of the superiority of pre-publication review by only 3 (usually quite busy) people max.
- KKKKkkkk1 11y ago
  I don't understand. What is the difference between post-publication peer review and no peer review? What is a journal for if the papers are not peer reviewed?
  - return0 11y ago
    It's not a journal, or it's a different kind of journal. I guess it would be like doing the whole review process in a very public way. Would that be worse? I don't know ... has anyone tried it? I think we are heading that way anyway, the open access publishers are pushing the envelope here by making peer review more and more public (publishing reviewer names, or comments along with the paper).
  - kriro 11y ago
    The simplest workflow I can think of: 1) Publish with no review 2) Open for review 3) If review is passed, tag as reviewed and bundle all those into a special reviewed section (which can be cited) 4) Reserve the right to remove from reviewed (some later reviewer finds major flaws etc.) You'd have to handle the dynamic nature of papers (git-like) but in essence I'm thinking of a website with "all submitted papers" and a checkbox for "reviewed papers only". I think that would be valuable and people would likely leave the checkbox on by default. I'm also curious if the review has to be blind in this model. I think it would be more valuable if the reviewers actually signed with their names and the review feedback was open as well. Shifts the value from being in a prestigious journal to being successfully reviewed by a smart/reputable person. Ideally doing these reviews would then become reputation building as well.
- dalke 11y ago
  I haven't had good experience with modern post-publication peer review so far. My biggest complaint is that few know those comments exist. For example, I commented on one paper to highlight methodological problems that make it impossible to trust their results. (They developed a new algorithm for X. They compared it to the naive implementation. The naive implementation was poorly coded. Most of the speedup disappears by in comparison to a well-written implementation.) I've talked to a few people about this exact paper. None noticed the link to the comment page. I say "modern" because post-publication peer review isn't new. One of the older mechanisms was the letter to the editor. Those letters (at least in the ACS journal I'm thinking of) have a DOI and are searchable. A few of these letters have proved useful to my research. But the modern post-publication peer reviews don't have a DOI and aren't indexed by Google Scholar or other systems, so they are less useful than a old Letter to the Editor. (I once asked about sending a Letter to the Editor to an Open Access journal, concerning problems in one of their publications. They said they don't support those sorts of short communications, and my only option was the $1,500 to publish a full paper, which would have to go through the normal peer review process.)
- visarga 11y ago
  You mean Elsevier doesn't need to earn $2 billion for hustling free editors to do their work proper? You know, it's more than just pushing emails around. You have to deal with people.
  - return0 11y ago
    Have you accounted the cost of the many work-hours that reviewers spend beautifying their journals? it's probably in the tens of billions, worldwide. Is it justifiable? It's certainly not what the public is paying for.
    - maaku 11y ago
      I think you missed an Office Space reference.
- Fomite 11y ago
  I have been extremely unimpressed with my experiences post-publication peer-review. Limited to no feedback for many papers, and some clearly "How dare you stray onto my turf" related comments.
notalaser 11y ago
Just saying -- perhaps "editors who have gone AWOL, referees who are late in responding, or who turn out to be inappropriate" would likely be less of a problem if they were properly paid for what they're doing. I haven't been in academia for a long time (and my stint was short), but I don't remember ever hearing of a reviewer who got anything more substantial than maybe a discount or a one month of free subscription to a journal from a publisher. Managing peer review is admittedly more than "sending a few e-mails", but not in a good way.
- dalke 11y ago
  What is "properly paid"? The last review I did (for an open access journal) took ~6 hours. At minimum wage that's about $50. There were two reviewers = $100. Reviewer must get paid even if the recommendation is "don't publish", otherwise reviewers will have an incentive to say "yes". The OA fee for that journal is about $1300. If 50% of the peer reviewed papers are published, then the average price for peer review would be $200, which would raise the price to publish by about 10%. It is an open access journal, so everyone effectively has a free subscription to it. One alternative is a credit model, where N reviews gives a discount on the price to publish. However, as the above points out, it may be more cost effective to mow lawns on the weekend than it is to plan for that discount. Then there's the question of perceived fairness. If one reviewer says "great job. Publish" and another gives 10 pages of useful critique, is it proper to pay each the same?
  - notalaser 11y ago
    My information may be out of date because, as I've said above, my dance with academia ended a long time ago. AFAIK most (all?) of Elsevier's journals aren't open access and reviewers aren't paid at all. They aren't improperly paid, they're not paid at all. The publisher gets a hefty amount of income out of their unpaid work, of which they share exactly nothing. This is well past the level where we can debate nuances of fairness. I don't know enough about this field to be able to properly debate the proper mechanics of payment. I just wanted to point out that one of the pillars of these publishers' income -- the credibility brought by peer review, such as it is -- is effectively the unpaid labour of a lot of very, very clever people, who contribute enormously to human progress (not just to the publishers' pockets) pretty much for free (sure, some of them are also assholes, but that's besides the point). I can understand why sometimes they'd be less then collaborative. Over here, in the industry, if I were offered the (industrial) equivalent of that deal, my answer would be a warm and heartfelt fuck you; in fact, I don't think any serious company would want to compromise its image by offering such a deal. Edit: at the risk of appearing to suffer of the anti-academic sentiment that is so plaguing our profession (which, I have to insist, I do not), I also think that some of this problem is self-inflicted and has to do with the way in which modern society treats higher education. I sometimes think that part of the solution could lay in discouraging the short-sighted, what-can-we-reliably-solve-in-no-longer-than-three-years, paper-focused approach to research activity that is self-feeding the publishing machine.
maaku 11y ago
People would be more sympathetic if peer reviewers were actually paid for what they do.

by czr80 11y ago

return0 11y ago

> I'm sure there must be some way to achieve this and also make the content freely available. But whatever system replaces the current one has to deal with these issues too. Post-publication peer review could solve all of these (and the access issue too!). I'm still not convinced of the superiority of pre-publication review by only 3 (usually quite busy) people max.

KKKKkkkk1 11y ago
I don't understand. What is the difference between post-publication peer review and no peer review? What is a journal for if the papers are not peer reviewed?
- return0 11y ago
  It's not a journal, or it's a different kind of journal. I guess it would be like doing the whole review process in a very public way. Would that be worse? I don't know ... has anyone tried it? I think we are heading that way anyway, the open access publishers are pushing the envelope here by making peer review more and more public (publishing reviewer names, or comments along with the paper).
- kriro 11y ago
  The simplest workflow I can think of: 1) Publish with no review 2) Open for review 3) If review is passed, tag as reviewed and bundle all those into a special reviewed section (which can be cited) 4) Reserve the right to remove from reviewed (some later reviewer finds major flaws etc.) You'd have to handle the dynamic nature of papers (git-like) but in essence I'm thinking of a website with "all submitted papers" and a checkbox for "reviewed papers only". I think that would be valuable and people would likely leave the checkbox on by default. I'm also curious if the review has to be blind in this model. I think it would be more valuable if the reviewers actually signed with their names and the review feedback was open as well. Shifts the value from being in a prestigious journal to being successfully reviewed by a smart/reputable person. Ideally doing these reviews would then become reputation building as well.
dalke 11y ago
I haven't had good experience with modern post-publication peer review so far. My biggest complaint is that few know those comments exist. For example, I commented on one paper to highlight methodological problems that make it impossible to trust their results. (They developed a new algorithm for X. They compared it to the naive implementation. The naive implementation was poorly coded. Most of the speedup disappears by in comparison to a well-written implementation.) I've talked to a few people about this exact paper. None noticed the link to the comment page. I say "modern" because post-publication peer review isn't new. One of the older mechanisms was the letter to the editor. Those letters (at least in the ACS journal I'm thinking of) have a DOI and are searchable. A few of these letters have proved useful to my research. But the modern post-publication peer reviews don't have a DOI and aren't indexed by Google Scholar or other systems, so they are less useful than a old Letter to the Editor. (I once asked about sending a Letter to the Editor to an Open Access journal, concerning problems in one of their publications. They said they don't support those sorts of short communications, and my only option was the $1,500 to publish a full paper, which would have to go through the normal peer review process.)
visarga 11y ago
You mean Elsevier doesn't need to earn $2 billion for hustling free editors to do their work proper? You know, it's more than just pushing emails around. You have to deal with people.
- return0 11y ago
  Have you accounted the cost of the many work-hours that reviewers spend beautifying their journals? it's probably in the tens of billions, worldwide. Is it justifiable? It's certainly not what the public is paying for.
  - maaku 11y ago
    I think you missed an Office Space reference.
Fomite 11y ago
I have been extremely unimpressed with my experiences post-publication peer-review. Limited to no feedback for many papers, and some clearly "How dare you stray onto my turf" related comments.

notalaser 11y ago

Just saying -- perhaps "editors who have gone AWOL, referees who are late in responding, or who turn out to be inappropriate" would likely be less of a problem if they were properly paid for what they're doing. I haven't been in academia for a long time (and my stint was short), but I don't remember ever hearing of a reviewer who got anything more substantial than maybe a discount or a one month of free subscription to a journal from a publisher. Managing peer review is admittedly more than "sending a few e-mails", but not in a good way.

dalke 11y ago
What is "properly paid"? The last review I did (for an open access journal) took ~6 hours. At minimum wage that's about $50. There were two reviewers = $100. Reviewer must get paid even if the recommendation is "don't publish", otherwise reviewers will have an incentive to say "yes". The OA fee for that journal is about $1300. If 50% of the peer reviewed papers are published, then the average price for peer review would be $200, which would raise the price to publish by about 10%. It is an open access journal, so everyone effectively has a free subscription to it. One alternative is a credit model, where N reviews gives a discount on the price to publish. However, as the above points out, it may be more cost effective to mow lawns on the weekend than it is to plan for that discount. Then there's the question of perceived fairness. If one reviewer says "great job. Publish" and another gives 10 pages of useful critique, is it proper to pay each the same?
- notalaser 11y ago
  My information may be out of date because, as I've said above, my dance with academia ended a long time ago. AFAIK most (all?) of Elsevier's journals aren't open access and reviewers aren't paid at all. They aren't improperly paid, they're not paid at all. The publisher gets a hefty amount of income out of their unpaid work, of which they share exactly nothing. This is well past the level where we can debate nuances of fairness. I don't know enough about this field to be able to properly debate the proper mechanics of payment. I just wanted to point out that one of the pillars of these publishers' income -- the credibility brought by peer review, such as it is -- is effectively the unpaid labour of a lot of very, very clever people, who contribute enormously to human progress (not just to the publishers' pockets) pretty much for free (sure, some of them are also assholes, but that's besides the point). I can understand why sometimes they'd be less then collaborative. Over here, in the industry, if I were offered the (industrial) equivalent of that deal, my answer would be a warm and heartfelt fuck you; in fact, I don't think any serious company would want to compromise its image by offering such a deal. Edit: at the risk of appearing to suffer of the anti-academic sentiment that is so plaguing our profession (which, I have to insist, I do not), I also think that some of this problem is self-inflicted and has to do with the way in which modern society treats higher education. I sometimes think that part of the solution could lay in discouraging the short-sighted, what-can-we-reliably-solve-in-no-longer-than-three-years, paper-focused approach to research activity that is self-feeding the publishing machine.

maaku 11y ago

People would be more sympathetic if peer reviewers were actually paid for what they do.