4 ms·

Other than your opinion, do you have any arguments to offer? Otherwise I can just tell you my opinion as someone who works in academia: It makes perfect sense,

by 13415 8y ago

Other than your opinion, do you have any arguments to offer? Otherwise I can just tell you my opinion as someone who works in academia: It makes perfect sense, and so far I haven't met anyone who was able to offer any better alternative to peer reviewing. That being said, there are some reasons why there are more shady papers and journals today than there used to be. The #1 reason is that many policy makers and science administrators got this unhealthy obsession with merely counting indicators, which leads young researchers to chase publication after publication regardless of the quality. The #2 reason is that there are way more researchers competing for tenure, but jobs have barely increased and public spending on education and universities are overall low in comparison to other areas. You get what you pay for. So there you go, another opinion, but at least a qualified one.

shard972 8y ago
> I haven't met anyone who was able to offer any better alternative to peer reviewing. The fact nobody has ever suggested public-review on-top of peer review makes me think academia really doesn't think about this issue much at all.
- detaro 8y ago
  > The fact nobody has ever suggested public-review How do you define "public-review" for that to be a fact?
- newen 8y ago
  Yeah that's just what I need. Aside from being reviewed by people with a passing familiarity with what I am doing, I would like every certified internet genius who has never heard of my field to comment on my unpublished work.
  - thrower123 8y ago
    If your work can't hold up against the assaults of a "certified internet genius" then it doesn't warrant serious consideration.
    - your-nanny 8y ago
      it is not whether it can hold up against an onslaught of cranks, trolls, and well meaning idiots, but whether a researcher should be expected to have to defend their hard work against them in order to get published. Bullshit can always be cranked put faster than you could ever address. Or do you think that everytime an astronomer tries to publish a paper she should have to address the very serious concerns of flat-earthrs and young-earth creationists???
    - woodson 8y ago
      The point isn't that, I think. It's that valuable reviews are easily downed out by YouTube-comment-quality reviews. Schemes like curation of reviews by up-/downvoting don't work for most papers, as they are of interest to only a relatively small number of people engaged in that particular niche of scientific inquiry (which, by the way, is kind of the nature of things as a field specializes more and more). The concern is that there wouldn't be enough "genuine" people to counteract the noise.
      - thrower123 8y ago
        I'd wager that the whole concept has staled. In 1800, when you could have the smartest polymaths in the world look at your paper, that stamp of approval meant something.
        woodson 8y ago
        I never looked at the peer review process in that way ("stamp of approval"). For a journal article, as long as it's not a desk-rejection (because the study is not in scope of the journal or such), it's usually an iterative process involving the authors, the reviewers, and the editor, which results in an often greatly improved article. A key person in that process is often the editor, rather than the reviewers (if they do their job well). Peer review as such does not provide much of a signal as to how important a paper is or a binary "good paper"/"bad paper" decision; ideally, it's a process that helps maintain a certain standard. Conference proceedings typically don't use this process, but (more or less) ask for a simple yes/no (or a 1-5 rating where everyone knows that it's going to be rejected if you don't give a high enough rating because of a 30% acceptance rate etc.).
- skeptic_69 8y ago
  > Well, maybe because it's all done by a priest-caste cartel, shielded from reality in ivory towers, and then presented to the public in a similar way religion once was forced on peasants (taxes included). Academia is the modern priesthood: usurping the right to all knowledge, in bed with the state, corrupted to the bones, fighting heretics. I don't mean to be rude....I actually fuck that. I mean to be rude. You are an irrefutable argument against public-review.
- cbkeller 8y ago
  Geologist here. Interactive public review plus transparent peer review has become standard on the journals of the European Geophysical Union [1]. These are also open-access. [1] https://www.egu.eu/publications/open-access-journals/ https://www.egu.eu/publications/open-access-journals/
- claudius 8y ago
  Check out scipost.org. Two-way open access, community-run and public peer reviews.
- jonathanstrange 8y ago
  > The fact nobody has ever suggested public-review on-top of peer review makes me think academia really doesn't think about this issue much at all. There are not only such suggestions, there is even a whole research discipline solely dedicated to evaluating peer reviewing, and there is a metric ton of papers about the topic written by disciples from many different areas.
manfredo 8y ago
I think the issue isn't with peer review. The issue is peer review coupled with barriers to entry. Some of it is monetary, which could conceivably be resolved through open access. Some of it is because it is socially walled off, this can be resolved through outreach and expansion into different media. Consider how much more approachable a DEF CON talk is as compared to a research paper. What if we promulgated biological research in a similar fashion. Chubbyemu [1] explains diseases in a very approachable fashion. Perhaps we could do more do make research and academia approachable for the average person? TL;DR: I think the issue is the need to demystify academia, peer review is only a part of that. 1. https://www.youtube.com/channel/UCKOvOaJv4GK-oDqx-sj7VVg https://www.youtube.com/channel/UCKOvOaJv4GK-oDqx-sj7VVg
dpc_pw 8y ago
I enjoyed your comment. I agree with your reasons, though I don't think they are complete. I am not an expert on academia and I don't claim to be, so I have no solutions. I can just see the parallels and the general mechanisms. I think Nassim Taleb does the best jobs at explaining the general mechanism of the problem: https://medium.com/incerto/an-expert-called-lindy-fdb30f146eaf https://medium.com/incerto/an-expert-called-lindy-fdb30f146e...

by 13415 8y ago

shard972 8y ago

> I haven't met anyone who was able to offer any better alternative to peer reviewing. The fact nobody has ever suggested public-review on-top of peer review makes me think academia really doesn't think about this issue much at all.

detaro 8y ago
> The fact nobody has ever suggested public-review How do you define "public-review" for that to be a fact?
newen 8y ago
Yeah that's just what I need. Aside from being reviewed by people with a passing familiarity with what I am doing, I would like every certified internet genius who has never heard of my field to comment on my unpublished work.
- thrower123 8y ago
  If your work can't hold up against the assaults of a "certified internet genius" then it doesn't warrant serious consideration.
  - your-nanny 8y ago
    it is not whether it can hold up against an onslaught of cranks, trolls, and well meaning idiots, but whether a researcher should be expected to have to defend their hard work against them in order to get published. Bullshit can always be cranked put faster than you could ever address. Or do you think that everytime an astronomer tries to publish a paper she should have to address the very serious concerns of flat-earthrs and young-earth creationists???
  - woodson 8y ago
    The point isn't that, I think. It's that valuable reviews are easily downed out by YouTube-comment-quality reviews. Schemes like curation of reviews by up-/downvoting don't work for most papers, as they are of interest to only a relatively small number of people engaged in that particular niche of scientific inquiry (which, by the way, is kind of the nature of things as a field specializes more and more). The concern is that there wouldn't be enough "genuine" people to counteract the noise.
    - thrower123 8y ago
      I'd wager that the whole concept has staled. In 1800, when you could have the smartest polymaths in the world look at your paper, that stamp of approval meant something.
      - woodson 8y ago
        I never looked at the peer review process in that way ("stamp of approval"). For a journal article, as long as it's not a desk-rejection (because the study is not in scope of the journal or such), it's usually an iterative process involving the authors, the reviewers, and the editor, which results in an often greatly improved article. A key person in that process is often the editor, rather than the reviewers (if they do their job well). Peer review as such does not provide much of a signal as to how important a paper is or a binary "good paper"/"bad paper" decision; ideally, it's a process that helps maintain a certain standard. Conference proceedings typically don't use this process, but (more or less) ask for a simple yes/no (or a 1-5 rating where everyone knows that it's going to be rejected if you don't give a high enough rating because of a 30% acceptance rate etc.).
skeptic_69 8y ago
> Well, maybe because it's all done by a priest-caste cartel, shielded from reality in ivory towers, and then presented to the public in a similar way religion once was forced on peasants (taxes included). Academia is the modern priesthood: usurping the right to all knowledge, in bed with the state, corrupted to the bones, fighting heretics. I don't mean to be rude....I actually fuck that. I mean to be rude. You are an irrefutable argument against public-review.
cbkeller 8y ago
Geologist here. Interactive public review plus transparent peer review has become standard on the journals of the European Geophysical Union [1]. These are also open-access. [1] https://www.egu.eu/publications/open-access-journals/ https://www.egu.eu/publications/open-access-journals/
claudius 8y ago
Check out scipost.org. Two-way open access, community-run and public peer reviews.
jonathanstrange 8y ago
> The fact nobody has ever suggested public-review on-top of peer review makes me think academia really doesn't think about this issue much at all. There are not only such suggestions, there is even a whole research discipline solely dedicated to evaluating peer reviewing, and there is a metric ton of papers about the topic written by disciples from many different areas.

manfredo 8y ago

I think the issue isn't with peer review. The issue is peer review coupled with barriers to entry. Some of it is monetary, which could conceivably be resolved through open access. Some of it is because it is socially walled off, this can be resolved through outreach and expansion into different media. Consider how much more approachable a DEF CON talk is as compared to a research paper. What if we promulgated biological research in a similar fashion. Chubbyemu [1] explains diseases in a very approachable fashion. Perhaps we could do more do make research and academia approachable for the average person? TL;DR: I think the issue is the need to demystify academia, peer review is only a part of that. 1. https://www.youtube.com/channel/UCKOvOaJv4GK-oDqx-sj7VVg https://www.youtube.com/channel/UCKOvOaJv4GK-oDqx-sj7VVg

dpc_pw 8y ago

I enjoyed your comment. I agree with your reasons, though I don't think they are complete. I am not an expert on academia and I don't claim to be, so I have no solutions. I can just see the parallels and the general mechanisms. I think Nassim Taleb does the best jobs at explaining the general mechanism of the problem: https://medium.com/incerto/an-expert-called-lindy-fdb30f146eaf https://medium.com/incerto/an-expert-called-lindy-fdb30f146e...