7 ms·

I assume hep = high energy physics in this context. PI = professor who received a government grant. Peer review has never really been blind and I suspect PIs w

by wmf 8mo ago

I assume hep = high energy physics in this context. PI = professor who received a government grant. Peer review has never really been blind and I suspect PIs will reject papers from "outsiders" even if they are higher quality. This already happens to some extent today when the stakes are lower.

selridge 8mo ago
Kinda. PI is principal investigator and usually they’re a professor with a grant (the grant being the thing they are the principal of investigating). That part is right. But they’re not really directly in the review loop. For some fields where things are small enough that folks can recognize style such as it exists, you could see reviewers passing over unfamiliar work and promoting familiar work. That was not the issue. The issue was that it still was kind of hard to produce crappy mid rate papers, so you kind of needed the infrastructure of a small lab to do that. Now you don’t. The success rate for those mediocre papers produced by grad students and postdocs will go way down. It is possible that will cease to be a useful signal for those early career researchers.
moregrist 8mo ago
Peer review isn’t the issue here. His comments are about Arxiv, which is a preprint server. Essentially anyone can publish a preprint. There’s no peer or other review involved.
- xamuel 8mo ago
  This is a common misconception. People without academic affiliation (based on their email address) require someone to vouch for them before they can submit to arxiv. And papers submitted to arxiv (with or without affiliation) are reviewed, and many are rejected.
  - bmacho 8mo ago
    Papers on arxiv are only reviewed for formal requirements. They don't review every pdf there, and reject them for being false or wrong. You are right that arxiv is an invite-only website, but once you are in, there is no peer review of any form.
    - deleted 8mo ago
      [deleted]
  - Aurornis 8mo ago
    arXiv does not review everything pushed to the site. It's very easy to get in. It's becoming a common target for grifters who will "publish" papers on arXiv because it looks formal to those who don't know any better.
    - deleted 8mo ago
      [deleted]
MarkusQ 8mo ago
But peer review (circa 1965-2010[1]) is just the prior iteration of the problem[2]; the wave of crap[3] produced by publish or perish (crica 1950-present[4]). Rejecting papers by outsiders is irrelevant; the problem is we want to determine which papers are good/interesting/worth considering out of the fire hose of bilge, and, though we were already arguably failing at this, the problem just got harder. (I say arguably, because there is always the old "try it yourself and see if it actually works" trick, but nobody seems to be fond of this; it smacks of "do your own research" and we're lazy monkeys at heart, who would much rather copy off of someone else's homework.) [1] https://books.google.com/ngrams/graph?content=peer+review&year_start=1800&year_end=2022&corpus=en&smoothing=3 https://books.google.com/ngrams/graph?content=peer+review&ye... [2] https://www.experimental-history.com/p/the-rise-and-fall-of-peer-review https://www.experimental-history.com/p/the-rise-and-fall-of-... [3] https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 https://journals.plos.org/plosmedicine/article?id=10.1371/jo... [4] https://books.google.com/ngrams/graph?content=publish+or+perish&year_start=1800&year_end=2022&corpus=en&smoothing=3 https://books.google.com/ngrams/graph?content=publish+or+per...
xamuel 8mo ago
>Peer review has never really been blind and I suspect PIs will reject papers from "outsiders" even if they are higher quality. I'm a complete outsider (not even in academia at all) and just got a paper accepted in the top math biology journal [1]. But granted, it took literally years to write it up and get it through. I do really worry that without academic affiliation it is going to get harder and harder for outsiders as gates are necessarily kept more and more securely because of all the slop. [1] "Specieslike clusters based on identical ancestor points" https://philpapers.org/archive/ALESCB.pdf https://philpapers.org/archive/ALESCB.pdf
- mmooss 8mo ago
  Congratulations. How did you do it? And how do you get access to resources such as reference products, journals, etc. that typically require institutions with budgets?
  - xamuel 8mo ago
    Sorry for not seeing your message until now. Journals, at least in mathematics, generally don't require you to have a university affiliation, so as long as the paper is good on its merits, you can get it in, though I don't know to what extent it might be more of an uphill battle due to implicit peer reviewer bias etc. Re: reference products: one generally scraps what one can together through a combination of arxiv.org, Anna's archive, or emailing the authors.
    - mmooss 8mo ago
      Thanks for responding! Have you ever tried to get access to for-pay reference sources such as JSTOR? I haven't found a way other than going in-person to libraries, which is not practical.
      - xamuel 8mo ago
        On rare occasions where the other methods don't yield anything, I have friends in academia who have access and who are happy to help out. But yeah, it's totally medieval that this knowledge isn't freely available to the world.

by wmf 8mo ago

selridge 8mo ago

Kinda. PI is principal investigator and usually they’re a professor with a grant (the grant being the thing they are the principal of investigating). That part is right. But they’re not really directly in the review loop. For some fields where things are small enough that folks can recognize style such as it exists, you could see reviewers passing over unfamiliar work and promoting familiar work. That was not the issue. The issue was that it still was kind of hard to produce crappy mid rate papers, so you kind of needed the infrastructure of a small lab to do that. Now you don’t. The success rate for those mediocre papers produced by grad students and postdocs will go way down. It is possible that will cease to be a useful signal for those early career researchers.

moregrist 8mo ago

Peer review isn’t the issue here. His comments are about Arxiv, which is a preprint server. Essentially anyone can publish a preprint. There’s no peer or other review involved.

xamuel 8mo ago
This is a common misconception. People without academic affiliation (based on their email address) require someone to vouch for them before they can submit to arxiv. And papers submitted to arxiv (with or without affiliation) are reviewed, and many are rejected.
- bmacho 8mo ago
  Papers on arxiv are only reviewed for formal requirements. They don't review every pdf there, and reject them for being false or wrong. You are right that arxiv is an invite-only website, but once you are in, there is no peer review of any form.
  - deleted 8mo ago
    [deleted]
- Aurornis 8mo ago
  arXiv does not review everything pushed to the site. It's very easy to get in. It's becoming a common target for grifters who will "publish" papers on arXiv because it looks formal to those who don't know any better.
  - deleted 8mo ago
    [deleted]

MarkusQ 8mo ago

But peer review (circa 1965-2010[1]) is just the prior iteration of the problem[2]; the wave of crap[3] produced by publish or perish (crica 1950-present[4]). Rejecting papers by outsiders is irrelevant; the problem is we want to determine which papers are good/interesting/worth considering out of the fire hose of bilge, and, though we were already arguably failing at this, the problem just got harder. (I say arguably, because there is always the old "try it yourself and see if it actually works" trick, but nobody seems to be fond of this; it smacks of "do your own research" and we're lazy monkeys at heart, who would much rather copy off of someone else's homework.) [1] https://books.google.com/ngrams/graph?content=peer+review&year_start=1800&year_end=2022&corpus=en&smoothing=3 https://books.google.com/ngrams/graph?content=peer+review&ye... [2] https://www.experimental-history.com/p/the-rise-and-fall-of-peer-review https://www.experimental-history.com/p/the-rise-and-fall-of-... [3] https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 https://journals.plos.org/plosmedicine/article?id=10.1371/jo... [4] https://books.google.com/ngrams/graph?content=publish+or+perish&year_start=1800&year_end=2022&corpus=en&smoothing=3 https://books.google.com/ngrams/graph?content=publish+or+per...

xamuel 8mo ago

>Peer review has never really been blind and I suspect PIs will reject papers from "outsiders" even if they are higher quality. I'm a complete outsider (not even in academia at all) and just got a paper accepted in the top math biology journal [1]. But granted, it took literally years to write it up and get it through. I do really worry that without academic affiliation it is going to get harder and harder for outsiders as gates are necessarily kept more and more securely because of all the slop. [1] "Specieslike clusters based on identical ancestor points" https://philpapers.org/archive/ALESCB.pdf https://philpapers.org/archive/ALESCB.pdf

mmooss 8mo ago
Congratulations. How did you do it? And how do you get access to resources such as reference products, journals, etc. that typically require institutions with budgets?
- xamuel 8mo ago
  Sorry for not seeing your message until now. Journals, at least in mathematics, generally don't require you to have a university affiliation, so as long as the paper is good on its merits, you can get it in, though I don't know to what extent it might be more of an uphill battle due to implicit peer reviewer bias etc. Re: reference products: one generally scraps what one can together through a combination of arxiv.org, Anna's archive, or emailing the authors.
  - mmooss 8mo ago
    Thanks for responding! Have you ever tried to get access to for-pay reference sources such as JSTOR? I haven't found a way other than going in-person to libraries, which is not practical.
    - xamuel 8mo ago
      On rare occasions where the other methods don't yield anything, I have friends in academia who have access and who are happy to help out. But yeah, it's totally medieval that this knowledge isn't freely available to the world.