4 ms·

> In a paper that hasn’t yet been peer-reviewed, he and his colleagues predict the existence of yet another kind of magnetism, which he calls antialtermagnetism

> In a paper that hasn’t yet been peer-reviewed, he and his colleagues predict the existence of yet another kind of magnetism, which he calls antialtermagnetism. Can we stop referring to ArXiv papers third way? And for the love of God, just link the fucking abstract, never NEVER link the html![0] You just change {html,pdf} -> abs We shouldn't say "not peer reviewed" because it isn't accurate. Being published in a journal doesn't mean a work is correct nor does it mean peers read it. Putting the paper on ArXiv does mean peers are reviewing it. The point of publishing is to communicate our work to others and journals and conferences can often be harmful to that process, making researchers oversell or even avoid looking in certain directions because a few opinionated peers shoot them down. It's happened to Nobel level works too. The review process is just fucked up. It might be able to tell you if a paper is wrong but it can't tell you if it has no mistakes or is right. I mean it took two years to confirm this one, right? (Physical validation) but the way we say "hasn't been peer reviewed" implies that if it has been published in a journal then it's factual. That's not how it works and frankly that's not how it should work. On top of that they take money from the government, gets articles for free, don't pay reviewers (meaning the universities pay for reviewers), and have the audacity to charge people to read that work. It's basically just a scheme to extract government money. Sorry, I really just hate the publication process. It stifles innovation and wastes so many people's time [0] https://arxiv.org/abs/2309.01607 https://arxiv.org/abs/2309.01607

gus_massa 1y ago
Nitpicking: > Putting the paper on ArXiv does mean peers are reviewing it. You can upload a pdf to the ArXiv and not send it to any journal. I think you need an invitation to create an account, but it's 99% like WordPress.
- mindcrime 1y ago
  > I think you need an invitation to create an account, Anybody can create an account on arXiv.[1][2] However, having an account and uploading are different things. Strictly speaking, you have to be endorsed to submit a paper. But some endorsements are granted automatically, based on various criteria. So not everybody has to go explicitly ask people for an endorsement: arXiv requires that users be endorsed before submitting their first paper to a category. arXiv may give some people automatic endorsements based on subject area, topic, previous submissions, and academic affiliation. In most cases, automatic endorsement is given to authors from known academic institutions and research facilities. Generally speaking, the "word on the street" is that it's as simple as "register with a .edu email address and you get auto endorsed, otherwise you don't." But I think the reality is slightly more complex than that. Although the exact details are kept private, probably at least in part to prevent people from gaming the system. Note that the page encourages you to register with an "institutional" email address, but doesn't specifically say it has to be a .edu one. arXiv submitters are therefore encouraged to associate an institutional email address, if they have one, with their arXiv account. This will expedite the endorsement process. [1]: https://info.arxiv.org/help/registerhelp.html https://info.arxiv.org/help/registerhelp.html [2]: https://arxiv.org/user/register https://arxiv.org/user/register
- godelski 1y ago
  I think you're right to nitpick, but nitpicking the wrong thing. > and not send it to any journal. This, doesn't matter. The main point of my comment is "journal != peers reading the work". Which, of course, you could even say is true about ArXiv (fine to nitpick[0]). You can put on ArXiv and no one will read it or the only people who read it are not peers. You're right that submitting to a journal or conference (nearly) guarantees someone has read the paper. The thing though is "peer" is weakly defined. I recognize that it is contextually defined, but I'd hope from the context of my comment you can tell that I'm using "peer" to mean "another person well versed in the niche topic of the paper." This is different from "a person well versed in the topic of the paper." The niche matters. I explain more in this comment[1] Or to put it differently, just because someone read it doesn't mean they /read/ it. [0] I'll revise the quoted text to "Just because it is on ArXiv doesn't mean peers aren't or haven't reviewed it". Though we can nitpick a little more and say that if there are more than one author on the paper, a peer has been much more likely to have reviewed the work (not all co-authors review the works they are authors on...) [1] https://news.ycombinator.com/item?id=44587535 https://news.ycombinator.com/item?id=44587535
devmor 1y ago
> Being published in a journal doesn't mean a work is correct nor does it mean peers read it. Putting the paper on ArXiv does mean peers are reviewing it. Sorry I don't get what you mean by this. How does putting the paper on ArXiv mean that peers are reviewing it any more than publishing it in a journal? Both do mean that peers have the opportunity to review it, but neither guarantees it, and ArXiv is infinitely easier to upload anything to and never have it even looked at.
- xondono 1y ago
  Technically speaking, being published means at least the editors have reviewed. The quality of their reviews is another thing entirely.
  - deleted 1y ago
    [deleted]
  - godelski 1y ago
    Well... it can be desk rejected. Which I've actually had happen because the paper was already on ArXiv, even though it wasn't against the journal's policy. Took 4 weeks to resolve and then got desk rejected again for "not citing the correct works", with no further information... I don't submit to that journal anymore...
    - xondono 1y ago
      Sure it can, but then it’s not published. My only point is that if you get published in a journal, at least two people have seen it.
- tensor 1y ago
  Obviously journals vary in their standards, but many of the more respected ones do require other scientists read and critique the paper. You can argue about the quality of these reviews, certainly it's a process that needs improving, but this is what "peer reviewed" means. Trying to claim ArXiv papers are "peer reviewed" is utter nonsense. As you correctly point out, the only requirement to being on ArXiv is that someone with an account uploaded it there. There are no requirements that it passes any sort of verification or vetting process whatsoever, let alone having other scientists read and critique it. There is a very vocal movement these days that is trying to argue that we should do away with the traditional peer review process. Apparently that sometimes includes trying to redefine the very definition of "peer review" as the OP did.
xondono 1y ago
That’s because we’ve basically reinterpreted what “peer review” is. Peer review used to mean “some peers have reviewed it”, mainly the editors, who pushed for correctness and novelty. There was a clear difference between publishing and making a paper public. It never meant “it’s right”, but it meant “it has passed basic quality control and it’s worth your time to read it”. Modern day academics push people to fragment into ever smaller niches, meaning most editors are nowadays completely out of their depth when evaluating papers, so now we keep referring to editor approval as “peer review” and try to diminish the public perception that comes with it.
- olddustytrail 1y ago
  > That’s because we’ve basically reinterpreted what “peer review” is. Who is "we" in this scenario? Because that's certainly not how I've seen peer review work. The editor would ask a small group of people in the field to act as reviewers and then send them the papers. They review it and send it back with any requests for changes prior to publication. So they're the peers that are reviewing, not the editor.
  - pessimizer 1y ago
    I think they meant "reinterpreted" over the last century, not over the span of your personal experience and career.
    - olddustytrail 1y ago
      They're saying it changed to not being reviewed properly and I'm saying from recent experience that it is.
  - godelski 1y ago
    Look at the history of Peer Review. What you see post 1950 is pretty different than what you see prior to that. I think this quote is the best one-liner, though I think everyone should dig much more into the question > in the early 20th century, "the burden of proof was generally on the opponents rather than the proponents of new ideas. That is, the reviewers had a higher burden than the authors. The bias is towards acceptance rather than rejection. In a perfect world we could only accept good papers and could reject bad papers, but we don't live in that world. So the question is "when we fail, which way do we want to fail?" Obviously, I'm on the side of Blackstone here https://en.wikipedia.org/wiki/Scholarly_peer_review#History https://en.wikipedia.org/wiki/Scholarly_peer_review#History
fc417fc802 1y ago
Fully agree with you that the way much (but not all) of the publication process currently works is bad for everyone involved (including the taxpayer). But I'd like to push back against the following. > the way we say "hasn't been peer reviewed" implies that if it has been published in a journal then it's factual. I think that's ascribing meaning that isn't actually there. Peer review _does_ tell you something. Certainly not "guaranteed correct" or even "guaranteed high quality" but it is definitely some sort of signal to the reader. Consider that this is nothing more than proactively pointing out that a linked paper which closely resembles work that would typically be affiliated with a journal (a signal gated by an editor) be peer reviewed (another signal) and those reviews possibly made public (yet another signal) does not have any of these signals. That is to say, it's an important disclaimer in order to avoid the appearance of attempting to deceive the audience.
- godelski 1y ago
  I don't think you're wrong, I just don't think those signals are very strong. I do think they used to be stronger, but paper discovery isn't really dominated that way anymore. Word of mouth was always common, but now that's expanded to social media as well as things like Google scholar or semantic. Hell, most journals and conferences are hard to search through. Maybe I'll land on their page from a Google search of the paper or when looking for those topics, but that's not any different from ArXiv. And truth is that I still have to do the same thing when landing in either location. I still have to read the abstract, give it a sniff test, and read it to varying depth depending on what's needed. There's a signal, I just haven't found it to be that meaningful
  - fc417fc802 1y ago
    I never claimed it was useful for targeted searches by modern standards. It's primarily a time saving measure from my perspective. If I come across a paper and it is in a publication that has a track record of publishing shoddy results I am likely to prioritize other works (assuming that I have a large potential pool of them to investigate). If a nature paper claims a seemingly impossible result I'll most likely spend the time to at least glance through it and see if maybe my preconceptions were mistaken. If something on arxiv looks like nonsense I'm not going to waste any time on it (unless someone else told me it's worth it).

gus_massa 1y ago

Nitpicking: > Putting the paper on ArXiv does mean peers are reviewing it. You can upload a pdf to the ArXiv and not send it to any journal. I think you need an invitation to create an account, but it's 99% like WordPress.

mindcrime 1y ago
> I think you need an invitation to create an account, Anybody can create an account on arXiv.[1][2] However, having an account and uploading are different things. Strictly speaking, you have to be endorsed to submit a paper. But some endorsements are granted automatically, based on various criteria. So not everybody has to go explicitly ask people for an endorsement: arXiv requires that users be endorsed before submitting their first paper to a category. arXiv may give some people automatic endorsements based on subject area, topic, previous submissions, and academic affiliation. In most cases, automatic endorsement is given to authors from known academic institutions and research facilities. Generally speaking, the "word on the street" is that it's as simple as "register with a .edu email address and you get auto endorsed, otherwise you don't." But I think the reality is slightly more complex than that. Although the exact details are kept private, probably at least in part to prevent people from gaming the system. Note that the page encourages you to register with an "institutional" email address, but doesn't specifically say it has to be a .edu one. arXiv submitters are therefore encouraged to associate an institutional email address, if they have one, with their arXiv account. This will expedite the endorsement process. [1]: https://info.arxiv.org/help/registerhelp.html https://info.arxiv.org/help/registerhelp.html [2]: https://arxiv.org/user/register https://arxiv.org/user/register
godelski 1y ago
I think you're right to nitpick, but nitpicking the wrong thing. > and not send it to any journal. This, doesn't matter. The main point of my comment is "journal != peers reading the work". Which, of course, you could even say is true about ArXiv (fine to nitpick[0]). You can put on ArXiv and no one will read it or the only people who read it are not peers. You're right that submitting to a journal or conference (nearly) guarantees someone has read the paper. The thing though is "peer" is weakly defined. I recognize that it is contextually defined, but I'd hope from the context of my comment you can tell that I'm using "peer" to mean "another person well versed in the niche topic of the paper." This is different from "a person well versed in the topic of the paper." The niche matters. I explain more in this comment[1] Or to put it differently, just because someone read it doesn't mean they /read/ it. [0] I'll revise the quoted text to "Just because it is on ArXiv doesn't mean peers aren't or haven't reviewed it". Though we can nitpick a little more and say that if there are more than one author on the paper, a peer has been much more likely to have reviewed the work (not all co-authors review the works they are authors on...) [1] https://news.ycombinator.com/item?id=44587535 https://news.ycombinator.com/item?id=44587535

devmor 1y ago

> Being published in a journal doesn't mean a work is correct nor does it mean peers read it. Putting the paper on ArXiv does mean peers are reviewing it. Sorry I don't get what you mean by this. How does putting the paper on ArXiv mean that peers are reviewing it any more than publishing it in a journal? Both do mean that peers have the opportunity to review it, but neither guarantees it, and ArXiv is infinitely easier to upload anything to and never have it even looked at.

xondono 1y ago
Technically speaking, being published means at least the editors have reviewed. The quality of their reviews is another thing entirely.
- deleted 1y ago
  [deleted]
- godelski 1y ago
  Well... it can be desk rejected. Which I've actually had happen because the paper was already on ArXiv, even though it wasn't against the journal's policy. Took 4 weeks to resolve and then got desk rejected again for "not citing the correct works", with no further information... I don't submit to that journal anymore...
  - xondono 1y ago
    Sure it can, but then it’s not published. My only point is that if you get published in a journal, at least two people have seen it.
tensor 1y ago
Obviously journals vary in their standards, but many of the more respected ones do require other scientists read and critique the paper. You can argue about the quality of these reviews, certainly it's a process that needs improving, but this is what "peer reviewed" means. Trying to claim ArXiv papers are "peer reviewed" is utter nonsense. As you correctly point out, the only requirement to being on ArXiv is that someone with an account uploaded it there. There are no requirements that it passes any sort of verification or vetting process whatsoever, let alone having other scientists read and critique it. There is a very vocal movement these days that is trying to argue that we should do away with the traditional peer review process. Apparently that sometimes includes trying to redefine the very definition of "peer review" as the OP did.

xondono 1y ago

That’s because we’ve basically reinterpreted what “peer review” is. Peer review used to mean “some peers have reviewed it”, mainly the editors, who pushed for correctness and novelty. There was a clear difference between publishing and making a paper public. It never meant “it’s right”, but it meant “it has passed basic quality control and it’s worth your time to read it”. Modern day academics push people to fragment into ever smaller niches, meaning most editors are nowadays completely out of their depth when evaluating papers, so now we keep referring to editor approval as “peer review” and try to diminish the public perception that comes with it.

olddustytrail 1y ago
> That’s because we’ve basically reinterpreted what “peer review” is. Who is "we" in this scenario? Because that's certainly not how I've seen peer review work. The editor would ask a small group of people in the field to act as reviewers and then send them the papers. They review it and send it back with any requests for changes prior to publication. So they're the peers that are reviewing, not the editor.
- pessimizer 1y ago
  I think they meant "reinterpreted" over the last century, not over the span of your personal experience and career.
  - olddustytrail 1y ago
    They're saying it changed to not being reviewed properly and I'm saying from recent experience that it is.
- godelski 1y ago
  Look at the history of Peer Review. What you see post 1950 is pretty different than what you see prior to that. I think this quote is the best one-liner, though I think everyone should dig much more into the question > in the early 20th century, "the burden of proof was generally on the opponents rather than the proponents of new ideas. That is, the reviewers had a higher burden than the authors. The bias is towards acceptance rather than rejection. In a perfect world we could only accept good papers and could reject bad papers, but we don't live in that world. So the question is "when we fail, which way do we want to fail?" Obviously, I'm on the side of Blackstone here https://en.wikipedia.org/wiki/Scholarly_peer_review#History https://en.wikipedia.org/wiki/Scholarly_peer_review#History

fc417fc802 1y ago

Fully agree with you that the way much (but not all) of the publication process currently works is bad for everyone involved (including the taxpayer). But I'd like to push back against the following. > the way we say "hasn't been peer reviewed" implies that if it has been published in a journal then it's factual. I think that's ascribing meaning that isn't actually there. Peer review _does_ tell you something. Certainly not "guaranteed correct" or even "guaranteed high quality" but it is definitely some sort of signal to the reader. Consider that this is nothing more than proactively pointing out that a linked paper which closely resembles work that would typically be affiliated with a journal (a signal gated by an editor) be peer reviewed (another signal) and those reviews possibly made public (yet another signal) does not have any of these signals. That is to say, it's an important disclaimer in order to avoid the appearance of attempting to deceive the audience.

godelski 1y ago
I don't think you're wrong, I just don't think those signals are very strong. I do think they used to be stronger, but paper discovery isn't really dominated that way anymore. Word of mouth was always common, but now that's expanded to social media as well as things like Google scholar or semantic. Hell, most journals and conferences are hard to search through. Maybe I'll land on their page from a Google search of the paper or when looking for those topics, but that's not any different from ArXiv. And truth is that I still have to do the same thing when landing in either location. I still have to read the abstract, give it a sniff test, and read it to varying depth depending on what's needed. There's a signal, I just haven't found it to be that meaningful
- fc417fc802 1y ago
  I never claimed it was useful for targeted searches by modern standards. It's primarily a time saving measure from my perspective. If I come across a paper and it is in a publication that has a track record of publishing shoddy results I am likely to prioritize other works (assuming that I have a large potential pool of them to investigate). If a nature paper claims a seemingly impossible result I'll most likely spend the time to at least glance through it and see if maybe my preconceptions were mistaken. If something on arxiv looks like nonsense I'm not going to waste any time on it (unless someone else told me it's worth it).