5 ms·

> Being published in a journal doesn't mean a work is correct nor does it mean peers read it. Putting the paper on ArXiv does mean peers are reviewing it. Sorr

by devmor 1y ago

> Being published in a journal doesn't mean a work is correct nor does it mean peers read it. Putting the paper on ArXiv does mean peers are reviewing it. Sorry I don't get what you mean by this. How does putting the paper on ArXiv mean that peers are reviewing it any more than publishing it in a journal? Both do mean that peers have the opportunity to review it, but neither guarantees it, and ArXiv is infinitely easier to upload anything to and never have it even looked at.

xondono 1y ago
Technically speaking, being published means at least the editors have reviewed. The quality of their reviews is another thing entirely.
- deleted 1y ago
  [deleted]
- godelski 1y ago
  Well... it can be desk rejected. Which I've actually had happen because the paper was already on ArXiv, even though it wasn't against the journal's policy. Took 4 weeks to resolve and then got desk rejected again for "not citing the correct works", with no further information... I don't submit to that journal anymore...
  - xondono 1y ago
    Sure it can, but then it’s not published. My only point is that if you get published in a journal, at least two people have seen it.
tensor 1y ago
Obviously journals vary in their standards, but many of the more respected ones do require other scientists read and critique the paper. You can argue about the quality of these reviews, certainly it's a process that needs improving, but this is what "peer reviewed" means. Trying to claim ArXiv papers are "peer reviewed" is utter nonsense. As you correctly point out, the only requirement to being on ArXiv is that someone with an account uploaded it there. There are no requirements that it passes any sort of verification or vetting process whatsoever, let alone having other scientists read and critique it. There is a very vocal movement these days that is trying to argue that we should do away with the traditional peer review process. Apparently that sometimes includes trying to redefine the very definition of "peer review" as the OP did.
- godelski 1y ago
  > There is a very vocal movement these days that is trying to argue that we should do away with the ***traditional*** peer review process. Apparently that sometimes includes trying to redefine the very definition of "peer review" as the OP did. (emphasis my own) Sorry, but I don't think you have an accurate image. You have it backwards. "Peer review" is a very new thing. We're talking really about mid 20th century, with this format not really being common until the 70's. If anything, we're trying to return to the earlier version of the system[0]. The older review system didn't have these concepts of acceptance rates, novelty, and all of that. They would mostly publish if your work was void of errors, did not copy others, or was not essentially a small variation of another work and trying to take credit (I'd still consider this plagiarism). We're fine with having conversations with reviewers, where we view this all as being "on the same team". A team trying to make the work the best work it can be. What we're not fine with is reviewers being set up as adversaries, who are looking for reasons to reject the works. That just creates perverse incentives. Really, it's all about addressing the question: What is the point of publishing? I'd say that the primary goal is to communicate the a work. A secondary goal is to help works be visible. But this form of review cannot validate works and does a bad job at invalidating works. Nor can this form of review determine how impactful a work will be and we have a clear tendency to reject works that end up being highly influential. We're checking the alignment of our current system with our intended goals. Even if we're wrong, I think it is obtuse to be outright dismissive. Should you not even ask the question? I'd argue that not asking the question is anti-scientific. Science requires challenging the status quo. If we don't, innovation slows to a crawl. You cannot rely on those before you having gotten everything right. Use the advantage of hindsight. Trust, but you must also verify (challenge). [0] We believe that this is the better formulation. Did it have problems? Yes. Can they all be solved? No. Does a global optima exist where all problems can be solved with a different system? Also no. So the belief is that good intentions trying to solve these problems only ended up creating worse ones. The cure was worse than the poison. > A 2003 editorial in Nature stated that, in the early 20th century, "the burden of proof was generally on the opponents rather than the proponents of new ideas." https://en.wikipedia.org/wiki/Scholarly_peer_review#History https://en.wikipedia.org/wiki/Scholarly_peer_review#History
godelski 1y ago
You're right. You can publish into the void. I was hoping what was said would be easily interpreted but I was incorrect. I'll revise to "putting on ArXiv doesn't mean peers haven't reviewed it." For what constitutes "a peer", I describe more here[0]. The definition varies wildly, and it is not an easy problem to determine who is even qualified to review a work. Hell, it can be hard to determine if you yourself are qualified to review a work! While I advocate for mostly abandoning the journals and conferences (or dramatic restructuring), I won't act like there's no problems with just publishing to ArXiv (or more preferably, OpenReview, since it allows commenting). But the truth is that there's no globally optimal solution to this problem. I just think the benefits outweigh the costs here. Frankly, most authors aren't publishing (posting on ArXiv or whatever) in bad faith. If anything, I think our current system incentivizes bad faith publishing, but that's a larger conversation (coupled with this one but requires talking about a few other factors). Frankly, we just waste a lot of time and effort for little to no gain (possibly even for losses). [0] https://news.ycombinator.com/item?id=44587535 https://news.ycombinator.com/item?id=44587535
- devmor 1y ago
  That makes your intention much more clear, and I am inclined to agree. A paper should not be dismissed as mere prose on the fact that it was published via ArXiv alone.

by devmor 1y ago

xondono 1y ago

Technically speaking, being published means at least the editors have reviewed. The quality of their reviews is another thing entirely.

deleted 1y ago
[deleted]
godelski 1y ago
Well... it can be desk rejected. Which I've actually had happen because the paper was already on ArXiv, even though it wasn't against the journal's policy. Took 4 weeks to resolve and then got desk rejected again for "not citing the correct works", with no further information... I don't submit to that journal anymore...
- xondono 1y ago
  Sure it can, but then it’s not published. My only point is that if you get published in a journal, at least two people have seen it.

tensor 1y ago

Obviously journals vary in their standards, but many of the more respected ones do require other scientists read and critique the paper. You can argue about the quality of these reviews, certainly it's a process that needs improving, but this is what "peer reviewed" means. Trying to claim ArXiv papers are "peer reviewed" is utter nonsense. As you correctly point out, the only requirement to being on ArXiv is that someone with an account uploaded it there. There are no requirements that it passes any sort of verification or vetting process whatsoever, let alone having other scientists read and critique it. There is a very vocal movement these days that is trying to argue that we should do away with the traditional peer review process. Apparently that sometimes includes trying to redefine the very definition of "peer review" as the OP did.

godelski 1y ago
> There is a very vocal movement these days that is trying to argue that we should do away with the ***traditional*** peer review process. Apparently that sometimes includes trying to redefine the very definition of "peer review" as the OP did. (emphasis my own) Sorry, but I don't think you have an accurate image. You have it backwards. "Peer review" is a very new thing. We're talking really about mid 20th century, with this format not really being common until the 70's. If anything, we're trying to return to the earlier version of the system[0]. The older review system didn't have these concepts of acceptance rates, novelty, and all of that. They would mostly publish if your work was void of errors, did not copy others, or was not essentially a small variation of another work and trying to take credit (I'd still consider this plagiarism). We're fine with having conversations with reviewers, where we view this all as being "on the same team". A team trying to make the work the best work it can be. What we're not fine with is reviewers being set up as adversaries, who are looking for reasons to reject the works. That just creates perverse incentives. Really, it's all about addressing the question: What is the point of publishing? I'd say that the primary goal is to communicate the a work. A secondary goal is to help works be visible. But this form of review cannot validate works and does a bad job at invalidating works. Nor can this form of review determine how impactful a work will be and we have a clear tendency to reject works that end up being highly influential. We're checking the alignment of our current system with our intended goals. Even if we're wrong, I think it is obtuse to be outright dismissive. Should you not even ask the question? I'd argue that not asking the question is anti-scientific. Science requires challenging the status quo. If we don't, innovation slows to a crawl. You cannot rely on those before you having gotten everything right. Use the advantage of hindsight. Trust, but you must also verify (challenge). [0] We believe that this is the better formulation. Did it have problems? Yes. Can they all be solved? No. Does a global optima exist where all problems can be solved with a different system? Also no. So the belief is that good intentions trying to solve these problems only ended up creating worse ones. The cure was worse than the poison. > A 2003 editorial in Nature stated that, in the early 20th century, "the burden of proof was generally on the opponents rather than the proponents of new ideas." https://en.wikipedia.org/wiki/Scholarly_peer_review#History https://en.wikipedia.org/wiki/Scholarly_peer_review#History

godelski 1y ago

You're right. You can publish into the void. I was hoping what was said would be easily interpreted but I was incorrect. I'll revise to "putting on ArXiv doesn't mean peers haven't reviewed it." For what constitutes "a peer", I describe more here[0]. The definition varies wildly, and it is not an easy problem to determine who is even qualified to review a work. Hell, it can be hard to determine if you yourself are qualified to review a work! While I advocate for mostly abandoning the journals and conferences (or dramatic restructuring), I won't act like there's no problems with just publishing to ArXiv (or more preferably, OpenReview, since it allows commenting). But the truth is that there's no globally optimal solution to this problem. I just think the benefits outweigh the costs here. Frankly, most authors aren't publishing (posting on ArXiv or whatever) in bad faith. If anything, I think our current system incentivizes bad faith publishing, but that's a larger conversation (coupled with this one but requires talking about a few other factors). Frankly, we just waste a lot of time and effort for little to no gain (possibly even for losses). [0] https://news.ycombinator.com/item?id=44587535 https://news.ycombinator.com/item?id=44587535

devmor 1y ago
That makes your intention much more clear, and I am inclined to agree. A paper should not be dismissed as mere prose on the fact that it was published via ArXiv alone.