3 ms·
> All research and code posted on them gets a quick once-over; good work gets the attention it deserves; bad work is quickly ignored. What is the basis for all
by wolverine876 5y ago
> All research and code posted on them gets a quick once-over; good work gets the attention it deserves; bad work is quickly ignored.
What is the basis for all these claims? Who is giving it the quick once-over?
Crowd-sourced review and information, despite some strengths and high initial hopes, has a record of extraordinary misinformation and disinformation. Why would we want to use that system for scientific research?
I prefer careful peer review, standards for announcing funders, etc.
- treesprite82 5y ago> Who is giving it the quick once-over? arXiv has 200 expert moderators spread through different fields to filter out papers that are blatantly misleading, unoriginal, non-substantive, or in need of significant review and revision. It's not a peer-review, but it's not a complete free-for-all either.
- wolverine876 5y agoThanks! I will look that up (but if you happen to have any links ...).
- gzer0 5y agoHere you go! [1] https://arxiv.org/moderators/ https://arxiv.org/moderators/ [2] https://arxiv.org/help/moderation https://arxiv.org/help/moderation
- wolverine876 5y agoThank you. And I found this one too: https://blog.arxiv.org/2019/08/29/our-moderation-process/ https://blog.arxiv.org/2019/08/29/our-moderation-process/
- light_hue_1 5y agoThis is completely false! Please stop telling people that arXiv moderators judge the technical content of papers. That's not their role and it leads to people trusting arXiv when they should not. It's a complete-free-for-all. arXiv moderators do not judge the technical content. They don't filter misleading submissions. They don't filter work in need of revision. This is literally described on the arXiv website: https://arxiv.org/help/moderation https://arxiv.org/help/moderation "What policies guide moderation before public announcement? " They filter spammers, check for total obvious nutjobs ("My grilled cheese said P=NP"), for crazy formatting, for blatant copyright violations. Publishing junk on arXiv is trivial if you're not too crazy and know a little bit how to use the right words. You can publish anything.
- treesprite82 5y ago> They don't filter work in need of revision. This is literally described on the arXiv website: https://arxiv.org/help/moderation https://arxiv.org/help/moderation "What policies guide moderation before public announcement? " The page you're linking backs up what I've listed. The third subheader in that section for example: > A submission may be declined if the moderators determine it lacks originality, novelty, or significance. > Submissions that do not contain original or substantive research, including undergraduate research, course projects, and research proposals, news, or information about political causes (even those with potential special interest to the academic community) may be declined. > Papers that contain inflammatory or fictitious content, papers that use highly dramatic and misrepresentative titles/abstracts/introductions, or papers in need of significant review and revision may be declined. --- > it leads to people trusting arXiv when they should not I'm only claiming that their moderation is a quick once-over to filter out papers blatantly in violation of those policies (like the "total obvious nutjobs" you describe), while being clear that it's not a peer review.
- robwwilliams 5y ago“May” is the key word in that long sentence. bioRxiv certainly does not review content of submissions using the standard definition of the word “review”. They may scan for style and general content type—for example rejecting reviews. But I agree with the parent comment that these archives are extremely valuable. I disagree that informal community comments are of much critical value. In most cases twitter comments and micro-reviews are relatively trivial and are usually based on quick reads rather than deep perusals.
- sombremesa 5y agoIf you happen to be interested in the "my grilled cheese said P=NP" type papers [0] for entertainment value, check out Vixra [1]. Though calling them papers can be a stretch. Here's one that "proves" P=NP [2]. [0] https://vixra.org/pdf/1903.0040v1.pdf https://vixra.org/pdf/1903.0040v1.pdf [1] https://vixra.org https://vixra.org [2] https://vixra.org/pdf/1502.0047v1.pdf https://vixra.org/pdf/1502.0047v1.pdf
- oakfr 5y agoThe problem is that careful (conference) reviews don’t scale. Large conferences end up suffering from a highly stochastic behavior where excellent work is borderline rejected on a regular basis while mediocre/incorrect work gets accepted every so often. github/arxiv are no silver bullet but offer an interesting alternative (with their own set of challenges, though).
- wolverine876 5y agoEvery solution has flaws, as you say. The careful reviews have worked very well - science is one of the great triumphs of humanity. That doesn't mean we shouldn't look for improvements, of course.
- mjn 5y ago> good work gets the attention it deserves I'm not sure I'd go that far with the optimism, at least in my field (artificial intelligence). Some obviously bad work does immediately fade into obscurity, and better work probably does on average get more attention, but the variation is huge. There is so much stuff on arXiv, that to get attention you need some kind of PR push so people notice it in the firehose, or a dice-roll around a viral tweet or science journalist noticing it. Some of the better funded university and corporate research groups have actual professional PR and science-comm teams doing coordinated social-media blitzes, press releases, and blog posts around new arXiv papers! That's a huge factor in determining whether a given paper gets attention.
- wolverine876 5y agoThanks for the perspectives. One important nitpick: > Some obviously bad work does immediately fade into obscurity, and better work probably does on average get more attention You don't know what you haven't seen, which is just a restatement of the core problem. You'd need a study of the entire population to know about the correlation between paper 'quality' and outcomes.