4 ms·

At least in math, that is just untrue. It can be years from when the first preprint comes out to when it's actually published. If everyone waited around for the

At least in math, that is just untrue. It can be years from when the first preprint comes out to when it's actually published. If everyone waited around for the official publication, nothing would ever get published. Journals are pretty pragmatic about it.

mankash666 9y ago
That doesn't make sense. I can publish anything on arXiv, and then refrence that in an academic journal, from what you're saying. That cannot and should not be how proper peer review happens. The delay in publication of math journals cannot be a reason to skip the proper peer review process!
- dagss 9y ago
  Same in some parts of astrophysics. The real peer review is social, you review when you cite so to speak, if you cite garbage then your work turn into garbage, so.... journal peer review is just an after-thought to collect points for funding and hiring bodies, it is not the real validation of research. Not to mention I know examples of respected journals in the field letting total crap through (not opinion, total misapplication of statistical theorems) so the journal stamp of approval isn't everything. Arxiv is not ideal, but I refuse to think going back to journals as gatekeeper is a step forwards. That would be a step backwards. Some sort of distributed social network peer review of arxiv papers, and having points scored there that count as the review for for papers for funding bodies etc would probably be best. The journal model is outdated and charging too much money to provide little value (I mean, they just coordinate the review process, a piece of software could do the same coordination).
- _delirium 9y ago
  It really depends on what you're using citations for. Peer-reviewed papers don't necessarily need a transitive property where they only cite other peer-reviewed papers (which only cite... etc.). Norms vary by field, but in most fields, it's always been acceptable to cite various kinds of non-peer-reviewed literature, such as technical reports. Even popular press articles or blogs are fine, as long as they're cited appropriately, not to back up critical claims. Ultimately the single paper under review is peer-reviewed for whether it demonstrates its own claims, and that depends in part on the reviewers judging whether it uses citations appropriately, meaning that it can cite anything it wants, as long as it doesn't cite things to demonstrate claims that the citation can't really support. In mathematics my impression is that citing arXiv papers is usually done appropriately. If a surprising proof of a longstanding open problem was recently posted on arXiv, people won't usually cite it as authoritative right away. Instead a citation will be worded with something like "a recent claimed proof by so-and-so...", making it clear that it's still preliminary and not to be treated as a sure thing yet (how strongly to add this caveat is a bit of a judgment call). As a reviewer that's usually what I look for. If someone cites an arXiv paper (or tech report, or even blog post) to give context, to credit other teams who are doing related work, to point out alternative approaches that the current paper isn't investigating, etc., that's perfectly fine. Those are more like informational "see also" citations. If a paper cites an arXiv preprint as authoritative evidence of something that is really critical to the current paper itself being correct, then yes, that's where it can be more iffy and I'd ask the authors to either qualify that claim more, or bolster it with another more solid citation. There is a little bit of a problem with this currently in machine learning, where some people make overly bold/general claims in arXiv preprints that their methodology/data can't really support, and other people sometimes cite these claims a bit too credulously. Important to push back on, but not to the extent of banning arXiv citations.
- evanb 9y ago
  The arXiv _is_ moderated. If there's something wacky the moderators tend to ding it.
  - versteegen 9y ago
    Irrelevant since as I understand it moderators only weed out papers that don't even have the appearance of on-topic research papers; that does nothing to ensure preprints on arXiv are trustworthy.

mankash666 9y ago

That doesn't make sense. I can publish anything on arXiv, and then refrence that in an academic journal, from what you're saying. That cannot and should not be how proper peer review happens. The delay in publication of math journals cannot be a reason to skip the proper peer review process!

dagss 9y ago
Same in some parts of astrophysics. The real peer review is social, you review when you cite so to speak, if you cite garbage then your work turn into garbage, so.... journal peer review is just an after-thought to collect points for funding and hiring bodies, it is not the real validation of research. Not to mention I know examples of respected journals in the field letting total crap through (not opinion, total misapplication of statistical theorems) so the journal stamp of approval isn't everything. Arxiv is not ideal, but I refuse to think going back to journals as gatekeeper is a step forwards. That would be a step backwards. Some sort of distributed social network peer review of arxiv papers, and having points scored there that count as the review for for papers for funding bodies etc would probably be best. The journal model is outdated and charging too much money to provide little value (I mean, they just coordinate the review process, a piece of software could do the same coordination).
_delirium 9y ago
It really depends on what you're using citations for. Peer-reviewed papers don't necessarily need a transitive property where they only cite other peer-reviewed papers (which only cite... etc.). Norms vary by field, but in most fields, it's always been acceptable to cite various kinds of non-peer-reviewed literature, such as technical reports. Even popular press articles or blogs are fine, as long as they're cited appropriately, not to back up critical claims. Ultimately the single paper under review is peer-reviewed for whether it demonstrates its own claims, and that depends in part on the reviewers judging whether it uses citations appropriately, meaning that it can cite anything it wants, as long as it doesn't cite things to demonstrate claims that the citation can't really support. In mathematics my impression is that citing arXiv papers is usually done appropriately. If a surprising proof of a longstanding open problem was recently posted on arXiv, people won't usually cite it as authoritative right away. Instead a citation will be worded with something like "a recent claimed proof by so-and-so...", making it clear that it's still preliminary and not to be treated as a sure thing yet (how strongly to add this caveat is a bit of a judgment call). As a reviewer that's usually what I look for. If someone cites an arXiv paper (or tech report, or even blog post) to give context, to credit other teams who are doing related work, to point out alternative approaches that the current paper isn't investigating, etc., that's perfectly fine. Those are more like informational "see also" citations. If a paper cites an arXiv preprint as authoritative evidence of something that is really critical to the current paper itself being correct, then yes, that's where it can be more iffy and I'd ask the authors to either qualify that claim more, or bolster it with another more solid citation. There is a little bit of a problem with this currently in machine learning, where some people make overly bold/general claims in arXiv preprints that their methodology/data can't really support, and other people sometimes cite these claims a bit too credulously. Important to push back on, but not to the extent of banning arXiv citations.
evanb 9y ago
The arXiv _is_ moderated. If there's something wacky the moderators tend to ding it.
- versteegen 9y ago
  Irrelevant since as I understand it moderators only weed out papers that don't even have the appearance of on-topic research papers; that does nothing to ensure preprints on arXiv are trustworthy.