3 ms·

Putting papers and code on arXiv shouldn't be punished. The incentive to do this is to protect your idea from getting scooped, and also to inform your close com

by muds 3y ago

Putting papers and code on arXiv shouldn't be punished. The incentive to do this is to protect your idea from getting scooped, and also to inform your close community on interesting problems that you're working on and get feedback. ArXiv is meant for work in progress ideas that won't necessarily stand the peer review process, but this isn't really acknowledged properly on social media. I highly doubt the Twitter storm would have been this intense if the twitter posts explicitly acknowledged this as a "Draft publication which hints as X." But I admit that pointing fingers at nobody in general and social media specifically is a pretty lazy solution. The takeaway IMO seems to be to prepend the abstract with a clear disclaimer sentence conveying the uncertainty of the research in question. For instance, adding a clear "WORKING DRAFT: ..." in the abstract section.

deleted 3y ago
[deleted]
cscurmudgeon 3y ago
> The incentive to do this is to protect your idea from getting scooped, and The other side is flag planting with half-baked ideas and results.
- deleted 3y ago
  [deleted]
ttpphd 3y ago
I think you missed the point that data needs to be collected and presented ethically. It's not about it being a work in progress and not peer reviewed.
- muds 3y ago
  I agree that the data collection process wasn't ethical, and the professor should definitely be reprimanded for that. It's extremely sad that the coauthors weren't aware of this as well. And I feel terrible for the undergrads: their first research experience was publically rebuked for no fault of their own. However, there is no shortage of projects with sketchy data collection methodologies on arXiv that haven't received this amount of attention. The point of putting stuff on arXiv _is_ that the paper will not pass / has not passed peer review in its current form! I might even call arXiv a safe space to publish ideas. We all benefit from this: a lot of interesting papers are only available on arxiv v.s. being shared between specific labs. I'm concerned that this fiasco was enabled by this new paradigm in AI social media reporting, where a project's findings are amplified and all the degrees of uncertainty are repressed. And I'm honestly not sure how to best deal with this other than either amplifying the uncertainty and jankyness in the paper itself to an annoyingly noticeable level, or just going back to the old way of privately sharing ideas. Maybe this is the best case scenario for these sorts of papers? They pushed a paper on a public journal, and got a public "peer review" of the paper. Turns out the community voted "strong reject;" and it also turns out that the stakes for public rejection are (uncomfortably, IMO) higher than for a normal rejection. Maybe this causes the researchers to only publically release better research, or (more likely) this causes the researchers to privately release all future papers.
  - PeterStuer 3y ago
    It was a "copyright" issue, nothing to do with "ethics".
  - caddemon 3y ago
    I agree with the ideal you're speaking of, but the lecturer that uploaded the paper was promoting it on Twitter in a way that really isn't consistent with that sort of intent. This entire scenario seems like a failed attempt to market his own academic brand, without care for the underlying scientific content. It's not a genuine sharing of an unfinished idea or early observation.
  - xdavidliu 3y ago
    he didnt have the permission to post the data. a similar protest would have been made if he had posted the data on github, stackoverflow, reddit, or hackernews, and none of those are as peer reviewed as academic journals
currymj 3y ago
the official stance of arXiv is that it is intended for papers that are finished and ready for submission to peer review, i.e. the authors believe it's finished and publishable. in a practical sense, most people don't think of it this way though. putting something on arXiv means "i want people to be able to cite this, and for it to show up on Google Scholar", which might include works in progress, short notes, or lots of other things that don't fit the official criteria.

by muds 3y ago

deleted 3y ago

[deleted]

cscurmudgeon 3y ago

> The incentive to do this is to protect your idea from getting scooped, and The other side is flag planting with half-baked ideas and results.

deleted 3y ago
[deleted]

ttpphd 3y ago

I think you missed the point that data needs to be collected and presented ethically. It's not about it being a work in progress and not peer reviewed.

muds 3y ago
I agree that the data collection process wasn't ethical, and the professor should definitely be reprimanded for that. It's extremely sad that the coauthors weren't aware of this as well. And I feel terrible for the undergrads: their first research experience was publically rebuked for no fault of their own. However, there is no shortage of projects with sketchy data collection methodologies on arXiv that haven't received this amount of attention. The point of putting stuff on arXiv _is_ that the paper will not pass / has not passed peer review in its current form! I might even call arXiv a safe space to publish ideas. We all benefit from this: a lot of interesting papers are only available on arxiv v.s. being shared between specific labs. I'm concerned that this fiasco was enabled by this new paradigm in AI social media reporting, where a project's findings are amplified and all the degrees of uncertainty are repressed. And I'm honestly not sure how to best deal with this other than either amplifying the uncertainty and jankyness in the paper itself to an annoyingly noticeable level, or just going back to the old way of privately sharing ideas. Maybe this is the best case scenario for these sorts of papers? They pushed a paper on a public journal, and got a public "peer review" of the paper. Turns out the community voted "strong reject;" and it also turns out that the stakes for public rejection are (uncomfortably, IMO) higher than for a normal rejection. Maybe this causes the researchers to only publically release better research, or (more likely) this causes the researchers to privately release all future papers.
- PeterStuer 3y ago
  It was a "copyright" issue, nothing to do with "ethics".
- caddemon 3y ago
  I agree with the ideal you're speaking of, but the lecturer that uploaded the paper was promoting it on Twitter in a way that really isn't consistent with that sort of intent. This entire scenario seems like a failed attempt to market his own academic brand, without care for the underlying scientific content. It's not a genuine sharing of an unfinished idea or early observation.
- xdavidliu 3y ago
  he didnt have the permission to post the data. a similar protest would have been made if he had posted the data on github, stackoverflow, reddit, or hackernews, and none of those are as peer reviewed as academic journals

currymj 3y ago

the official stance of arXiv is that it is intended for papers that are finished and ready for submission to peer review, i.e. the authors believe it's finished and publishable. in a practical sense, most people don't think of it this way though. putting something on arXiv means "i want people to be able to cite this, and for it to show up on Google Scholar", which might include works in progress, short notes, or lots of other things that don't fit the official criteria.