4 ms·
> There should be consequences for the people who publish these things. Should there be consequences for people who publish buggy code on github? Because that
by wsxcde 13y ago
> There should be consequences for the people who publish these things.
Should there be consequences for people who publish buggy code on github? Because that situation is completely equivalent to what these phd students did. ArXiv isn't a repository of peer-reviewed accepted papers, it's just a repository of papers, just like how github is repository of code.
The people to blame here aren't the students themselves. They just wrote up a mildly interesting observation and put it up on a repository. The people to blame are journalists and internet dumbasses who used this paper to serve their own selfish ends.
> I can't tell you how many times someone has thrown a junk science article in my face, thinking that the issue in question was now settled.
There's no denying the existence of a lot of peer-reviewed junk science but sometimes there are perfectly good reasons why papers with faulty conclusions are accepted. The question addressed by the paper might be interesting, the experimental methodology might be interesting, the data sources used in the research might be interesting, or the editors/program committee might just want to highlight the lack of progress in an area that deserves more attention.
Publishing something that is completely fault-free isn't a desirable or even an achievable goal. This is a fundamental misunderstanding of the purpose of scientific publishing. The goal of scientific publication is to advance the field by stimulating discussion at the boundaries of human knowledge. This can often be done even with "buggy" results. Think of it as the scientific equivalent of fail often, fail early and fail fast mantra that drives many startups.
I'm not dismissing your frustration though because the problem you've described is a real one. We aren't very good at communicating scientific results to laypersons. But unlike you, I don't have a glib solution to a problem I don't fully understand. Magazines like the communications of the ACM with accessible, professionally-edited articles written by experts in the field are probably one step in the right direction, but there is a lot more that needs to be done here.
- hrjet 13y agoWhat matters is the effect. If someone puts up code on GitHub which purports to compute the Fibonacci series, but mistakenly does a `rm -r ~/*` then surely there should be a way to flag that code or author ("consequences").
- ameister14 13y agoI think that's a pretty postmodern way of looking at things. Intent definitely matters to me.
- wsxcde 13y agoIf you find a flaw in a published paper, the normal course of action is to e-mail the authors letting them know what you think is wrong. If authors think this is a devastating flaw, they will retract their paper. Even if they don't, you can still publish a note yourself explaining why these results are flawed. This happens all the time and the 'consequences' you desire are already built into the system. Flawed papers don't do very well on citation or influence metrics. But much more importantly, the flaw in your mental model is thinking that claims in published papers are expected to be correct. They aren't. The goal of a paper is to advance the field by stimulating discussion and new research. The unfortunate consequence of this is that laypersons can't directly use research papers as an authoritative source describing our current understanding of a subject. The github analogy still holds up. Non-programmers shouldn't be downloading code off github, compiling and running it because they have no way of evaluating whether the code really does what it claims to do. They're probably better off getting applications from curated app stores. Accessible scientific publications written by recognized experts in the field are the equivalent of curated app stores. That is where laypersons should be looking to understand the state of the art in scientific knowledge.
- OverlordXenu 13y agoThis is just kind of ridiculously fallacious. Surely a slightly inaccurate paper is not comparable to malicious code. That's not a fair comparison to make.
- hrjet 13y agoI wasn't comparing that particular paper with malicious code. Inaccurate papers in other fields (like biology) might have a more significant impact. If we were to continue the GitHub analogy, it would be great if ArXiv had a public "issue" list.
- dllthomas 13y ago
- judk 13y agoArxiv is an invitation-only preprint-server. Authors who post junk are abusing their invitation and hurting the reputation of all Arxiv contributors. In HN-speak, Arxiv is like a Svtle blog.
- _delirium 13y agoThe arXiv is not invitation-only. Anyone can sign up for an account: https://arxiv.org/edit-user/?tapir_dest=https%3A%2F%2Farxiv.org%2Fauth&submit=Register+for+the+first+time https://arxiv.org/edit-user/?tapir_dest=https%3A%2F%2Farxiv....
- walrus 13y agoYou need to be endorsed by an active user in order to submit papers: https://arxiv.org/help/endorsement https://arxiv.org/help/endorsement