10 ms·
I feel especially strongly that papers that result from taxpayer-funded research should be free.
by payne92 11y ago
I feel especially strongly that papers that result from taxpayer-funded research should be free.
- mmmBacon 11y agoI also think code used to produce results should be published as well.
- SalmoShalazar 11y agoI agree with this in theory... but do you really want my garbage, ad hoc, messy code that I used to pipe output from one program to another?
- bravura 11y agoYes. Especially because if you are forced to publish your code, it will be better.
- return0 11y agoAbsolutely yes. Science isnt supposed to be clean and tidy, but reproducible.
- nv-vn 11y agoConsidering that a mistake in such a program could completely skew results, yeah that seems pretty important.
- lacker 11y agoYes, because if it doesn't even compile then I will call BS on your results.
- skybrian 11y agoSee, that's exactly what you shouldn't do if you want to encourage academics to release the source code. Academic code sometimes only has been made to work on one machine (the author's). Someone reproducing the results should still have access to the source, even if they have to hack it to work on their system.
- deleted 11y ago[deleted]
- themartorana 11y agoI don't know... Did you do it right? ;) But seriously.
- CompuHacker 11y agoIt's better than nothing.
- cube00 11y agoDefinitely yes, then I can see if somewhere in that garbage, ad hoc, messy code why I can't replicate the result myself.
- skybrian 11y agoI'm not actually recommending this license, but it seems relevant to mention: The CRAPL: An academic-strength open source license http://matt.might.net/articles/crapl/ http://matt.might.net/articles/crapl/
- marvy 11y ago"the Community Research and Academic Programming License"
- mrcsparker 11y agoYes, please. To be fair, a lot of people do publish their working code with the paper. It helps people like me understand how to do the work. Bad code is fine, good code is better, great code is great. Stick to fine and I can do my job and you can write papers knowing that your work will be used by many.
- zmmmmm 11y agoIf it's that garbage, how do you know your results are correct? It should be at least non-garbage enough that someone else could theoretically check it, or you're not doing science at all.
- jjoonathan 11y agoYou don't. But there are 5x-100x more bugs lurking in the methodology so triage is the name of the game. A much bigger problem is that grantsmanship strongly incentivizes against verifying your methodology and being diligent in your construction of null hypotheses.
- nilkn 11y agoTo be honest, I'm surprised anybody even takes research seriously which depends heavily on a custom program whose source code is kept hidden.
- euske 11y agoThis is like saying "I don't want to file a tax because my handwriting is bad". Well, if you're embarrassed by your terrible handwriting, maybe it's a time to put some effort to improve it.
- dinkumthinkum 11y agoOk I don't mean this in a mean way but you don't seem to appreciate what CS research is like. It is not about handwriting.
- Sacho 11y agoFiling taxes is not about handwriting either - I guess you missed the analogy. Research isn't about writing good code, but publishing your code is useful(for reproducibility, which is part of research), and if you're embarrassed by your code quality, then you should put some effort into it.
- lacker 11y agoThat would also have the benefit of making sure the code actually works. From my time in CS grad school I'm pretty sure there are many published papers where the author never had code that achieved the results they described.
- taneq 11y agoOr code that worked on the specific test case described in the paper but was flaky as hell on anything else.
- albinofrenchy 11y agoThe most disillusioning thing for me as an undergrad was to try to replicate results from a paper on a relatively complicated problem, and not achieving anywhere near the papers level of results. I got access to the source code, and the super complicated algorithm added almost nothing to the results, the glossed over / hand waved past data normalization worked so well that there wasn't a need for any further classification. This paper was pretty well received and cited, despite it basically not work.
- andreasvc 11y agoI am afraid that what you describe is a pretty fundamental limitation of peer review. Currently it is not expected of reviewers to try to replicate experiments, but only to check if the methodology of the paper seems sound and the relevant previous work is cited. Should this be expected of reviewers? Should the authors provide all code and data so that the results can be replicated at the push of a button? That seems a pretty tall order. Seems like we'll be stuck with a modus operandi of lots of papers being published, peer review only catching obvious errors, and replication being very rare.
- albinofrenchy 11y agoI get that, and I don't even think that that should be the reviewers job. But for CS in particular, I do wish more people published source code and build procedures. I think a central hub for public comments, with source, and available PDF would be incredibly useful for CS as a field. For most papers I had access to, to get the source I'd have to go through at least one person. So I tried for quite a while to replicate the result before even trying to get access. If it was available, on say github, I'd have grabbed the source code as a resource to understand the paper, and with something like publicly available commenting, I think it'd have been a non-issue. The problem is, I can see how such a system would be incredibly constructive to the field and the community, but it could be a liability to the authors, and would make publishers meaningless, so I doubt it happens anytime soon.
- dinkumthinkum 11y agoI think you are providing a large barrier unnecessarily. Research projects and publishing a paper is more complicated than that. It is good when the researcher can do that but it is not always practical. Really, a lot of the comments to this and the downvotes seem to betray a lack of understanding of what cs research is like, which is understandable I guess. CS research is not like creating physics experiments, there are methods, techniques, and algorithms. When there is an experimental result, CS researchers do not treat those as sacrosanct and do have an informed skepticism but we are generally able to determine and discern the bulk is what is important from the content of the paper itself, in most cases. Yeah, if you look at the experimental results section from papers in SOSP, you could come up with all kinds of complaints probably all day long but there is more to it and the community than is being taken for granted here.
- studentrob 11y agoWow, I never considered that this might fall under the same umbrella. It seems very reasonable because it isn't very hard to publish that now. Even if the shared code wouldn't run without a ton of requirements and very specific setup, it's still something. I know we all hate to release code that doesn't compile or run outside our own setup because we then feel we will be responsible for helping someone else getting it to run on their machines to which we have no physical access. Not to mention it is always a mess figuring that stuff out. But other research fields do not feel compelled to hold the hands of their researchers to replicate experiments. Presumably, whoever's code is easiest to run will more likely serve as the basis for other future research, thereby enhancing career growth of the original publisher. I'm not a researcher by the way, and this just dawned on me, so if this is obvious to all of you, then I apologize for the block of text.
- Balgair 11y agoHave you worked with biologists? If they could figure out how to use git, let alone github, I would be astounded. No forgetting, these are incredibly smart people, they just are bench-top scientists, not laptop ones. If you want their code, it will be, at best, a copy-paste of badly hardcoded Matlab, years in the making, with for-loops for the sake of for-loops alone, and without ANY comments or documentation whatsoever. Honestly, it would take less time for you to write it yourself and compare, rather than to try to peek under the hood of a spaghetti mess. Honest to god and on the grave of my mother this happened to a friend. He was going over some old Fortran 88 code, filled with GOTO statements. The code, at best, was a rat's nest. Only a deep and long fight could get it into your brain. At about 3 am after a long day in the lab, she finally gets to a line in the code that says GOTO LINE 12345 with the comment 'HAHA MADE YOU LOOK' . Her boss bought her a new computer after she threw that one off the roof. That experience is universal with Biologist code.
- AnthonyMouse 11y ago> Honestly, it would take less time for you to write it yourself and compare, rather than to try to peek under the hood of a spaghetti mess. That's not actually the point. The point is reproducibility. If you write your own code and get a different result, why? If the reason is the code then having the other code, no matter how terrible it is, allows you to figure out what the difference is. And then somebody is wrong and you know why and can publish that result.
- timr 11y agoExactly. IMO, this is a big problem in some areas of computational biology right now. Code to a famous project gets released, and instead of independently validating the results, other researchers (rationally) choose to just build on top of the famous project. It's faster, easier, and the "collaboration" leads to social prestige for everyone involved. The end result is that empires are built, reproducibility is (paradoxically) harmed, and we're someday going to end up finding out that some big, high-profile projects were built on pillars of sand.
- cheald 11y ago
- ikeboy 11y agoAre you, as a taxpayer, willing to pay (your share of) the extra $1000 or so per paper that it would take?
- east2west 11y agoGiven academic salary and researcher publication rate, I am confident that it is far less than $1000 you pulled out of air. How much did your tax go up when NIH mandated free publication, after a 2-years period?
- ikeboy 11y agoFTA: >But that financial model requires authors to pay a processing charge that can run anywhere from $1,500 to $3,000 per article so the publisher can recoup its costs. They may have pulled it out of thin air (although I've seen similar numbers elsewhere, this doesn't seem to be in dispute).
- anbende 11y agohttps://www.plos.org/publications/publication-fees/ https://www.plos.org/publications/publication-fees/ Other open access journals are similar. Publication rates are high, because the vast majority of published research is NOT open access.
- matwood 11y ago$1000 for what? The research and paper writing was already funded by the tax payer. Peer review is done for free by other researchers today. Journals provide very little value for the money they extort for paper access.
- ikeboy 11y agoFTA: >But that financial model requires authors to pay a processing charge that can run anywhere from $1,500 to $3,000 per article so the publisher can recoup its costs. Edit: also, I don't expect margins of 30% from companies producing very low value. Do you really think that if they'd lower their margin to 0, with prices of around 30% less, everyone would stop complaining?
- wodenokoto 11y agoFrom a government / national perspective should it be free world wide? If so, why should nations give away their expensive research that potentially give them an edge either military or commercially? Personally I almost never traded technology when I played civilization, let alone give it away!
- knorby 11y agoNationally-funded research that's published in expensive journals is still very much available to the rest of the world. That's true with about anything that's published...
- wodenokoto 11y agoI was gonna say that the rest of the world purchase from these journals, but of course the journal might as well be foreign so that argument doesn't hold. Good point about the current situation.
- return0 11y agoJournals don't buy the science. It's given to them for free from scientists in return for "prestige". Most of the time this is damaging to their government's interests, since they give away taxfunded research usually to a foreign entity.
- AnthonyMouse 11y ago> From a government / national perspective should it be free world wide? Yes. The answer is unequivocally yes. Your country will benefit vastly more from the research being freely available than other people having a better world on your dime will disadvantage you. And here's the thing. Other countries face the same calculus. Scientific research brings the greatest return on investment of anything ever. It's why we have electricity and computers and satellites and penicillin. At the national level you can pay for it and give it away and still come out ahead regardless of what anybody else does. Since every country benefits from doing it by more than it costs them, everybody can "cooperate" by doing research and giving it away without having to worry about anybody else defecting -- because defectors can't hurt you, they can only not help you. The sooner everybody realizes that the sooner everybody can choose to cooperate and the better off the whole world will be. And the returns when every country publishes research for free are... large.
- beambot 11y agoWhat about all the IP generated as a result of taxpayer-funded research?
- jjoonathan 11y agoThere are three types of funding going into US labs: money, green cards, and IP rights. Science is already horribly under-funded and removing IP rights, one of the sources of non-conventional funding, would make it even more so unless the removed non-conventional funding was replaced with conventional funding. Which is unlikely. Given that context, I'm all for letting government funded labs keep their IP.
- mc808 11y agoThat too, although a case could potentially be made that licensing of university-owned patents reduces the need for tax subsidies. E.g. the University of Wisconsin recently won a $234 million judgment against Apple just as their state funding was being cut by $250 million. http://news.wisc.edu/warf-wins-patent-infringement-lawsuit-against-apple/ http://news.wisc.edu/warf-wins-patent-infringement-lawsuit-a...
- AnthonyMouse 11y agoWouldn't it be more efficient to just tax Apple by the same amount of money and then use it to fund research, rather than turning universities into patent litigants and then having a large proportion of the money go to lawyers rather than research?
- mc808 11y agoI'm not really interested in those particulars but offered it as possible rationale that isn't there for academic publishing. I don't think patents (or copyrights) should be granted for publicly funded research at all, as it's effectively work-for-hire already owned by the public. But it gets more difficult when public grants are only a minor part of the funding.
- 11y ago
- beambot 11y agoWhat about all the IP generated as a result of taxpayer-funded research?