17 ms·
Thousands of AI researchers are boycotting the new Nature journal
- jbob2000 8y agoedit: people don't like discussing things, just want to see their own viewpoints reinforced.
- collyw 8y agoWith biology at least a lot of your reputation relies on your reputation, and a lot of that comes from publications and the impact factor of the journals you publish in, Nature being the one to aim for. Not sure if its as relevant to AI.
- tabtab 8y agoThe old model may be obsolete, or at least need tuning. What about offering a "vetting grade"? Anybody can submit (assuming minimum standards) and anyone can read, but you'd have to pay to have your research vetted, perhaps with different levels of vetting. Don't make it all-or-nothing.
- gowld 8y agoComputer science doesn't need journals because (1) It's generally easy enough to reproduce any work you care about, so you don't need to take anyone's word for its correctness, and (2) researches can make a fortune in industry doing their (profoitable) work or something related, so the don't need to suck up prestige factories in order to get funding.
- collyw 8y agoI was thinking as well that its less abstract. Biology research takes a lot time and of work to filer through to practical applications. AI research is generally doing something practical straight away.
- adw 8y agoThat role (in machine learning) is already filled by conference proceedings (plus ArXiV), so this journal is an opportunistic attempt to monetize a niche which doesn't need filling.
- jononor 8y agoIn some fields, like machine learning, many researchers put their work onto arxiv. There is like 50 papers a day now. https://arxiv.org/list/stat.ML/recent https://arxiv.org/list/stat.ML/recent
- Eridrus 8y agoI work with a fair amount of people from academia, a lot of them seem to have the view that things that have not been peer reviewed are not worth reading. But ML already has conferences and journals where work is peer reviewed and published in an open access manner. So it's not that academics see journals/conferences as value-less, but that they are already doing all this work for free, why do it for free for Springer who will then charge everyone?
- mediocrejoker 8y agoI think a lot non-academics (myself included) tend to hear the phrase "peer reviewed" and conflate it (rightly or wrongly) with "these results are replicable and therefore correct." I don't know what the long term solution is but I have a feeling it won't involve companies that charge a lot of money for a service (publishing) that has been effectively free since the Internet has existed.
- robkop 8y agoA lot of peer reviews in ML are currently completely open. For instance you can view the reviews for papers in previous and upcoming conferences here [0]. Also in terms of replicability ML is in a unique position where a lot of research is on standard datasets with models that don't take an excessive amount of compute power to train. This means that if the authors include a github link to their code you can fully replicate their results. So some "peer reviewed" ML papers can actually subscribe to your first description of peer reviewed. [0]: https://openreview.net/ https://openreview.net/
- Eridrus 8y agoI think a lot of ML research, despite being replicable is not necessary useful, because what we really want, approaches we can add in addition to all the other methods we have, that introduce meaningful improvements without unnecessary complexity and work across datasets, are quite rare. And that's assuming the evaluation was done well. Not to say the research is worthless, just that straight replicability is not necessarily enough.
- auntienomen 8y agoPeople don't find out about research by reading journals. They go to lectures and read mailing lists for that. Journal articles are typically a year too late. The value-add from journals is that they (at least theoretically) guarantee another person has read the paper carefully and believes it is correct.
- outlace 8y agoIf people can post a pre-print to Arxiv then why is it a big deal if it also gets published in Nature Machine Intelligence? Sure a better formatted and edited version may be behind a paywall but anyone can find the original arxiv paper and get the important points. edit: To make it clear, I'm talking about submitting to BOTH Arxiv, where it's freely available, AND a paywalled journal like Nature. edit 2: I get that we don't want to waste, often government, money on mere distribution of research, but I also think researchers need a way to distinguish their work from the thousands of arxiv papers that get posted every year. Whether that's through a Nature publication or something else is immaterial.
- stanfordkid 8y agoAcademic journals don't result in any value creation for society. If "publishing in Nature" is no longer a credential for academics, then Nature can't charge universities big bucks... that money can be spent by institutions on other things (like funding GPU's to do more actual machine learning research)... so boycotting the journal is strictly a good thing.
- mooneater 8y agoHopefully all that money does not have to go into GPUs! Universities in less rich countries would be on a more level playing field, if these massive subscriptions were not table stakes.
- cossatot 8y agoUniversities in developing nations get heavily discounted if not free subscription and individual access fees to most journals. Library subscriptions are negotiated on a school-by-school basis and you can be sure that a university in sub-Saharan Africa isn't paying the same as Yale.
- throwawayjava 8y ago> Academic journals don't result in any value creation for society Journals do provide some value. Editing, printing, and herding academics through the camera-ready process all require non-trivial effort. I would have absolutely no problem with a handful of universities/non-profits/even for-profits devoting some staff to the curation/archival process. And also no problem with those organizations charging others modest fees to fund that work. The problem with journals is that they capture a lot more value than they provide, that value is not distributed back to the people doing the actual work, and the prestige signal/branding provides a strong moat. I REALLY think this is an important point. If we continue to behave like all of this stuff is trivial and obvious to replicate, then we'll be forever stuck with predatory journals like Nature.
- ClassAndBurn 8y agoJournals are based on prestige; researchers give their work for free to a publisher where it is reviewed by other researchers (usually for free) and then published such that other researchers must pay to get the work. It does create a well respected system since publishers want only quality work but the arrangement has become more and more one sided favoring the publishers. TL;DR This isn't that surprising. The cost of publishing is lower than ever and researchers are tired of getting charged for what they give away for free.
- dwc 8y ago> the arrangement has become more and more one sided favoring the publishers Rather than "has become" this is publishers using their leverage to achieve this situation.
- mooneater 8y agoSome heavy hitters on the signatory list: Jeff Dean, Yoshua Bengio, Volodymyr Mnih, Ilya Sutskever, Geoffrey Hinton, Chelsea Finn, Sergey Levine. Would like to see Demis Hassabis and David Silver join the effort. If Deepmind, OpenAI and Berkeley (well represented here) all boycott, that's a huge chunk of key AI researchers.
- adw 8y agoAlso Ian Goodfellow (GANs) and John Langford (Vowpal Wabbit).
- icc97 8y agoNot that I'm for journals, but those heavy hitters typically have their own brand in that of Google or Microsoft etc. Without an external 'brand' of Nature the importance of working for the recognised companies when submitting your research papers goes up.
- make3 8y agoYoshua Bengio doesn't work for a large brand
- robotresearcher 8y agoFor the unfamiliar, Bengio has his own company: https://www.elementai.com/en/who-we-are https://www.elementai.com/en/who-we-are
- deleted 8y ago[deleted]
- make3 8y agoHe still spends most of his time at MILA, of which he is the director, which is UdeM and McGill's shared AI research group. source: I study there. http://mila.quebec/ http://mila.quebec/
- 8y ago
- deleted 8y ago[deleted]
- skywhopper 8y agoAdmittedly I work outside academia (though my wife is a tenured professor) and thus have some unavoidable naïveté about how the culture and community work, but I’m always confused about why universities aren’t just sponsoring the organizational costs of coordinating the review process, and establishing community-driven open journals to replace the rent-seeking corporate journals. What am I missing about the benefits of the status quo keep it so entrenched?
- yread 8y agoUniversities could be in conflict of interest - organizing review of competing (or worse its own) research
- apetresc 8y agoIsn't the peer review process blind, anyway? University affiliation would presumably be one the scrubbed fields, no?
- cli 8y agoThe process is blind, but it's fairly easy to know who the author is just due to the topic at hand. At least for specific subfields.
- Mediterraneo10 8y agoAgreed. Within my own field, there are less than 10 people who are presently working on the same topics as I do. Consequently, we are constantly asked to review each other’s papers, and it is pretty trivial to tell who the author is, either because of the views and interests shown in the paper, or because of language use (i.e. you can tell that the author is a native speaker of the language in which the paper was written, or from their unidiomatic skills in it you can often guess at their native language). The hard part in this is remaining silent and pretending you don’t know anything about this, when you meet your peers at conferences.
- 8y ago
- danmg 8y agoLegality aside, has there ever been an instance of a researcher getting a DMCA notice for posting a paper they wrote on their own website? Even if you got a DMCA, couldn't you just direct link to a scihub search result that contains your paper? Edit: I love it when I find my work on scihub. It means it was good enough to 'steal'.
- alexschiller 8y agoThe American Psychological Association tried and they quickly backtracked after blowback: https://www.the-scientist.com/?articles.view/articleNo/49670/title/Authors-Peeved-by-APA-s-Article-Takedown-Pilot/ https://www.the-scientist.com/?articles.view/articleNo/49670...
- mandor 8y agoMost journals are OK with authors posting a paper on their own website, especially the 'pre-print' (pre-review, pre-layout). See: http://www.sherpa.ac.uk/romeo/index.php http://www.sherpa.ac.uk/romeo/index.php (even Nature is OK for all the pre-prints, arxiv and your own website).
- grawprog 8y agoGood. Science publishing is a ridiculous racket.
- sytelus 8y agoWhile we are on the topic, why are we supporting IEEE? They have so many valuable as well as historical papers behind their paywall. I'm IEEE member and want to find out what can I do about this.
- akvadrako 8y agoWhy are you an IEEE member? Can't you just be not.
- dboreham 8y agoHmm...hadn't thought about this but...same and same.
- zwieback 8y agoIEEE to me is a different case. They do more than publish and IEEE, along with SPIE, although expensive still carries weight for me. I can assume the content is reasonably high quality and well reviewed. In general, though, for-profit publishing will hopefully go away soon. As an engineer I need quality scientific publications without the distorted politics behind it. I could come up with any number of ways to improve on the current situation.
- drpgq 8y agoI find it irritating as an IEEE member that you don't get access to everything with a membership.
- mkohler 8y ago
- tokipin 8y agoJournal reviewing is oversold in my opinion. Just like Wikipedia became a better encyclopedia than traditional encyclopedias, I don't see why open-access approaches can't be at least as efficient at truth-seeking and consesus-building than traditional journals.
- c3534l 8y agoThe whole idea behind journals is peer-review. That's very nearly a wikipedia-style open access system anyway. There's just not a great reason to keep chargind exhorbitant publishing fees when it's published online.
- Vinnl 8y agoThey often are at least as efficient and truth-seeking. Unfortunately, researchers do not pick journals on the basis of who efficient and truth-seeking they are, but on how much they can help their career. (I mean "unfortunately" here in the sense that the circumstances are unfortunate - I can't blame the researchers.)
- stilley2 8y agoEverytime something like this comes up I feel compelled to remind everyone that in most cases I know of (NSF and NIH funded research) US taxpayer funded research must be made freely available. See https://publicaccess.nih.gov/ https://publicaccess.nih.gov/ and https://www.research.gov/research-portal/appmanager/base/desktop?_nfpb=true&_pageLabel=research_node_display&_nodePath=/researchGov/Service/Desktop/AboutPublicAccess.html https://www.research.gov/research-portal/appmanager/base/des... . Don't get me wrong, I think the current publishing model is archiac, exploitative, and needs to change, but this whole "taxpayer funded research should be freely available argument" seems to me to oversimplify the issue and ignore the facts.
- freeone3000 8y agoMostly papers are thrown up on arxiv. I don't see why we should have to pay to host our own papers when we could just host our own papers. Seriously, three grand? How many months of AWS S3 does that pay for?
- stilley2 8y agoIn my experience those high fees usually only apply when publishing open access. In our lab we publish using the free closed access method, and then a free version goes on PubMed (potentially after an embargo). So the research is available for free, the lab still gets the journal prestige, and it doesn't cost the lab anything. This is by no means a perfect system, but it's not as dire as a lot of these articles want us to think.
- Vinnl 8y agoUnfortunately, it does cost your institution a lot, in terms of the subscription to that journal.
- Moter8 8y agoIt's not the hosting, but the publication process, I think. I am not involved in this stuff, just read quite some posts on StackExchange Academia.
- neolefty 8y agoCould ML's open peer review practices be adopted by researchers in other fields?
- cossatot 8y agoCopernicus Publications publishes a host of journals, many on the behalf of non-profit scholarly and scientific societies [1]. For some or all of their journals, including those by the European Geosciences Union, peer review is open and there is a month-long public discussion forum for each manuscript. I've got a paper in review with one of the journals right now, but several of my colleagues have expressed caution with open peer review. I think a lot of scientists are worried about making work public that hasn't been privately vetted during peer review, and maybe being caught with an embarrassing mistake. [1]: https://publications.copernicus.org/ https://publications.copernicus.org/
- mbid 8y ago> In my own field of machine learning, itself an academic descendant of Gauss’s pioneering work, [...]. Yes, and I'm a descendand of Julius Caesar, Confucius and Charlemagne. I won't tell you this when introducing myself though, because so is everybody else. It wouldn't have occured to me that AI researchers have such inferiority complexes that they need to descend to such name-dropping, but I guess I was wrong.
- lccarrasco 8y agoI think he means that the field of ML has been built in part from Gauss's work on algebra and statistics, not that the author is related to Gauss himself.
- Jach 8y agoThe criticism is along the lines of "Yeah basically all fields are academic descendants of Gauss. Euler too." The article's aside is just odd, as is in the first place bringing up Gauss (linking to another Guardian article, maybe the real intent?) and an astronomer both paying another dude (some amount) what sounds like club member fees for letter redistribution.
- lactau 8y agoA small nitpick. Nobody alive is a descendant of Julius Caesar.
- mrybczyn 8y agoNot a "legitimate" descendant, whatever that means after 60 generations. However, he was an aristocrat, and a soldier, in a time when birth control was rarely practiced. I imagine he's got plenty of his DNA in the modern pool.
- gowld 8y agoIt's just a literary segue from their oldest known closed-access publication to the modern day.
- nyxtom 8y agoArXiv forever!
- baxtr 8y agoFormer scientist here, too. People tend to forget what value a Nature article carries for the successful career in academia. The technical “hosting” of Nature and Arxiv might be the same, but the first will buy a lot of fame and potentially a good position at an research institute.
- pedrocr 8y agoNo one forgets that. Everyone just thinks we should solve the chicken-and-egg problem and move that prestige to a set of journals that don't gouge everyone for research that was mostly paid for by public money already and was peer-reviewed by public employees volunteering.
- baxtr 8y agoWell, then you have to come up with a good idea how to change the nature of humans. We thrive on fame as social status currency Not just in science. The equivalent in startups might be a famous VC or whatever. It’s about attaching yourself to a brand.
- aeorgnoieang 8y agoThe problem (in this case) isn't "social status currency". It's that the social status currency system has real (financial) costs too! The value of a prestige journal brand is that it is (supposedly) selective in which papers it publishes; not that it costs a lot of money for people to read those papers.
- Barrin92 8y agothis has nothing to do with some mystical 'nature of humans', we're just stuck in a bad nash equilibrium. Journals thrive on reputation and nobody can individually dissent because they lose out. This is the same mechanism that keeps overpriced tuition fees at ivy league colleges afloat. The answer is to collectively get out by legislating the practice or offering a free public alternative that everybody can immediately jump too. Given that the efforts become more and more organised, (German wholesale boycott of Elsevier) this might be happening sooner than later.
- quizme2000 8y agoA bit off topic but have research journals become a whipping boy for the anti-science crowd? The general public is being spoon fed either very poor quality click-bait "break through/cure cancer" articles or worse. Recently the peer-reviewed != reproducible issue was highlighted. I feel there is a common thread but it could my commonality bias of an HN reader. Is attacking science journals a new trend?
- whatshisface 8y agoSupporting reproduction, complaining about public presentation of science, and disliking the rent-seeking behaviors of journals are all pro-science positions held more often by scientists than any other group.
- quizme2000 8y agoThank-you! I haven't talked to a research scientist in quite awhile. Its one of the reasons I like HN's discussion about these types of issues.
- no1name 8y agoQuite a lot of big hitters in the signatory list. This might be a game changer.
- a-dub 8y agoHonestly, they should just flip the journal funding model on its head. Instead of readers paying for subscriptions, publishing institutions can pay for publication rights. In fact, they could probably take all the subscription money they get from institutions and change nothing but the name (something like "authorship fee", "membership fee" or "institutional publication service fee" or whatever). make the journals open access and not feel any change at all in revenue. Besides, if you look at the money, fancy journals are responsible for a lot of funding that ends up in the coffers of researchers or the institutions that support them, so it really wouldn't be crazy. I read somewhere that a Science/Nature paper in some fields is thought to be worth >$1mm in funding.
- gowld 8y agoYou just invented the open access publishing model. https://www.plos.org/publication-fees https://www.plos.org/publication-fees It seems less profitable than the closed-access regime, since copyright is an economically powerful tool.
- a-dub 8y agoI don't know for sure, but I'd guess that the vast majority of NPG's revenue comes from institutional subscriptions and that the vast majority of those institutions make use of NPG's review/editorial process. I guess there's less of a notion of the accumulation of valuable property in the pay to publish regime. But seriously, just call the subscriptions "membership fees." Revenue would probably dip 5% from non-publisher subscriptions and things would stabilize. It would also neutralize the threat of open access journals gaining more ground.
- bitL 8y agoWhat's the point of journals these days, especially in computing with reproducible results (i.e. authors publish a paper on arxiv, code & dataset on github)?
- nunya213 8y agoReproducible results? lol. A fraction of researchers actually put out code and data freely, definitely not a majority.
- fourthark 8y agoAnd even when it's out, does the code actually run? Designing your code and data so it's truly reproducible is a ton of work, and most researchers are focused on the paper.
- Vinnl 8y agoAs a researcher, you can get a stamp of having had an article published in a certain journal, which helps your career.
- bedros 8y agothe article is about this statement https://openaccess.engineering.oregonstate.edu/home https://openaccess.engineering.oregonstate.edu/home
- blauditore 8y agoOne thing I really wonder is wheather those costs are actually necessary for simple online publications: - Peer review is basically "free", meaning it's done by volunteers without compensation. - Hosting cost is magnitudes lower than publication/reading fees. A few dollars per paper should be enough by far. - Organisation of peer reviewing, editing and so on imposes some costs, but a lot of it could be automated and the remaining cost per published paper should still be quite low. So, would it be technically possible to create such journals? Of course, prestige might be a hurdle, but given some high-profile researchers would join as editors initially, that could work out.
- rlayton2 8y agoPrestige is a made up concept to keep the big players at the top. If the paper is good then it shouldn't matter where it's published.
- fwip 8y agoYou've got it backwards though. A paper isn't good because it's in Nature - it's in Nature because it's good.
- rlayton2 8y agoI'm actually more concerned about the negative (your statement is mostly true), i.e. "if it's in a second-tier journal, it doesn't exist". Sure, some just look for an outlet for their crappy research, but the publication outlet shouldn't be used as a shortcut to evaluating quality.
- fourthark 8y agoI have heard the organization, editing, and publication cost is underestimated because it's really boring work and not as automatible as you might think. But it's still not high enough to justify journal subscription prices. I don't know if typical open access journal fees are lower margin or not.
- Vinnl 8y ago
- Animats 8y agoNature used to be a prestigious journal. But its spinoffs, not so much. When they stuck to bio, they were good. Computer science, not so good. Nature Energy's hype pieces from flaky battery inventors have been mentioned on HN.
- deleted 8y ago[deleted]
- segmondy 8y agoIt's really strange to me that in 2018, with open web and were we are publishing massive codebases on github that folks are still going through a paid/closed publisher to get published.
- fwip 8y agoWhile many eyes may make all bugs shallow; getting those eyes isn't free. That is, the primary value-add of the traditional paper publishing model is trust in the editorial staff and peer-reviewers. Anybody can throw up a repo on github - not everyone can get that repo published in Nature. Journals provide a curated list of exciting, valuable, and reliable research.
- amorphid 8y agoI have a thought... Force car owners, business or consumer, to buy driverless car insurance. No insurance, no autonomous vehicle functionality. Insurance companies would love a new market opportunity, and they'd lead the push to make driverless cars safe enough to insure.
- billfruit 8y agoMay be these journals should move away from publishing and rather be in the business of curation from a source such as arxiv; Arxiv has a wealth of content, may be journals could sift through them and produce periodic curated lists of work that is the most significant.
- Vinnl 8y agoThose journals do not really have an incentive to do that, do they?
- sologoub 8y agoWonder if we are repeating the sentiment from 1958: https://news.ycombinator.com/item?id=17184576 https://news.ycombinator.com/item?id=17184576
- sandGorgon 8y agoWhat is the alternative here - there doesnt seem to be one. Is there a open, free, peer-reviewed/crowd-reviewed journal paper website for artificial intelligence ?
- Pywarrior 8y agoWho runs Nature, why don't you all hold the actual people personally responsible? Change in leadership.
- chrisseaton 8y agoHow are you going to change the leadership of a company you don't own? What are you planning we do? Get together and buy 51% of RELX? It's a $33 billion company.
- stefek99 8y agohttps://en.wikipedia.org/wiki/Aaron_Swartz https://en.wikipedia.org/wiki/Aaron_Swartz "setting it to download academic journal articles systematically from JSTOR using a guest user account issued to him by MIT" INFORMATION IS FREE: "Federal prosecutors later charged him with two counts of wire fraud and eleven violations of the Computer Fraud and Abuse Act, carrying a cumulative maximum penalty of $1 million in fines, 35 years in prison, asset forfeiture, restitution, and supervised release."
- cannotbenull 8y agoI found some blockchain project teams trying to solve the problems in scholarship communication. Pluto(https://pluto.network https://pluto.network) is non-profit organization and trying to make standard academic data for free. Check out this video with English subtitle. (https://youtu.be/IKqUhJZN6Zg https://youtu.be/IKqUhJZN6Zg). And also there is scienceroot project(https://www.scienceroot.com https://www.scienceroot.com) and Orvium. If you wnat to see the details, check this post. (https://hackernoon.com/mapping-the-blockchain-for-science-landscape-546b61bfbd1 https://hackernoon.com/mapping-the-blockchain-for-science-la...)