6 ms·
Troubling Trends in Machine Learning Scholarship
- gcb0 8y agoisn't that true for absolutely everything in CS?
- noelwelsh 8y agoDoing bad science is an issue that all scientific fields need to be vigilant about (e.g. the "replication crisis" in psychology). It is particularly relevant to machine learning at this point in time as i) the field is growing extremely rapidly and ii) there is lots of $s on the table for those who achieve commercialisable results. So the incentives to do bad science are increasing while the checks against are weakening. These forces are not at play in, say, theoretical CS.
- ItsMe000001 8y ago> e.g. the "replication crisis" in psychology Just an aside, it's not just psychology. When you google "replication crisis" most articles say "... in science" for good reason. https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis When mentioning "replication crisis" I should probably add articles like this one too: https://www.insidehighered.com/news/2018/04/05/scholar-challenges-idea-reproducibility-crisis https://www.insidehighered.com/news/2018/04/05/scholar-chall...
- thanatropism 8y agoThat article is incredibly frightening. > to conclude that, although misconduct and questionable research methods do occur in “relatively small” frequencies, which is kind of expected in a career track that doesn't exactly come with hedge fund-type money -- why would many people be dishonest for grants? > there is “no evidence” that the issue is growing. which might mean that the issue isn't in how science is practiced nowadays, but that the scientific process (grants, publication, peer review, research programs) has always been broken. It's not clear that any of these is the case, but it's depressing to see "well, people are generally not crooks and the game has always been played this way" as an optimistic view of the replication crisis.
- nonbel 8y agoWhile outright fraud is probably rare, "questionable research methods/practices" are standard. Most people doing them dont even know there is something wrong with it. Pick any biomed or social science paper and I will quickly find some indication of them (lack of blinding, weird unexplained sample size changes, missing controls, failure to show a direct comparison of the variables you claim are related, etc).
- joe_the_user 8y ago"Failure to identify the sources of empirical gains, e.g. emphasizing unnecessary modifications to neural architectures when gains actually stem from hyper-parameter tuning." I think everyone is looking for gains coming from architecture modification since more power and clever hyperparameter tuning can only go so far. But given that any of systems requires hyperparameter tuning and that requires a lot of time, it is inherently hard to distinguish between this and novel architectures. If someone says "X worked for me but Y didn't", tuning could always be the explanation. It seems like one could only really scientifically distinguish two architectures if the hyperparameter methodology was fixed. But that's about the opposite of now as far as my amateur exposure to the field goes.
- AstralStorm 8y agoYou could also try to devise optimal hyperparameters by genetic algorithm (or similar global optimizer) for every architecture with a big fixed number of generations. That should be apples to apples comparison... Or best to best. At the same time fragility of hyperparameters can be evaluated. I really appreciate the detailed analysis in the article.
- abhgh 8y agoToday you can do much better - there is a whole active and evolving sub-field of solving hyperparameter tuning as an optimization problem. An ex is performing bayesian optimization using gaussian processes. I hope the research from the community becomes mainstream soon.
- srean 8y agoOne could optimize test error over hyperparameters by various methods. But at that point does test or validation error continue to provide the functionality that a test or validation set is supposed to provide ?
- nonbel 8y agoDo you really consider this an obstacle? Obviously you need to have a validation set for the optimization and a new/unused test set for estimating out of sample performance.
- drewm1980 8y ago"The ubiquity of this issue is evidenced by the paper introducing the Adam optimizer [35]. In the course of introducing an optimizer with strong empirical performance, it also offers a theorem regarding convergence in the convex case, which is perhaps unnecessary in an applied paper focusing on non-convex optimization. The proof was later shown to be incorrect in [63]." I strongly disagree here. Researchers, please keep trying to prove that your proposed optimizers are correct, even if you can only manage to prove it for a subclass of problems, and even if it risks publishing an accidentally incorrect proof.
- AstralStorm 8y agoYes, but why do it on paper when you can use an automated prover instead, since you're proving properties of an algorithm? It smells of laziness...
- shoo 8y agocan you cite any non-trivial examples where people writing papers include machine-checked proofs , in fields of study unrelated to the study of machine-checked proofs?
- blt 8y agoYeah I read that proof as more of a reassurance than an emphasized part of the paper. If the method does not even work on convex problems, it's highly suspicious as a tool for nonconvex problems.
- cosmic_ape 8y agoAdam may be not the clearest example. I'd say the clearest example is the original GAN paper, where "theorems" where completely unrelated to what they were doing and trivial at the same time. >>and even if it risks publishing an accidentally incorrect proof. No. If a researcher has even a slightest doubt about the correctness of their proof, they should either spend the time to verify it or talk to an expert. There is no lack of experts in convex optimization in particular. Or, instead of publishing the proof, one could just say "we believe this is provable by these methods". But one can not try and shift the burden of verification to the readers, while claiming the results. This is not how trust is built.
- tw1010 8y agoThis is only going to get worse. I see literally no counter force putting any kind of barrier against the incentive to e.g. overfit in order to get exaggerated results. The lucrative reward is too large not to fall for the use of shady tactics (which are sometimes not even consciously obvious; not being critical enough of your own work is a form of model loosening which acts in favour of more impressive results).
- chriskanan 8y agoA major factor is that machine learning is so conference deadline driven. Does the main result look good enough? Submit it! Ablation studies are the last thing you do, and often you are pressed for time due to the deadline. Since so much work is done by PhD students, they are incentivized to get their work out as fast as possible because they need to graduate and papers in top-conferences play a huge factor in getting jobs in academia and industry. I'm aware of multiple papers by top labs where their state-of-the-art results are really the result of some very minor change (e.g., some pre-processing technique, bigger images) that was swept under the rug in their paper. All the math and the complex model had minimal impact on the actual numeric result, with the actual reason being unpublishable in a top venue.
- at-fates-hands 8y agoYour entire first paragraph makes me cringe. Coming from an academic/research background, all of the things you mentioned goes against everything I was taught in terms of being rigorous with your research. Is this more evidence we have really entered the "click baiting" era? Where the end result of the research is now secondary to getting published and into big conferences to get some notoriety?
- michaelt 8y agoall of the things you mentioned goes against everything I was taught in terms of being rigorous with your research. There are certain areas where "the system" asks people to do things right - but provides vast rewards for doing things wrong, if you can avoid detection. If you're a pro cyclist who isn't doping, good for you! But there's no medal for finishing fourth; you'll be rewarded with medals and sponsorship cash for doping undetectably. Academics whose rigorous papers aren't quite getting into top-tier journals are in a similar situation.
- tjpaudio 8y ago"Mathiness" as the author puts it is one of my main complaints of many academic papers, this is not at all limited to ML. I think we really need to get away from typical math notation and start to present the calculations as code. This is more readable to the modern practicing professionally and the algorithm's degree of parallelization opportunities (or troubling lack thereof) would be more clear.
- gota 8y agoI've struggled with that a couple of times. Anecdotal evidence: I wrote a technical report describing an algorithm and its novel applications, etc. A good compromise between a high-level description of the concepts and a documentation of the actual code (and thus a guide for implementing it), in my opinion. I tried to turn that into a scientific paper and it was rejected. Changed the entire notation to a "math-y" notation that actually "obfuscates or impresses rather than clarifies" (direct quote from the article). Accepted with positive comments on the very same aspects that got it rejected in the first place. This in a Computer Science conference.
- ssivark 8y agoThe paper needs to explain what is being done, why and under what assumptions. As long as each of those are clearly explained, in my experience, it doesn't really matter whether each of them is explained in words/math/code. Yes, one form might be a little bit more work for people from a certain background, but that's a hump you can get over relatively easily after reading a few papers on the topic. The worst is when you have to guess at what the authors might be doing and why they're doing it that way, and what assumptions they might be operating under... and all they've done is dumped their results within the page limit and before the conference submission deadline.
- nmca 8y agoI skimmed this, found it troubling and started trying to make a checklist of does/don'ts to apply to my own work - https://github.com/N-McA/ml-paper-checklist/blob/master/README.md https://github.com/N-McA/ml-paper-checklist/blob/master/READ...
- sctb 8y agoYesterday: https://news.ycombinator.com/item?id=17497235 https://news.ycombinator.com/item?id=17497235.