5 ms·
A major factor is that machine learning is so conference deadline driven. Does the main result look good enough? Submit it! Ablation studies are the last thing
by chriskanan 8y ago
A major factor is that machine learning is so conference deadline driven. Does the main result look good enough? Submit it! Ablation studies are the last thing you do, and often you are pressed for time due to the deadline. Since so much work is done by PhD students, they are incentivized to get their work out as fast as possible because they need to graduate and papers in top-conferences play a huge factor in getting jobs in academia and industry.
I'm aware of multiple papers by top labs where their state-of-the-art results are really the result of some very minor change (e.g., some pre-processing technique, bigger images) that was swept under the rug in their paper. All the math and the complex model had minimal impact on the actual numeric result, with the actual reason being unpublishable in a top venue.
- at-fates-hands 8y agoYour entire first paragraph makes me cringe. Coming from an academic/research background, all of the things you mentioned goes against everything I was taught in terms of being rigorous with your research. Is this more evidence we have really entered the "click baiting" era? Where the end result of the research is now secondary to getting published and into big conferences to get some notoriety?
- michaelt 8y agoall of the things you mentioned goes against everything I was taught in terms of being rigorous with your research. There are certain areas where "the system" asks people to do things right - but provides vast rewards for doing things wrong, if you can avoid detection. If you're a pro cyclist who isn't doping, good for you! But there's no medal for finishing fourth; you'll be rewarded with medals and sponsorship cash for doping undetectably. Academics whose rigorous papers aren't quite getting into top-tier journals are in a similar situation.
- lord_ring_111 8y agoThis is exactly why professors in universities has tenures and were govt funded. The equation has changed considerably with private employment/funding out growing the older system. Not sure when bew system will reach its equilibrium. Most research scientist jobs need publications and phd, so there is huge incentive to get something published.
- 3pt14159 8y agoI also struggle to understand why PhD led research in AI is considered so highly. Only very rarely does is hold a candle to team lead research in the private sector.