5 ms·
The article resonates with me a lot, and reminded of the couple of years of personal experience of applied machine learning in another field of science; because
by hmwhy 6y ago
The article resonates with me a lot, and reminded of the couple of years of personal experience of applied machine learning in another field of science; because the points made are exactly what I encountered then and made me quit my job.
There is a lot of machine learning snake oil out there, even in the most prestigious journals. The issue is that most people wouldn't know about it until they actually try to reproduce those results (if they can at all in the first place).
It is a huge issue now that machine learning has made its way into every field of research, while almost nobody has adequate training to even gauge what is and isn't questionable. Even the grad students who end up doing all the "hard work" usually don't have much of an idea of what exactly they hacked together and how to access its validity.
The crazy thing is that, despite all of this, you can still get ahead if you have know just slightly more buzzwords than others and have a paper or two (don't have to be first author) to show for. At the end of the day, I suppose all of this is nothing new (machine learning practitioners are the new SEO consultants?), and I'm just venting my frustration for not being able to accept this is how things work.
- ehnto 6y ago> (machine learning practitioners are the new SEO consultants?) If we want to continue the snake oil salesman analogy, SEO consultants were soothsayers, attempting to divine meaning from The Algorithm of Great Google. Now the apothecaries have begun to legitimize their potion making into tonics, but there is still room for a charlatan or two to push hodgepodge concoctions onto the unsuspecting Product Manager or CTO.
- Radim 6y ago> There is a lot of machine learning snake oil out there, even in the most prestigious journals. I'm also deep in that industry, and can confirm this is the case. Even for research coming out of the biggest names: FB, Google, etc. The problem of "story over substance" has been observed by many; it even has a name in the context of ML, the "Mummy Effect" [0]. Like you, I'm segueing out of "AI" because of the unfavourable signal-to-noise ratio. Frustrating indeed, but life is too short to try to outcompete pocketless BS generators, or clean up their mess on my dime. [0] https://rare-technologies.com/mummy-effect-bridging-gap-between-academia-industry/#the_mummy_effect https://rare-technologies.com/mummy-effect-bridging-gap-betw...
- heisenbit 6y agoPublish or perish - there are a lot of half baked ideas sold by researchers, particularly in fast advancing fields. As long as we as a society value output volume and showmanship over substance it continues. The best defense these days where distribution is instantaneous and global are critical readers and investments into education.
- Siira 6y ago“Investment into education” translates into stupid politicians dumping more money into the status quo with no indication that this will better their own metrics (which are mostly irrelevant in the first place as they don’t teach any useful cognitive skills).
- Upvoter33 6y agoThis isn't really much of a case of publish or perish - the two senior authors, to wit, are tenured, established, and very well off. They could each not write a paper for the rest of their careers and be fine. It's really a case of having a small idea - which, upon deeper examination, doesn't really work - and selling it way too hard. The (senior) authors should know better.