3 ms·
Uhh, the elephant in the room for CS/ML publications is that a lot of people hardcore lie on their results page since almost no one tries to reproduce anything.
by Der_Einzige 2y ago
Uhh, the elephant in the room for CS/ML publications is that a lot of people hardcore lie on their results page since almost no one tries to reproduce anything.
I've witnessed folks straight out of their undergrad with literally 1 workshop paper at an ACL track conference get thrown 150K starting offers by no-name startups. Imagine how important it must be to get your first NeurIPS main conference publication. The ROI likely amounts to 300K+ per year, so if anything, I'm surprised that cheating/lies in the academy aren't even more brazen. Dynamics like this are why both Stanford and Harvard's presidents had to step-down for gross academic plagerism/misconduct.
1. https://www.science.org/doi/10.1126/science.359.6377.725 https://www.science.org/doi/10.1126/science.359.6377.725 (2018)
2. https://www.technologyreview.com/2020/11/12/1011944/artificial-intelligence-replication-crisis-science-big-tech-google-deepmind-facebook-openai/ https://www.technologyreview.com/2020/11/12/1011944/artifici... (2020)
3. https://www.nature.com/articles/d41586-023-03817-6 https://www.nature.com/articles/d41586-023-03817-6 (2023, Nature)
- godelski 2y agoI've seen it happen even when source is available. I've also seen a lot of really terrible math mistakes in ML because people don't understand the underlying assumptions of the metrics they use. The rot is everywhere because people forgot the goal. We all get caught up in the metrics that are used to judge us that we forgot what we were proxying in the first place.