3 ms·
not only social sciences. Except for the very visible areas of ML, the same happens in actual science and engineering... Example: the thousands of fraudulent X
by fock 4y ago
not only social sciences. Except for the very visible areas of ML, the same happens in actual science and engineering...
Example: the thousands of fraudulent XRD spectra of made-up compounds.
- bonoboTP 4y agoEven in ML, it's common knowledge that the long tail of papers demonstrate brittle effects that don't really replicate/generalize and often do uncomparable evaluations, fiddle with hyperparameters to fit the test data, use various evaluation tricks (Goodhart's Law) to improve the metrics, sometimes don't cite better prior work, etc. etc. Industry people definitely know not to just take a random ML paper and believe that it has any use for applications. This isn't to say there are no good works, but in a field that produces >10,000 papers per year, the bulk of it can't be all that great, but academics have to keep their jobs, PhD students have to graduate etc. So everyone keeps pretending.
- fock 4y agothose papers are not "very visible" ML (NeurIPS and co.), but domainspecific "applications" and there's tons of it (as I am acutely aware).
- bonoboTP 4y agoWhat I wrote also applies to most papers at top tier conferences, like Neurips and CVPR. There are thousands of papers published per year even just in those top conferences. What gets picked up by the media or even just reaches the HN crowd is just a small tip of the iceberg.
- Qem 4y ago> Example: the thousands of fraudulent XRD spectra of made-up compounds. Interesting, didn't hear about this case before. Can you provide a link?
- fock 4y agohttps://news.ycombinator.com/item?id=32280756 https://news.ycombinator.com/item?id=32280756 as discussed here ;)