3 ms·
I agree, but all of that is also true of a lot of papers in a lot of other fields. I've seen papers about better complex simulations of natural processes - I do
by TallGuyShort 7y ago
I agree, but all of that is also true of a lot of papers in a lot of other fields. I've seen papers about better complex simulations of natural processes - I don't think I've ever seen one with complete code. I've seen papers that described the experiment and the results of their analysis with hand-wavy descriptions of which algorithms and statistical methods they used - but no code.
My point is that the findings of this article are not necessarily indicative of machine learning being junk science, but it's more likely indicative of a systemic problem with all research, where there's incentive to publish the paper, but not the data, code, models, or even generally enough information of any kind to successfully replicate the experiment more often than 10-15% of the time.
I remember when LIGO first detected a black-hole collision, they released the raw data and the Jupyter notebooks that were actually used to go from the raw data to the published results. Everyone was floored. It shouldn't be that amazing - it should be the standard. For every field.