I work in ML research and I used to do experimental physics. I'd agree that specific results in papers can be hard or impossible to reproduce, but that never re

I work in ML research and I used to do experimental physics. I'd agree that specific results in papers can be hard or impossible to reproduce, but that never really bothers me because at least in my work, the specific experimental result is rarely material to why I'm interested in the paper. It's more of a demo, and like a demo, you know its orchestrated to look good. What I'm interested in is what is the mechanism behind the advance and do I think its applicable or relevant to what I'm doing. If the paper is really just a random observation of something that worked better, without a causal explanation, it's not very interesting, but I don't see those often. Maybe health research is very different, and people are latchjng on to surprising results they find in papers, but I doubt it's a big problem in academia, much more likely in the media. If I was a doctor and saw an out of the blue study claiming a surprising result, I'd discount it accordingly. If I saw a causal explanation with evidence, I'd give it closer scrutiny and follow up if it seemed relevant to me. That is how research works in my experience.