I am no expert on non-physics sciences, but recently I am starting to worry that the way we do science today depends on ideal experimental and theoretical condi

I am no expert on non-physics sciences, but recently I am starting to worry that the way we do science today depends on ideal experimental and theoretical conditions.[1] In physics we see 2 sigma events vanish all the time, just last week we have seen the disappearance of the di-photon excess at the LHC. That was an excess that two experiments did see independently to a significance better than p < 5%, with well controlled statistics and well understood experimental set up. Similar the Auger anisotropy [2] was on the cover of Science and did just vanish. Again an effect with p < 5%, using blinded data analysis. My understanding is, that this data analysis is quite a bit better than the standard in other fields, and it does not really work reliably in physics, a field that is to some extend defined by ideal experimental conditions. (The saving grace for physics is, that the Standard Model and General Relativity tie different experiments together. So an experiment in conflict with expectations is immediately in conflict with a lot of other experiments.) To be clear what I am saying, I do not think that other sciences are a waste of time in general. But I think that their methods are too much inspired by physics. I think that we need a much more cautious approach to the scientific method and a lot more robust tools. In a way, it looks to me like building a car by taking inspiration from Formula 1, when the task at hand is crossing rivers and navigating rough terrain. [1] Previously I assumed that it is just complicated, that all I see from studies is the headline "Coffee is/is not healthy" and if one would look into the studies I would find that a better picture is Coffee has some positive and some negative effects. [2] An anisotropy in the arrival directions of ultra-high-energy cosmic rays. https://arxiv.org/abs/0711.2256 https://arxiv.org/abs/0711.2256