7 ms·
A bit disheartening that there is a reproducibility crisis not only in psychology (and maybe social sciences in general?), but also physics... On the other han
by FabHK 6y ago
A bit disheartening that there is a reproducibility crisis not only in psychology (and maybe social sciences in general?), but also physics...
On the other hand, after the breakthroughs in physics in the first half (third?) of the 20th century and the stagnation in the latter half of that century, it seems to me (as a layperson) that the number of anomalies in physics seems to be increasing, so that maybe we'll transition from a period of "normal" science to a scientific revolution again soon (in Kuhnian terms). Exciting!
- Chris2048 6y agoAFAIK the crisis in psychology applies generally, but this is just QC? Are there other physics-based reproducibility issues?
- klyrs 6y agoOne big one comes to mind... nobody else has an LHC. It's not hard to imagine a systematic issue in an experiment producing a wrong "5 sigma" result that isn't caught until the world has an equivalent / higher energy beam to play with. This particular issue seems to be of a similar nature. You've got a research group who made and tested a device, and nobody else has duplicated that (not sure -- but if there's patents covering their fab, there could be Problems for anybody seeking to reproduce the result).
- andi999 6y agoI don't know about lhc, but at Hera they had two independent groups running similiar experiments at different sides of the ring without talking to each other. This was exactly to adress the problem you mentioned.
- pa7x1 6y agoIt's the same in LHC there are two experiments that run independently, ATLAS and CMS. For exactly this reason. The Higgs was discovered by both independently with higher than 5 sigma each. The combined sigma of the 2 experiments was 7-ish if my memory serves correct. EDIT: My memory serves correct because sqrt(5^2 + 5^2) ~ 7
- evanb 6y agoFor very big and expensive projects that have an infrastructure part and a science part, like the LHC’s tunnel vs detectors, we try to avoid this by having multiple, separately-designed detectors at different intersection points. Hence the CMS and ATLAS detectors. Even though they share a beam, the rest of their systematics should be independent. They even unblinded their Higgs results together, to ensure neither used the other’s result as prior knowledge.
- tremon 6y agoThere are multiple groups at different universities involved in analyzing the LHC results though. Yes, there is only one machine performing the experiments but the data processing effort is truly global: https://wlcg-public.web.cern.ch/tier-centres https://wlcg-public.web.cern.ch/tier-centres I haven't been able to find a simple list of all the universities and research groups working on the data though, but the tools to analyze the data are an open effort too: https://root.cern/about/license/ https://root.cern/about/license/
- api 6y agoYou get what you incentivize. We incentivize quantity of papers and gaming the citation system, and that's going to drive down quality.
- neuronic 6y agoWe incentivize anything that gets grants approved. It's $$$ all the way.
- fpoling 6y agoThe problem is not reproducibility, but rather omission of data and details without which it is not apparent that alternative explanations are possible.
- 6gvONxR4sf7o 6y ago> A bit disheartening that there is a reproducibility crisis not only in psychology (and maybe social sciences in general?), but also physics... It seems to be across science in general, though it's better or worse in different communities. ML is definitely struggling with this as a field.