4 ms·

AFAIK the crisis in psychology applies generally, but this is just QC? Are there other physics-based reproducibility issues?

by Chris2048 6y ago

klyrs 6y ago
One big one comes to mind... nobody else has an LHC. It's not hard to imagine a systematic issue in an experiment producing a wrong "5 sigma" result that isn't caught until the world has an equivalent / higher energy beam to play with. This particular issue seems to be of a similar nature. You've got a research group who made and tested a device, and nobody else has duplicated that (not sure -- but if there's patents covering their fab, there could be Problems for anybody seeking to reproduce the result).
- andi999 6y ago
  I don't know about lhc, but at Hera they had two independent groups running similiar experiments at different sides of the ring without talking to each other. This was exactly to adress the problem you mentioned.
  - pa7x1 6y ago
    It's the same in LHC there are two experiments that run independently, ATLAS and CMS. For exactly this reason. The Higgs was discovered by both independently with higher than 5 sigma each. The combined sigma of the 2 experiments was 7-ish if my memory serves correct. EDIT: My memory serves correct because sqrt(5^2 + 5^2) ~ 7
- evanb 6y ago
  For very big and expensive projects that have an infrastructure part and a science part, like the LHC’s tunnel vs detectors, we try to avoid this by having multiple, separately-designed detectors at different intersection points. Hence the CMS and ATLAS detectors. Even though they share a beam, the rest of their systematics should be independent. They even unblinded their Higgs results together, to ensure neither used the other’s result as prior knowledge.
- tremon 6y ago
  There are multiple groups at different universities involved in analyzing the LHC results though. Yes, there is only one machine performing the experiments but the data processing effort is truly global: https://wlcg-public.web.cern.ch/tier-centres https://wlcg-public.web.cern.ch/tier-centres I haven't been able to find a simple list of all the universities and research groups working on the data though, but the tools to analyze the data are an open effort too: https://root.cern/about/license/ https://root.cern/about/license/

AFAIK the crisis in psychology applies generally, but this is just QC? Are there other physics-based reproducibility issues?

klyrs 6y ago

One big one comes to mind... nobody else has an LHC. It's not hard to imagine a systematic issue in an experiment producing a wrong "5 sigma" result that isn't caught until the world has an equivalent / higher energy beam to play with. This particular issue seems to be of a similar nature. You've got a research group who made and tested a device, and nobody else has duplicated that (not sure -- but if there's patents covering their fab, there could be Problems for anybody seeking to reproduce the result).

andi999 6y ago
I don't know about lhc, but at Hera they had two independent groups running similiar experiments at different sides of the ring without talking to each other. This was exactly to adress the problem you mentioned.
- pa7x1 6y ago
  It's the same in LHC there are two experiments that run independently, ATLAS and CMS. For exactly this reason. The Higgs was discovered by both independently with higher than 5 sigma each. The combined sigma of the 2 experiments was 7-ish if my memory serves correct. EDIT: My memory serves correct because sqrt(5^2 + 5^2) ~ 7
evanb 6y ago
For very big and expensive projects that have an infrastructure part and a science part, like the LHC’s tunnel vs detectors, we try to avoid this by having multiple, separately-designed detectors at different intersection points. Hence the CMS and ATLAS detectors. Even though they share a beam, the rest of their systematics should be independent. They even unblinded their Higgs results together, to ensure neither used the other’s result as prior knowledge.
tremon 6y ago
There are multiple groups at different universities involved in analyzing the LHC results though. Yes, there is only one machine performing the experiments but the data processing effort is truly global: https://wlcg-public.web.cern.ch/tier-centres https://wlcg-public.web.cern.ch/tier-centres I haven't been able to find a simple list of all the universities and research groups working on the data though, but the tools to analyze the data are an open effort too: https://root.cern/about/license/ https://root.cern/about/license/