8 ms·

Scientists struggle to stay grounded after possible gravitational wave signal

80 points by naves 11y ago

KenoFischer 11y ago
LIGO is very serious about doing blinded test runs, even going as far as writing the whole paper before they check whether the signal was real. There's only a couple of people in the entire collaboration who even know when the blinded test runs occur. In any case, this rumor has been going around for a couple of months, so if it's true, we should know soon. Between this and the LHC blip, I'm quite excited about the next couple of months.
- obstbraende 11y ago
  Can you explain that a bit more? I'm fascinated but don't quite get the rationale behind having not even the authors of the paper know the results. What is their process?
  - DavidSJ 11y ago
    It means you have to develop your experimental method prior to seeing the data, so you can't try a dozen different ways of slicing that data up until you get the result you want.
    - marcusgarvey 11y ago
      ...and now knowing this, I wish it were the approach for _all_ scientific research.
      - TeMPOraL 11y ago
        Yeah. This is what's absolutely missing in many scientific fields, and seems to be the reason for so many results turning out to be utter bullshit (hello, psychology). There's a strong push coming from research community to make this approach, which is known as 'study pre-registration', mandatory.
        lucozade 11y ago
        Presumably pre-registration could also help with positive results bias. Assuming the registration was made public, one could mandate that something had to be produced from the study, if only a statement that nothing was found. Wouldn't solve the problem but would presumably aid meta-studies?
      - jessriedel 11y ago
        It's not really feasible for most research (although there are plenty of needed improvements along these lines). Big physics projects are unusual in that they are enormous undertakings of 100s or 1000s of PhDs, and few or no other experiments will take data that can cross-check the results. (For instance, no other machine will produce the conditions at the LHC for several decades at least, which is why they go to the fantastic expense of building two completely separate general purpose detectors.) Devoting ~5 full-time PhDs for the sake of super-duper methodological rigor is doable for LIGO, but not for smaller experiments.
      - amelius 11y ago
        Well, keep in mind that if hundreds of scientists take their chance on the same data using different methods, we basically have the same problem. One of them will be the lucky one that took the approach that shows the favorable results, whether the results are truly there or not.
- lutusp 11y ago
  Another level of experimental blinding would be to avoid conferring with the other observatory (there are two components to LIGO, one in Hanford, WA and one in Livingston, LA) for confirmation/correlation until after one site's evaluation is complete. I don't know whether this is the current practice. In any case, if the observatories are actually able to detect gravitational waves and if there are no theoretical obstacles, eventually two black holes will merge, which will produce a completely fantastic observation consisting of a rapidly rising frequency (as the two BHs circle each other closer and closer, shedding their orbital energy as gravitational waves), followed by a very loud cosmological pop as the BHs finally collapse into each other.
  - Steuard 11y ago
    My impression (from well off to the side in a tangentially related field, mind you) has been that false positives are common enough in the individual detectors that they really do need the comparison with the other location's data as a filter. (Not common enough for an actual coincidence to be at all likely, though.) I'm happy to be corrected if I'm wrong, of course! But I think the system has only been designed to be reliable as a whole.
    - lutusp 11y ago
      > But I think the system has only been designed to be reliable as a whole. Yes, absolutely, I only meant that initial evaluations might be more objective if the two observatories cannot compare notes (at first) -- that might lead to a kind of confirmation bias. Only after one observatory satisfies itself that they have a sufficiently unambiguous signal should they be allowed to correlate with the other observatory's results. Considering the absence of plausible signals to date, this idea might be bending over backwards to avoid observational bias.
      - Steuard 11y ago
        I see your point, but I still have a sense that the false positive rate is just too high. I don't know the actual false positive rate, but I've somehow pictured maybe a dozen or so very brief (<1 sec?) signals per year. Under anything like those circumstances, an accidental coincidence would still be vanishingly unlikely (since there are 3x10^7 sec/year, it would be something like a one in a million chance), but the collaboration can't be expected to do full, detailed analysis of false signals a dozen times a year.
- yk 11y ago
  Source for the blind injection of signals: http://www.ligo.org/news/blind-injection.php http://www.ligo.org/news/blind-injection.php
bootload 11y ago
"In 2014, researchers on another US experiment, called BICEP2, [0] called a press conference to announce the discovery of gravitational waves, but others have since pointed out that the signal could be due entirely to space dust." The BICEP2 team thought they detected gravitational waves with their Cosmic Microwave Background (CMB ) measurements from their Antarctic observatory.[0],[1],[2] By chance the ESA had the Planck observation satellite measuring a broader spectrum. Matching the two observations the BICEP2 measurements were capturing light interference from space dust in-between the observatory and the source. [3] If gravitational waves are found, this will be the biggest thing since Smoot and COBE [4] and confirmation of the Gurth Inflation theory. [5] [0] http://www.theguardian.com/science/2014/mar/17/primordial-gravitational-wave-discovery-physics-bicep http://www.theguardian.com/science/2014/mar/17/primordial-gr... [1] https://www.cfa.harvard.edu/CMB/bicep2/ https://www.cfa.harvard.edu/CMB/bicep2/ [2] https://www.cfa.harvard.edu/CMB/bicep2/science.html https://www.cfa.harvard.edu/CMB/bicep2/science.html [3] http://www.theguardian.com/science/2014/sep/22/gravitational-waves-space-dust http://www.theguardian.com/science/2014/sep/22/gravitational... [4] http://aether.lbl.gov/www/projects/cobe/ http://aether.lbl.gov/www/projects/cobe/ and https://en.wikipedia.org/wiki/George_Smoot https://en.wikipedia.org/wiki/George_Smoot [5] http://www.nytimes.com/2014/03/18/science/space/detection-of-waves-in-space-buttresses-landmark-theory-of-big-bang.html http://www.nytimes.com/2014/03/18/science/space/detection-of...
- privong 11y ago
  > The BICEP2 team thought they detected gravitational waves with their Cosmic Microwave Background (CMB ) measurements from their Antarctic observatory. By chance the ESA had the Planck observation satellite measuring a broader spectrum. Matching the two observations the BICEP2 measurements were capturing light interference from space dust in-between the observatory and the source. This is a bit different, though. BICEP2 was claiming a detection of a polarization signal in the cosmic microwave background, which if present, would point to existence of primorial gravitational waves resulting from cosmic inflation. But, as you note, the multi-wavelength observations of Planck (not taken by chance, but part of a carefully planned all-sky survey with somewhat coarser resolution compared to BICEP2) showed that the polarization signal measured by BICEP2 was consistent with foreground dust, rather than primordial gravitational waves. > If gravitational waves are found, this will be the biggest thing since Smoot and COBE and confirmation of the Gurth Inflation theory. It will be huge, but the gravitational waves LIGO would detect are not primordial or from inflation, but rather from supernovae and compact binary mergers (neutron star pairs, stellar-mass black hole pairs, etc.). So a LIGO gravitational wave detection would not provide any evidence for or against inflation. But it would tell us a lot about stellar explosions and the evolution of compact binaries. EDIT: I should note that LIGO could detect or place constraints on some "exotic" gravitational wave sources – cosmic strings, for example.
deleted 11y ago
[deleted]
cmpb 11y ago
Off-topic, but if anyone has a chance to go tour the LIGO facilities, definitely take it! It is a very interesting experience, and highly educational, even for those that don't have a physics background. I've been to the one in Livingston while studying at LSU, and even after several years, I'm still astounded at the precision of the technology used there.

80 points by naves 11y ago

LIGO is very serious about doing blinded test runs, even going as far as writing the whole paper before they check whether the signal was real. There's only a couple of people in the entire collaboration who even know when the blinded test runs occur. In any case, this rumor has been going around for a couple of months, so if it's true, we should know soon. Between this and the LHC blip, I'm quite excited about the next couple of months.

obstbraende 11y ago
Can you explain that a bit more? I'm fascinated but don't quite get the rationale behind having not even the authors of the paper know the results. What is their process?
- DavidSJ 11y ago
  It means you have to develop your experimental method prior to seeing the data, so you can't try a dozen different ways of slicing that data up until you get the result you want.
  - marcusgarvey 11y ago
    ...and now knowing this, I wish it were the approach for _all_ scientific research.
    - TeMPOraL 11y ago
      Yeah. This is what's absolutely missing in many scientific fields, and seems to be the reason for so many results turning out to be utter bullshit (hello, psychology). There's a strong push coming from research community to make this approach, which is known as 'study pre-registration', mandatory.
      - lucozade 11y ago
        Presumably pre-registration could also help with positive results bias. Assuming the registration was made public, one could mandate that something had to be produced from the study, if only a statement that nothing was found. Wouldn't solve the problem but would presumably aid meta-studies?
    - jessriedel 11y ago
      It's not really feasible for most research (although there are plenty of needed improvements along these lines). Big physics projects are unusual in that they are enormous undertakings of 100s or 1000s of PhDs, and few or no other experiments will take data that can cross-check the results. (For instance, no other machine will produce the conditions at the LHC for several decades at least, which is why they go to the fantastic expense of building two completely separate general purpose detectors.) Devoting ~5 full-time PhDs for the sake of super-duper methodological rigor is doable for LIGO, but not for smaller experiments.
    - amelius 11y ago
      Well, keep in mind that if hundreds of scientists take their chance on the same data using different methods, we basically have the same problem. One of them will be the lucky one that took the approach that shows the favorable results, whether the results are truly there or not.
lutusp 11y ago
Another level of experimental blinding would be to avoid conferring with the other observatory (there are two components to LIGO, one in Hanford, WA and one in Livingston, LA) for confirmation/correlation until after one site's evaluation is complete. I don't know whether this is the current practice. In any case, if the observatories are actually able to detect gravitational waves and if there are no theoretical obstacles, eventually two black holes will merge, which will produce a completely fantastic observation consisting of a rapidly rising frequency (as the two BHs circle each other closer and closer, shedding their orbital energy as gravitational waves), followed by a very loud cosmological pop as the BHs finally collapse into each other.
- Steuard 11y ago
  My impression (from well off to the side in a tangentially related field, mind you) has been that false positives are common enough in the individual detectors that they really do need the comparison with the other location's data as a filter. (Not common enough for an actual coincidence to be at all likely, though.) I'm happy to be corrected if I'm wrong, of course! But I think the system has only been designed to be reliable as a whole.
  - lutusp 11y ago
    > But I think the system has only been designed to be reliable as a whole. Yes, absolutely, I only meant that initial evaluations might be more objective if the two observatories cannot compare notes (at first) -- that might lead to a kind of confirmation bias. Only after one observatory satisfies itself that they have a sufficiently unambiguous signal should they be allowed to correlate with the other observatory's results. Considering the absence of plausible signals to date, this idea might be bending over backwards to avoid observational bias.
    - Steuard 11y ago
      I see your point, but I still have a sense that the false positive rate is just too high. I don't know the actual false positive rate, but I've somehow pictured maybe a dozen or so very brief (<1 sec?) signals per year. Under anything like those circumstances, an accidental coincidence would still be vanishingly unlikely (since there are 3x10^7 sec/year, it would be something like a one in a million chance), but the collaboration can't be expected to do full, detailed analysis of false signals a dozen times a year.
yk 11y ago
Source for the blind injection of signals: http://www.ligo.org/news/blind-injection.php http://www.ligo.org/news/blind-injection.php

bootload 11y ago

"In 2014, researchers on another US experiment, called BICEP2, [0] called a press conference to announce the discovery of gravitational waves, but others have since pointed out that the signal could be due entirely to space dust." The BICEP2 team thought they detected gravitational waves with their Cosmic Microwave Background (CMB ) measurements from their Antarctic observatory.[0],[1],[2] By chance the ESA had the Planck observation satellite measuring a broader spectrum. Matching the two observations the BICEP2 measurements were capturing light interference from space dust in-between the observatory and the source. [3] If gravitational waves are found, this will be the biggest thing since Smoot and COBE [4] and confirmation of the Gurth Inflation theory. [5] [0] http://www.theguardian.com/science/2014/mar/17/primordial-gravitational-wave-discovery-physics-bicep http://www.theguardian.com/science/2014/mar/17/primordial-gr... [1] https://www.cfa.harvard.edu/CMB/bicep2/ https://www.cfa.harvard.edu/CMB/bicep2/ [2] https://www.cfa.harvard.edu/CMB/bicep2/science.html https://www.cfa.harvard.edu/CMB/bicep2/science.html [3] http://www.theguardian.com/science/2014/sep/22/gravitational-waves-space-dust http://www.theguardian.com/science/2014/sep/22/gravitational... [4] http://aether.lbl.gov/www/projects/cobe/ http://aether.lbl.gov/www/projects/cobe/ and https://en.wikipedia.org/wiki/George_Smoot https://en.wikipedia.org/wiki/George_Smoot [5] http://www.nytimes.com/2014/03/18/science/space/detection-of-waves-in-space-buttresses-landmark-theory-of-big-bang.html http://www.nytimes.com/2014/03/18/science/space/detection-of...

privong 11y ago
> The BICEP2 team thought they detected gravitational waves with their Cosmic Microwave Background (CMB ) measurements from their Antarctic observatory. By chance the ESA had the Planck observation satellite measuring a broader spectrum. Matching the two observations the BICEP2 measurements were capturing light interference from space dust in-between the observatory and the source. This is a bit different, though. BICEP2 was claiming a detection of a polarization signal in the cosmic microwave background, which if present, would point to existence of primorial gravitational waves resulting from cosmic inflation. But, as you note, the multi-wavelength observations of Planck (not taken by chance, but part of a carefully planned all-sky survey with somewhat coarser resolution compared to BICEP2) showed that the polarization signal measured by BICEP2 was consistent with foreground dust, rather than primordial gravitational waves. > If gravitational waves are found, this will be the biggest thing since Smoot and COBE and confirmation of the Gurth Inflation theory. It will be huge, but the gravitational waves LIGO would detect are not primordial or from inflation, but rather from supernovae and compact binary mergers (neutron star pairs, stellar-mass black hole pairs, etc.). So a LIGO gravitational wave detection would not provide any evidence for or against inflation. But it would tell us a lot about stellar explosions and the evolution of compact binaries. EDIT: I should note that LIGO could detect or place constraints on some "exotic" gravitational wave sources – cosmic strings, for example.

deleted 11y ago

[deleted]

cmpb 11y ago

Off-topic, but if anyone has a chance to go tour the LIGO facilities, definitely take it! It is a very interesting experience, and highly educational, even for those that don't have a physics background. I've been to the one in Livingston while studying at LSU, and even after several years, I'm still astounded at the precision of the technology used there.