5 ms·

"the controversy all boils down to a misunderstanding about LIGO's methods for analyzing its data." This plus the mentioned secrecy is the modus operandi in the

"the controversy all boils down to a misunderstanding about LIGO's methods for analyzing its data." This plus the mentioned secrecy is the modus operandi in the physics community. No open data, no open calculation how you reached a certain theoretical result and no open code with which the simulation/analysis is done. There is also the case to be made for a reproducibility crisis in the hard sciences.

punchingwater 8y ago
the article states: > “But Reitze counters that the complete data from that first run is already available online. According to Shoemaker, this includes the relevant time series data and the programs used, but "it's not a trivial matter to use them." Caltech even held a training workshop on how to deal with gravitational-wave data. That's a pretty far cry from asking the physics community to take its analysis on faith...” Are you refuting this statement? To me, it looks like the data/analysis are open, but not yet independently verified due to difficult nature of problem space.
- smueller1234 8y ago
  I used to work in particle/astrophysics. By and large people weren't opposed to sharing their data and software. The degree of specific expertise it took to get results was staggerng though. It takes significant technical infrastructure and expertise to even just rerun an analysis on a big corpus of data. At that stage, you haven't even validated squat, just pushed some buttons to run other people's logic. Doing this properly takes person years of effort with little to no reward. In practice, there's usually at least two big experiments (not necessarily quite of the same generation) that were built by different groups which corroborate results. This is currently the best defense against big mistakes.
- splittingTimes 8y ago
  I did not read the article till the very last paragraph. My bad. Of course, I do not refute it. So in this particular case they went open source. However, from my experience in years of publishing to and reviewing for several APS journals, I stand by my statement. Because of 'publish or perish' nobody wants to give up any 'competitive edge' of their particular research. So there is no incentive system in place to foster an open source everything attitude.
  - maxnoe 8y ago
    This is changing. Slowly, as it seems to be mostly a generational issue. The younger scientists, current PhD generation and a little up are all fed up with closed source/private data as there are no enough bright examples how good open source/data can work out.
dschuetz 8y ago
The Nobel Prize Committee doesn't do mistakes, right? Right? https://en.wikipedia.org/wiki/Nobel_Prize_controversies#Physics https://en.wikipedia.org/wiki/Nobel_Prize_controversies#Phys... Thousands of people worked on gravitational waves, but somehow only LIGO is the one taking credit. I'd like to know what other physicists think of the LIGO results and methods.
- consp 8y ago
  Since VIRGO wasn't running at high enough signal to noise ratio before 2017, yes. But then again "LIGO" is not just one person but hundreds if not more. The same could be said for CERN which had initial papers with around 1000 coauthors.
- maxnoe 8y ago
  Astroparticle physicist here. I work in gammay-ray astronomy. We work closely together with LIGO, they alert us in case something interesting happens so we can observe the same position. This is e.g. what happened with the Neutron star merger. The community has great respect for the awesome work our colleagues at LIGO do. Especially since they made data and code public, which sadly is something that just starts gaining track in gammaray astronomy.
- grigjd3 8y ago
  As a former numrel person who definitely did not get credit, the LIGO collaboration deserved this.
- canhascodez 8y ago
  The idea that Nobel prizes are inherently unfair actually has wide credence[0][1][2]: basically everyone agrees that singling out individuals or even single organizations ignores the real degree of collaboration between scientists of all fields and levels. However, they're a tradition, and a good show, they're fantastic as a tool for publicity, the money is usually appreciated by the recipients. However, the real point of the Nobel Prize is that it's a great honor, and the effect of that honor is diminished in proportion to the number of recipients--ditto for the prize money. [0] https://www.theatlantic.com/science/archive/2017/10/the-absurdity-of-the-nobel-prizes-in-science/541863/ [1] https://blogs.scientificamerican.com/observations/its-time-to-rethink-the-nobel-prizes/ [2] https://www.researchgate.net/post/Do_you_believe_the_Nobel_Prize_is_fair_and_or_even_relevant
djsumdog 8y ago
Well the equipment to reproduce the findings includes multimillion dollar lasers that are required to have precision under a nanometre. If I understand correctly, gravational waves require data to line up between multiple telescopes around the world, right? So all these labs have to be sharing their data through some organized means, right? Here's a podcast I head recently that went over gravational waves in detail: https://www.danielandjorge.com/podcasts/what-are-gravitational-waves.htm https://www.danielandjorge.com/podcasts/what-are-gravitation...
maxnoe 8y ago
All three points you mentioned are true for large parts of physics research. The point is: Not so for LIGO. The data is open, the code is open. They even had Jupiter notebooks doing the later parts of the analysis when they published the first detection. And, one cannot overstate that while the black hole mergers might have been bogus for some reason, the neutron star merger as been observed by over 80 different experiments in all wavelengths of the electromagnetic spectrum.
grigjd3 8y ago
I don't want to pile on, but I worked in numrel on binary blackholes and I have my gripes about the general gw community, but this is definitely not one of them. In fact, the GW community is probably one of the most open ones in physics. It's just such a hard subject that no one would survive acting like you describe.
ajford 8y ago
Ummm... where are you getting this idea from? Nearly every field I've worked in, and any NSF funded facility has the requirement to publish and open source the data collected. I will admit that most telescopes operate on a rolling window of privacy (usually 12-18 months) so that an observer with an original idea can't have their research snatched out from under them, but most will gladly share their data for someone who has doubts or wants to open up communications on reproducibility. I worked very closely with this field, participating with a similar experiment to use frequent observations of accurately modeled millisecond pulsars to study and detect gravitational wave, and while we did have that rolling window, nearly all of our data was shared far and wide often well before that window was up. Perhaps LIGO's behavior seems secretive, but they have a multi-national group with hundreds of researchers working on it, so if you're even remotely within the field, I don't see how you can come to that conclusion at all. Their analysis techniques have been public for at least the last 5 years (some of their repos go back at least 7 years: https://github.com/gwastro https://github.com/gwastro). Their data is completely available (https://www.gw-openscience.org/data/ https://www.gw-openscience.org/data/), and even have an entire series of materials on how they perform their techniques (https://www.gw-openscience.org/static/workshop1/course.html https://www.gw-openscience.org/static/workshop1/course.html). I've found Astrophysics to be one of the most reproducible and open fields in science, since it rarely has the same kind of financial motivations that bio-med, minerals and materials science, and even environmental science can have.
- sleavey 8y ago
  Agreed. The "secrecy" thing is just hyperbole from New Scientist. What LIGO are doing with their data is completely standard practice in the community.

punchingwater 8y ago

the article states: > “But Reitze counters that the complete data from that first run is already available online. According to Shoemaker, this includes the relevant time series data and the programs used, but "it's not a trivial matter to use them." Caltech even held a training workshop on how to deal with gravitational-wave data. That's a pretty far cry from asking the physics community to take its analysis on faith...” Are you refuting this statement? To me, it looks like the data/analysis are open, but not yet independently verified due to difficult nature of problem space.

smueller1234 8y ago
I used to work in particle/astrophysics. By and large people weren't opposed to sharing their data and software. The degree of specific expertise it took to get results was staggerng though. It takes significant technical infrastructure and expertise to even just rerun an analysis on a big corpus of data. At that stage, you haven't even validated squat, just pushed some buttons to run other people's logic. Doing this properly takes person years of effort with little to no reward. In practice, there's usually at least two big experiments (not necessarily quite of the same generation) that were built by different groups which corroborate results. This is currently the best defense against big mistakes.
splittingTimes 8y ago
I did not read the article till the very last paragraph. My bad. Of course, I do not refute it. So in this particular case they went open source. However, from my experience in years of publishing to and reviewing for several APS journals, I stand by my statement. Because of 'publish or perish' nobody wants to give up any 'competitive edge' of their particular research. So there is no incentive system in place to foster an open source everything attitude.
- maxnoe 8y ago
  This is changing. Slowly, as it seems to be mostly a generational issue. The younger scientists, current PhD generation and a little up are all fed up with closed source/private data as there are no enough bright examples how good open source/data can work out.

dschuetz 8y ago

The Nobel Prize Committee doesn't do mistakes, right? Right? https://en.wikipedia.org/wiki/Nobel_Prize_controversies#Physics https://en.wikipedia.org/wiki/Nobel_Prize_controversies#Phys... Thousands of people worked on gravitational waves, but somehow only LIGO is the one taking credit. I'd like to know what other physicists think of the LIGO results and methods.

consp 8y ago
Since VIRGO wasn't running at high enough signal to noise ratio before 2017, yes. But then again "LIGO" is not just one person but hundreds if not more. The same could be said for CERN which had initial papers with around 1000 coauthors.
maxnoe 8y ago
Astroparticle physicist here. I work in gammay-ray astronomy. We work closely together with LIGO, they alert us in case something interesting happens so we can observe the same position. This is e.g. what happened with the Neutron star merger. The community has great respect for the awesome work our colleagues at LIGO do. Especially since they made data and code public, which sadly is something that just starts gaining track in gammaray astronomy.
grigjd3 8y ago
As a former numrel person who definitely did not get credit, the LIGO collaboration deserved this.
canhascodez 8y ago
The idea that Nobel prizes are inherently unfair actually has wide credence[0][1][2]: basically everyone agrees that singling out individuals or even single organizations ignores the real degree of collaboration between scientists of all fields and levels. However, they're a tradition, and a good show, they're fantastic as a tool for publicity, the money is usually appreciated by the recipients. However, the real point of the Nobel Prize is that it's a great honor, and the effect of that honor is diminished in proportion to the number of recipients--ditto for the prize money. [0] https://www.theatlantic.com/science/archive/2017/10/the-absurdity-of-the-nobel-prizes-in-science/541863/ [1] https://blogs.scientificamerican.com/observations/its-time-to-rethink-the-nobel-prizes/ [2] https://www.researchgate.net/post/Do_you_believe_the_Nobel_Prize_is_fair_and_or_even_relevant

djsumdog 8y ago

Well the equipment to reproduce the findings includes multimillion dollar lasers that are required to have precision under a nanometre. If I understand correctly, gravational waves require data to line up between multiple telescopes around the world, right? So all these labs have to be sharing their data through some organized means, right? Here's a podcast I head recently that went over gravational waves in detail: https://www.danielandjorge.com/podcasts/what-are-gravitational-waves.htm https://www.danielandjorge.com/podcasts/what-are-gravitation...

maxnoe 8y ago

All three points you mentioned are true for large parts of physics research. The point is: Not so for LIGO. The data is open, the code is open. They even had Jupiter notebooks doing the later parts of the analysis when they published the first detection. And, one cannot overstate that while the black hole mergers might have been bogus for some reason, the neutron star merger as been observed by over 80 different experiments in all wavelengths of the electromagnetic spectrum.

grigjd3 8y ago

I don't want to pile on, but I worked in numrel on binary blackholes and I have my gripes about the general gw community, but this is definitely not one of them. In fact, the GW community is probably one of the most open ones in physics. It's just such a hard subject that no one would survive acting like you describe.

ajford 8y ago

Ummm... where are you getting this idea from? Nearly every field I've worked in, and any NSF funded facility has the requirement to publish and open source the data collected. I will admit that most telescopes operate on a rolling window of privacy (usually 12-18 months) so that an observer with an original idea can't have their research snatched out from under them, but most will gladly share their data for someone who has doubts or wants to open up communications on reproducibility. I worked very closely with this field, participating with a similar experiment to use frequent observations of accurately modeled millisecond pulsars to study and detect gravitational wave, and while we did have that rolling window, nearly all of our data was shared far and wide often well before that window was up. Perhaps LIGO's behavior seems secretive, but they have a multi-national group with hundreds of researchers working on it, so if you're even remotely within the field, I don't see how you can come to that conclusion at all. Their analysis techniques have been public for at least the last 5 years (some of their repos go back at least 7 years: https://github.com/gwastro https://github.com/gwastro). Their data is completely available (https://www.gw-openscience.org/data/ https://www.gw-openscience.org/data/), and even have an entire series of materials on how they perform their techniques (https://www.gw-openscience.org/static/workshop1/course.html https://www.gw-openscience.org/static/workshop1/course.html). I've found Astrophysics to be one of the most reproducible and open fields in science, since it rarely has the same kind of financial motivations that bio-med, minerals and materials science, and even environmental science can have.

sleavey 8y ago
Agreed. The "secrecy" thing is just hyperbole from New Scientist. What LIGO are doing with their data is completely standard practice in the community.