3 ms·

Please don't ignore this very important quote from the article: "In fact, many of the initial 50 papers have been confirmed by other groups, as some of the RP:

by timr 8y ago

Please don't ignore this very important quote from the article: "In fact, many of the initial 50 papers have been confirmed by other groups, as some of the RP:CB’s critics have pointed out." This article is staying assiduously neutral, but one perfectly valid interpretation is that the original initiative was flawed in a way that was predicted by the initiative's critics. Science is routinely reproduced -- just not by labs working in isolation, using publications as clean-room instruction books. This is the sort of thing that programmers believe about science, not something that scientists believe themselves. There are a great many serious, legitimate scientists who believe that this "reproducibility crisis" is verging on irrational, and it's important to consider their arguments. It's particularly scary to me that so many comments here are dovetailing with the sort of nonsense you encounter on anti-vax and global warming denial forums. We're literally gaslighting the process that has done more to advance society than any other in human history: http://www.pnas.org/content/115/20/5042 http://www.pnas.org/content/115/20/5042 "The discovery that an experiment does not replicate is not a lack of success but an opportunity. Many of the current concerns about reproducibility overlook the dynamic, iterative nature of the process of discovery where discordant results are essential to producing more integrated accounts and (eventually) translation. A failure to reproduce is only the first step in scientific inquiry. In many ways, how science responds to these failures is what determines whether it succeeds."

southern_cross 8y ago
And the attitude being described above seems to me to border on religious "just trust us" dogma. I mean, one of the big things that's supposed to distinguish real science from pseudoscience and such is the ability to fairly easily and routinely reproduce its results. And if that's not happening, and in fact it's being actively discouraged, then that's a real problem! "In fact, many of the initial 50 papers have been confirmed by other groups, as some of the RP:CB’s critics have pointed out." I don't know that I would be willing to take a statement like that at face value. I once read of a scientist who claimed that there was absolutely no need to try and independently reproduce his work, since it was being routinely reproduced in college labs everywhere as part of other experiments - or something like that. To me that meant that it should then be trivial to reproduce these results when attempted by trained professionals who were specifically trying to do that very thing. So why the reluctance to allow that to happen?
- timr 8y ago
  "And the attitude being described above seems to me to border on religious "just trust us" dogma. I mean, one of the big things that's supposed to distinguish real science from pseudoscience and such is the ability to fairly easily and routinely reproduce its results." Nowhere is it written that you should be able to "easily and routinely" reproduce a scientific result. It's damned hard for skilled scientists to reproduce most scientific results. And yes, it's utterly impossible for amateurs. I'm sorry if that makes you uncomfortable, but it's the truth. There are many things in life that you routinely accept on authority: When you fly in a plane, that it isn't going to crash into the earth. When you turn on the tap, that clean, safe water comes out. When you go to your doctor, that she is prescribing you medication that will help you, not hurt you. When you vaccinate your children, that you are protecting them from horrible diseases. I'm sorry that it bothers you, but "science" is just one more thing in the world that you're going to have to accept, because nobody -- scientist or otherwise -- can independently verify all of human knowledge. If it makes you feel better, you can look around, realize that we're no starving from famine, or dying from minor cuts, or poisoning ourselves with lead, or dying of Polio or Smallpox, and you can try to convince yourself that in the long run, the method works. Because that's the argument you're missing. You don't have to take my word for it, or anyone else's. It's a system. The system works. You just don't fully understand why it works, and you're not willing to read when a group of scientists write a long article that tries to explain it to you. Because you'd rather believe that explanation is "dogma". "I don't know that I would be willing to take a statement like that at face value." Nobody is stopping you from digging into it. If you're really so bothered by it, I encourage you to follow up. "So why the reluctance to allow that to happen?" There is no reluctance, whatsoever. Nobody is trying to stop these people. They're just beginning to realize that their approach is a lot damned harder than they originally expected, and those of us who knew it would be are pointing it out.
  - nonbel 8y ago
    >"If it makes you feel better, you can look around, realize that we're no starving from famine, or dying from minor cuts, or poisoning ourselves with lead, or dying of Polio or Smallpox, and you can try to convince yourself that in the long run, the method works." Don't you have any more recent examples? Perhaps people used to be doing reproducible science but this got lost somewhere along the way, that's why all your examples are decades old.
nonbel 8y ago
>"In fact, many of the initial 50 papers have been confirmed by other groups, as some of the RP:CB’s critics have pointed out." Something is wrong about this. How did these "other groups" get the protocols while this reproducibility project couldn't? Are they being stonewalled? Did the other groups not actually get the real protocol but instead fiddled around until they got the same result? Unfortunately that claim doesnt have any citations.

by timr 8y ago

southern_cross 8y ago

And the attitude being described above seems to me to border on religious "just trust us" dogma. I mean, one of the big things that's supposed to distinguish real science from pseudoscience and such is the ability to fairly easily and routinely reproduce its results. And if that's not happening, and in fact it's being actively discouraged, then that's a real problem! "In fact, many of the initial 50 papers have been confirmed by other groups, as some of the RP:CB’s critics have pointed out." I don't know that I would be willing to take a statement like that at face value. I once read of a scientist who claimed that there was absolutely no need to try and independently reproduce his work, since it was being routinely reproduced in college labs everywhere as part of other experiments - or something like that. To me that meant that it should then be trivial to reproduce these results when attempted by trained professionals who were specifically trying to do that very thing. So why the reluctance to allow that to happen?

timr 8y ago
"And the attitude being described above seems to me to border on religious "just trust us" dogma. I mean, one of the big things that's supposed to distinguish real science from pseudoscience and such is the ability to fairly easily and routinely reproduce its results." Nowhere is it written that you should be able to "easily and routinely" reproduce a scientific result. It's damned hard for skilled scientists to reproduce most scientific results. And yes, it's utterly impossible for amateurs. I'm sorry if that makes you uncomfortable, but it's the truth. There are many things in life that you routinely accept on authority: When you fly in a plane, that it isn't going to crash into the earth. When you turn on the tap, that clean, safe water comes out. When you go to your doctor, that she is prescribing you medication that will help you, not hurt you. When you vaccinate your children, that you are protecting them from horrible diseases. I'm sorry that it bothers you, but "science" is just one more thing in the world that you're going to have to accept, because nobody -- scientist or otherwise -- can independently verify all of human knowledge. If it makes you feel better, you can look around, realize that we're no starving from famine, or dying from minor cuts, or poisoning ourselves with lead, or dying of Polio or Smallpox, and you can try to convince yourself that in the long run, the method works. Because that's the argument you're missing. You don't have to take my word for it, or anyone else's. It's a system. The system works. You just don't fully understand why it works, and you're not willing to read when a group of scientists write a long article that tries to explain it to you. Because you'd rather believe that explanation is "dogma". "I don't know that I would be willing to take a statement like that at face value." Nobody is stopping you from digging into it. If you're really so bothered by it, I encourage you to follow up. "So why the reluctance to allow that to happen?" There is no reluctance, whatsoever. Nobody is trying to stop these people. They're just beginning to realize that their approach is a lot damned harder than they originally expected, and those of us who knew it would be are pointing it out.
- nonbel 8y ago
  >"If it makes you feel better, you can look around, realize that we're no starving from famine, or dying from minor cuts, or poisoning ourselves with lead, or dying of Polio or Smallpox, and you can try to convince yourself that in the long run, the method works." Don't you have any more recent examples? Perhaps people used to be doing reproducible science but this got lost somewhere along the way, that's why all your examples are decades old.

nonbel 8y ago

>"In fact, many of the initial 50 papers have been confirmed by other groups, as some of the RP:CB’s critics have pointed out." Something is wrong about this. How did these "other groups" get the protocols while this reproducibility project couldn't? Are they being stonewalled? Did the other groups not actually get the real protocol but instead fiddled around until they got the same result? Unfortunately that claim doesnt have any citations.