61 ms·

You don't trust Monsanto, someone else only trusts Monsanto, and I don't trust anyone. But trust doesn't matter because the system isn't built on trust. It's bu

You don't trust Monsanto, someone else only trusts Monsanto, and I don't trust anyone. But trust doesn't matter because the system isn't built on trust. It's built on peer review and repeatability. Right?

philipkglass 10y ago
The underlying incentives can influence which studies make it all the way to publication. This has happened with drug trials, for example; if you don't force drug companies to pre-register trials and publish the results after the fact for every trial, they can run multiple for the same drug and only report the ones that show favorable results. I could imagine something similar happening if you wanted to "prove" the safety of a particular herbicide. Run tests for carcinogenicity, endocrine disruption, whatever across common test species A, B, C, D, and E. D shows significant adverse effects from exposure to your product. The others don't. Silently drop the D results, write up the "proof of safety" from the others, and optionally launder the origins of the research through unaffiliated scientists. Repeat for a few years and later a completely independent researcher may say "all N of these published papers used sound methodology, and a meta-review of this high-quality subset shows no demonstrated risk at ordinary exposure levels." That last researcher won't know about the deliberately introduced systemic error that influenced which results were published.
- trentnix 10y ago
  But that's exactly the logic global warming skeptics use - there are underlying incentives to arrive at certain conclusions. If you find yourself in the business of having to defend or attack incentives and influences, you're not addressing the science. Instead, you're just playing in the politics. If bad science is making it to publication, or bad science is being accepted by other scientists, then it's not really science. Right? Don't mistake my statements for justifying the lack of transparency Monsanto has apparently engaged in. It's unfortunate, for sure, and those who are carrying their water should be castigated for it. It's not in the spirit of good science. But it's reasonable to think that had they been transparent, it wouldn't have been published. Because we have decided their underlying incentives are bad (and consequently, the underlying incentives of others are good). And that's not in the spirit of good science, either.
  - philipkglass 10y ago
    If bad science is making it to publication, or bad science is being accepted by other scientists, then it's not really science. Right? That's a very interesting question. I used to be a researcher myself. I love Feynman's "The first principle is that you must not fool yourself — and you are the easiest person to fool." In practice, I would say that there are often incentives for researchers to fool themselves, and researchers do sometimes succeed in fooling themselves and their colleagues. The practice of science has a better capacity for correcting these errors over time than other practices, but I wouldn't retroactively declare a paper "not really science" just because later scientists falsified its findings. (E.g. in chemistry, reports of iron-catalyzed reactions that actually turned out to be catalyzed by traces of platinum group metals in the iron. How did the incentive to report something new and wonderful -- reactions catalyzed by a very cheap catalyst -- influence the original researcher's experimental design and evaluation of the evidence? Unknown.) Nor would I declare that a publication showing "compound X does not induce cancer in mice" not-really-science even if it turned out later that the same researchers silently dropped a line of inquiry that showed X does induce cancer in rats. I tend toward thinking that social phenomena -- including the practice of scientific research, or the practice of religion -- are defined by what their participants do rather than what the ideal definition says they should do. So bad science is still science (if the practitioners are still recognized socially as such), bad Christianity is still Christianity, and so on. These issues are more visible when there's a lot at stake outside the scope of the published findings (global warming, tobacco health impacts, etc.) AGW skeptics are not exactly wrong to mention incentives, but I think they're weighting incentives backwards. I think that many of them see the likely policy implications of certain research conclusions and motivated-reason their way backwards to an exceptional level of skepticism about the research itself. I myself think that AGW skeptics are wrong, that GMOs are a facially neutral tool with potential for both benefits and harms, that herbicides too present both harms and benefits, and that industries trying to influence research to their own financial benefit (Monsanto here, drug companies with drug trials, food and beverage companies with nutrition research...) is both bad and common. This article that we're discussing does not make me think that glyphosate is more harmful than previously reported. I still require affirmative research findings in that direction to change my view. It does make me think that regulatory agencies should be sufficiently funded and staffed to set up their own experimental design and trials for safety, because industry-linked research (worse, facially independent but covertly industry-linked research) is incentivized toward affirming safety rather than investigating it neutrally.
TallGuyShort 10y ago
But only because peer review and repeatability increases the number of people in the pool whose credibility can be attached to the research. If you don't trust anyone, why would you ever trust any research that wasn't done 100% by you under conditions 100% controlled by you (which I'm sure includes exactly no research)? Don't trust Monsanto? Okay. But what if one does trust some government lab that repeated it? Or some environmental non-profit that reviewed it and signed off? It's still based on trust, but you're able to choose from more people to trust. Transparency also promotes trust - like I said, going out of their way to not be transparent about it hints that there IS something to hide but they don't want a lot of eyes on it.

philipkglass 10y ago

The underlying incentives can influence which studies make it all the way to publication. This has happened with drug trials, for example; if you don't force drug companies to pre-register trials and publish the results after the fact for every trial, they can run multiple for the same drug and only report the ones that show favorable results. I could imagine something similar happening if you wanted to "prove" the safety of a particular herbicide. Run tests for carcinogenicity, endocrine disruption, whatever across common test species A, B, C, D, and E. D shows significant adverse effects from exposure to your product. The others don't. Silently drop the D results, write up the "proof of safety" from the others, and optionally launder the origins of the research through unaffiliated scientists. Repeat for a few years and later a completely independent researcher may say "all N of these published papers used sound methodology, and a meta-review of this high-quality subset shows no demonstrated risk at ordinary exposure levels." That last researcher won't know about the deliberately introduced systemic error that influenced which results were published.

trentnix 10y ago
But that's exactly the logic global warming skeptics use - there are underlying incentives to arrive at certain conclusions. If you find yourself in the business of having to defend or attack incentives and influences, you're not addressing the science. Instead, you're just playing in the politics. If bad science is making it to publication, or bad science is being accepted by other scientists, then it's not really science. Right? Don't mistake my statements for justifying the lack of transparency Monsanto has apparently engaged in. It's unfortunate, for sure, and those who are carrying their water should be castigated for it. It's not in the spirit of good science. But it's reasonable to think that had they been transparent, it wouldn't have been published. Because we have decided their underlying incentives are bad (and consequently, the underlying incentives of others are good). And that's not in the spirit of good science, either.
- philipkglass 10y ago
  If bad science is making it to publication, or bad science is being accepted by other scientists, then it's not really science. Right? That's a very interesting question. I used to be a researcher myself. I love Feynman's "The first principle is that you must not fool yourself — and you are the easiest person to fool." In practice, I would say that there are often incentives for researchers to fool themselves, and researchers do sometimes succeed in fooling themselves and their colleagues. The practice of science has a better capacity for correcting these errors over time than other practices, but I wouldn't retroactively declare a paper "not really science" just because later scientists falsified its findings. (E.g. in chemistry, reports of iron-catalyzed reactions that actually turned out to be catalyzed by traces of platinum group metals in the iron. How did the incentive to report something new and wonderful -- reactions catalyzed by a very cheap catalyst -- influence the original researcher's experimental design and evaluation of the evidence? Unknown.) Nor would I declare that a publication showing "compound X does not induce cancer in mice" not-really-science even if it turned out later that the same researchers silently dropped a line of inquiry that showed X does induce cancer in rats. I tend toward thinking that social phenomena -- including the practice of scientific research, or the practice of religion -- are defined by what their participants do rather than what the ideal definition says they should do. So bad science is still science (if the practitioners are still recognized socially as such), bad Christianity is still Christianity, and so on. These issues are more visible when there's a lot at stake outside the scope of the published findings (global warming, tobacco health impacts, etc.) AGW skeptics are not exactly wrong to mention incentives, but I think they're weighting incentives backwards. I think that many of them see the likely policy implications of certain research conclusions and motivated-reason their way backwards to an exceptional level of skepticism about the research itself. I myself think that AGW skeptics are wrong, that GMOs are a facially neutral tool with potential for both benefits and harms, that herbicides too present both harms and benefits, and that industries trying to influence research to their own financial benefit (Monsanto here, drug companies with drug trials, food and beverage companies with nutrition research...) is both bad and common. This article that we're discussing does not make me think that glyphosate is more harmful than previously reported. I still require affirmative research findings in that direction to change my view. It does make me think that regulatory agencies should be sufficiently funded and staffed to set up their own experimental design and trials for safety, because industry-linked research (worse, facially independent but covertly industry-linked research) is incentivized toward affirming safety rather than investigating it neutrally.

TallGuyShort 10y ago

But only because peer review and repeatability increases the number of people in the pool whose credibility can be attached to the research. If you don't trust anyone, why would you ever trust any research that wasn't done 100% by you under conditions 100% controlled by you (which I'm sure includes exactly no research)? Don't trust Monsanto? Okay. But what if one does trust some government lab that repeated it? Or some environmental non-profit that reviewed it and signed off? It's still based on trust, but you're able to choose from more people to trust. Transparency also promotes trust - like I said, going out of their way to not be transparent about it hints that there IS something to hide but they don't want a lot of eyes on it.