3 ms·

The most interesting part in my opinion is the linked meta-study from 2006 which had the following conclusion: > Forty (68%) studies reported one or more stati

by pera 8y ago

The most interesting part in my opinion is the linked meta-study from 2006 which had the following conclusion: > Forty (68%) studies reported one or more statistically significant results (p < 0.05) indicating an effect of the exposure (Table 2). Studies funded exclusively by industry reported on the largest number of outcomes but were less likely to report statistically significant results [...] > Our findings add to the existing evidence that single-source sponsorship is associated with outcomes that favor the sponsors’ products https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1797826/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1797826/ This may be a fact that scientist are well aware of, but I would say it's not something that the general population knows; for most people there is this fantasy where the "scientific community" reaches consensus almost spontaneously and that's the end of the story, we got a new universal law. If only the media said "some new analysis indicates that there could be a relation between this and that" instead of the usual "Science says"...

dvfjsdhgfv 8y ago
That's interesting and sad at the same time.
gus_massa 8y ago
The problem with a fixed criteria of p < 0.05 for statistically significant results is that it is only useful if each experiment measures only one thing and all the studies are published. I remember an study of a few year ago where they had * rats and mice -> x2 * male/female/both -> x2 (or x3) * three level of radiation -> x3 * two or three types of cancer -> x2 (or x3) So the report has at least 2x2x3x2=24 results. There is a high probability that by chance one of them gets a p<0.05. Also, studies with a positive result have a higher chance of been published, because the researchers are not interested in publishing boring studies without a positive result so they don't write it, and the journals are not interested in publishing boring studies without a positive result so they don't approve it. (And I'm ignoring intentional fraud and p hacking ...) Anyway, perhaps the researchers sponsored by the telephone companies/manufacters just publish all their studies and the independent researches publish only the interesting studies. This is why in certain areas it is mandatory to register the experiments before they start, to ensure (reduce) there is no publication bias. In particle physics it's too easy to do experiments, filtering and analysis, so they use a criteria of 5 sigmas to call it a discovery. This is roughly p<0.0000003, because if they use p<0.05 you would get every month a announcement of a new particle and a few month later the retraction. For experiments with mice/rats it's impossible to ask for p<0.0000003 because you will need trillions of rats, but remember that p<0.05 is just an arbitrary threshold.

by pera 8y ago

dvfjsdhgfv 8y ago

That's interesting and sad at the same time.

gus_massa 8y ago

The problem with a fixed criteria of p < 0.05 for statistically significant results is that it is only useful if each experiment measures only one thing and all the studies are published. I remember an study of a few year ago where they had * rats and mice -> x2 * male/female/both -> x2 (or x3) * three level of radiation -> x3 * two or three types of cancer -> x2 (or x3) So the report has at least 2x2x3x2=24 results. There is a high probability that by chance one of them gets a p<0.05. Also, studies with a positive result have a higher chance of been published, because the researchers are not interested in publishing boring studies without a positive result so they don't write it, and the journals are not interested in publishing boring studies without a positive result so they don't approve it. (And I'm ignoring intentional fraud and p hacking ...) Anyway, perhaps the researchers sponsored by the telephone companies/manufacters just publish all their studies and the independent researches publish only the interesting studies. This is why in certain areas it is mandatory to register the experiments before they start, to ensure (reduce) there is no publication bias. In particle physics it's too easy to do experiments, filtering and analysis, so they use a criteria of 5 sigmas to call it a discovery. This is roughly p<0.0000003, because if they use p<0.05 you would get every month a announcement of a new particle and a few month later the retraction. For experiments with mice/rats it's impossible to ask for p<0.0000003 because you will need trillions of rats, but remember that p<0.05 is just an arbitrary threshold.