6 ms·
Great entertainment for anyone with basic knowledge of statistics: read medical papers. Lots of laughs to be had. Most experiments are bogus. Most conclusions a
by general_ai 10y ago
Great entertainment for anyone with basic knowledge of statistics: read medical papers. Lots of laughs to be had. Most experiments are bogus. Most conclusions are drawn with little in the way of supporting evidence, often without controlling for obvious confounding factors. Correlation is implied to mean causation. Extrapolations abound, both for concentrations (i.e. if I give this mouse a megadose of X it'll die, hence X is harmful) and between species (mice grow tumors if I give them a ton of X, so humans will too if they consume a tiny fraction of the amount per unit of body weight). Most of it is basically bro science, except expressed in much longer words.
- timcederman 10y agoActually sourcing the data via PubMed has been eye opening for me, and I saw all those things. I am amazed by the number of news articles that cite conclusions as factual for humans, but on reading the source, they have only been shown in mice studies.
- general_ai 10y agoAnd quite often they're obviously bogus even in mice, especially if the author seems to have a political axe to grind (e.g. anti-GMO, anti-sweetener, anti-herbicide, etc).
- amelius 10y agoIt would be nice to see a meta-study on this.
- general_ai 10y agoI agree. But peeing in other people's cereal is a perilous endeavor.
- DanBC 10y agoCochrane reviews frequntly say "the available evidence is poor, more research is needed", and "we found X relavent papers, but had to exclude Y which were low quality."
- cperciva 10y agoExtrapolations Technically, going from "megadose is lethal" to "smaller doses are harmful" is an interpolation, not an extrapolation.
- TeMPOraL 10y agoOnly if an even smaller dose was tested too.
- petertodd 10y agoWell, technically anyone with a pet mouse is doing the experiment "Does a dose of ~0ppm of cyanide/arsenic/etc. kill mice?" :)
- AstralStorm 10y agoThat is called a control group. You need to actually have a formal one and big enough too. It has to be kept in as similar conditions as the group under treatment and have genetically similar makeup. Most importantly, the composition of both groups and the conditions the experiment is ran under should be rigorously described.
- TheCoelacanth 10y agoIt does, but it seems to be very slow acting.
- deleted 10y ago[deleted]
- ArkyBeagle 10y agoThe dose is the poison.
- marcosdumay 10y agoThere are infinite interpolation possibilities, one decides it is linear by extrapolation.
- Fnoord 10y ago> and between species (mice grow tumors if I give them a ton of X, so humans will too if they consume a tiny fraction of the amount per unit of body weight). Not only that, but to add to the injury using mice or rats which are known to be sensitive to growing tumors due to factor X (which is henceforth completely ignored, and fails to be mentioned). Sad, very sad. I wasted a lot of energy & focus in my life on this.
- dekhn 10y agoSadly, this applies beyond just medical research. Most of the biology papers we discussed in Journal Club (a weekly meeting of grad students who go over a single paper in detail) turned out to have at least one invalidating error (one that makes the conclusion unsupported). As for dose relationships- a lot of drug dose responses are linear over a range. But they tend to not be linear outside that range; the error is in not correcting for low-dose effects.
- jpmattia 10y agoJust to put your rant on more sound statistical footing, there is the whole Replication Crisis issue: https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
- Balgair 10y agoHoly cow! I've been looking for a 'clearinghouse' site that addresses all the issues swiftly and clearly. It's obvious in retrospect, but I never though Wikipedia would be that place. Thank you!
- mtw 10y agoI disagree. This is a bit like browsing open source repos on github and laughing because of bad algorithms, issues or quick fixes. There is indeed bad medical research. Typically, animal research that focuses on just one marker and then extrapolates on what it means for humans. You should not rely on these. At best, they give you cues on what might work for humans. But there are multiple studies that reviews all medical research. They study 5 million people over 20 years, follow their health as well as their habits. Variables are adjusted. In such studies, if you discover that green tea decreases risk of type 2 diabetes, then evidence is high. Much, much higher than the simple animal research where mice were given green tea. The biggest problem is not medical research. It's the army of bloggers, writers from NYTimes and overall the general population hungry for the latest superfood or cure of cancer. They think that throwing billions of dollars will find a revolutionary food, herb or medicine for cancer. So medical researchers present their results in a way that's pleasing. And then most bloggers are just too guilible. That's amplificated by social media, or even boards like Hacker News (see the upvotes for effect of coffee on mortality yesterday. That study is non-conclusive but the story got hundreds of upvotes. People are just too happy to discover that their daily drink will extend their lifespan, even though evidence is limited)
- hawleyal 10y agoI mean, medicine is not science, it is an applied science. Applying biology, chemistry, and physics to the attempted betterment of health. Akin to the difference between physics and engineering.
- briHass 10y agoDon't forget all the BS drawn from epidemiological "research". We ask 10,000 people how many cheeseburgers, eggs, tomatoes, etc. they eat in a week, at the start of the study, extrapolate that out for 10 years, see how many died, break the participants into arbitrary quartiles, and report everything as relative risk to pump the headlines: "Eating 2 cheeseburgers a week increases your risk of death by 75%" None of this research passes even the most basic statistical sniff-test, and even if it did, the hazard ratios are so small as to be easily influenced by noise or confounders.
- general_ai 10y agoMy favorite, especially because this ends up being reported in the media as infallible truths.