3 ms·

I wonder if any efforts have been made to conduct a deep review of clinical trials to see if there are patterns of protocol design that are likely to lead to un

I wonder if any efforts have been made to conduct a deep review of clinical trials to see if there are patterns of protocol design that are likely to lead to underreporting of adverse effects. As (unfortunately) a user of a number of prescription medications, I have encountered adverse effects that were considered uncommon, yet heard about similar issues from fellow users. Of course, this is anecdata, so one can draw no conclusions, but I have a nagging suspicion that enough digging by people who truly understand statistics will find flaws, perhaps major ones. Not likely to happen, though.

pdfernhout 9y ago
See my other comment quoting Marcia Angell on how most of the clinical literature is rife with conflict of interest and, in general, questionable science. Some other related links: http://en.wikipedia.org/wiki/Bruno_Latour http://en.wikipedia.org/wiki/Bruno_Latour "In the laboratory, Latour and Woolgar observed that a typical experiment produces only inconclusive data that is attributed to failure of the apparatus or experimental method, and that a large part of scientific training involves learning how to make the subjective decision of what data to keep and what data to throw out. To an untrained outsider, Latour and Woolgar argued the entire process resembles not an unbiased search for truth and accuracy but a mechanism for ignoring data that contradicts scientific orthodoxy." "Lies, Damned Lies, and Medical Science" http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/ http://www.theatlantic.com/magazine/archive/2010/11/lies-dam... "Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors -- to a striking extent -- still drawing upon misinformation in their everyday practice? Dr. John Ioannidis has spent his career challenging his peers by exposing their bad science." "Fat, Sick, and Nearly Dead" is a funny documentary about someone who got off all his prescriptions medications by changing his diet and then helped others to do the same: http://fatsickandnearlydead.com/ http://fatsickandnearlydead.com/ This may not apply in your case, but in general, as Dr. Joel Furhman says, many prescriptions are essentially just "permission slips" to avoid lifestyle changes. http://www.meatlessmonday.com/articles/mm-interviewwith-dr-joel-fuhrman/ http://www.meatlessmonday.com/articles/mm-interviewwith-dr-j... "Our healthcare system has evolved into an industry where doctors mostly provide drugs, instead of being teachers of healthy living. The medical profession is not predominantly focused on preventing disease. They are a profession that’s diagnosing and treating disease, and the reality is, the treatments hardly work and the small benefits place people at significant risk, while the underlying disease process continues to advance." Dr. Mark Hyman and Dr. Andrew Weil are other good resources.
rjdagost 9y ago
There are a number of reasons why problems can slip through clinical trials, even with proper clinical trials design. Some side effects are extremely rare, so rare that even with “large” clinical trials groups the problems don’t occur. Or they might occur, but at rates no greater than what the placebo group experiences. Some side effects only show up with repeated long-term use of a drug. That’s why there exists “post approval monitoring” to watch for these problems. Some side effects only show up when a drug is taken concomitantly with other drugs. Some side effects only show up when users have certain diets. Some side effects (and some efficacies) only show up for certain ethnicities and sub-populations. Some side effects only show up when patients take much larger doses than were tested in clinical trials. Larger statistical samples would help to reduce these problems, but it’s often not as easy as you might think to recruit large numbers of people for your study. First, it is quite expensive to put a large test group together. Some diseases / conditions are quite rare, so it can be difficult to even find enough people with the condition. As the test runs through, some fraction of the participants will drop out for one reason or another. Some people will just stop taking the medication for one reason or another. How you treat these “dropouts” is not a simple question and can have a huge impact on your clinical trials conclusions. Some test measures are highly subjective and experts will disagree on how efficacious / dangerous a drug is when presented with the same exact data. The bottom line is that drug approval is extremely complicated. Better test design and monitoring helps but it will not catch all problems. Regulators have to balance safety with benefits in the face of uncertainty.
Waterluvian 9y ago
Does "uncommon" have a strict definition? Under 50% maybe? Or under 10%? No clue.
brainfire 9y ago
Ben Goldacre (of "Bad Science" fame) has been doing work in this area for a while: http://www.badscience.net/ http://www.badscience.net/

pdfernhout 9y ago

See my other comment quoting Marcia Angell on how most of the clinical literature is rife with conflict of interest and, in general, questionable science. Some other related links: http://en.wikipedia.org/wiki/Bruno_Latour http://en.wikipedia.org/wiki/Bruno_Latour "In the laboratory, Latour and Woolgar observed that a typical experiment produces only inconclusive data that is attributed to failure of the apparatus or experimental method, and that a large part of scientific training involves learning how to make the subjective decision of what data to keep and what data to throw out. To an untrained outsider, Latour and Woolgar argued the entire process resembles not an unbiased search for truth and accuracy but a mechanism for ignoring data that contradicts scientific orthodoxy." "Lies, Damned Lies, and Medical Science" http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/ http://www.theatlantic.com/magazine/archive/2010/11/lies-dam... "Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors -- to a striking extent -- still drawing upon misinformation in their everyday practice? Dr. John Ioannidis has spent his career challenging his peers by exposing their bad science." "Fat, Sick, and Nearly Dead" is a funny documentary about someone who got off all his prescriptions medications by changing his diet and then helped others to do the same: http://fatsickandnearlydead.com/ http://fatsickandnearlydead.com/ This may not apply in your case, but in general, as Dr. Joel Furhman says, many prescriptions are essentially just "permission slips" to avoid lifestyle changes. http://www.meatlessmonday.com/articles/mm-interviewwith-dr-joel-fuhrman/ http://www.meatlessmonday.com/articles/mm-interviewwith-dr-j... "Our healthcare system has evolved into an industry where doctors mostly provide drugs, instead of being teachers of healthy living. The medical profession is not predominantly focused on preventing disease. They are a profession that’s diagnosing and treating disease, and the reality is, the treatments hardly work and the small benefits place people at significant risk, while the underlying disease process continues to advance." Dr. Mark Hyman and Dr. Andrew Weil are other good resources.

rjdagost 9y ago

There are a number of reasons why problems can slip through clinical trials, even with proper clinical trials design. Some side effects are extremely rare, so rare that even with “large” clinical trials groups the problems don’t occur. Or they might occur, but at rates no greater than what the placebo group experiences. Some side effects only show up with repeated long-term use of a drug. That’s why there exists “post approval monitoring” to watch for these problems. Some side effects only show up when a drug is taken concomitantly with other drugs. Some side effects only show up when users have certain diets. Some side effects (and some efficacies) only show up for certain ethnicities and sub-populations. Some side effects only show up when patients take much larger doses than were tested in clinical trials. Larger statistical samples would help to reduce these problems, but it’s often not as easy as you might think to recruit large numbers of people for your study. First, it is quite expensive to put a large test group together. Some diseases / conditions are quite rare, so it can be difficult to even find enough people with the condition. As the test runs through, some fraction of the participants will drop out for one reason or another. Some people will just stop taking the medication for one reason or another. How you treat these “dropouts” is not a simple question and can have a huge impact on your clinical trials conclusions. Some test measures are highly subjective and experts will disagree on how efficacious / dangerous a drug is when presented with the same exact data. The bottom line is that drug approval is extremely complicated. Better test design and monitoring helps but it will not catch all problems. Regulators have to balance safety with benefits in the face of uncertainty.

Waterluvian 9y ago

Does "uncommon" have a strict definition? Under 50% maybe? Or under 10%? No clue.

brainfire 9y ago

Ben Goldacre (of "Bad Science" fame) has been doing work in this area for a while: http://www.badscience.net/ http://www.badscience.net/