8 ms·

The Deadly Corruption of Clinical Trials

111 points by kumarski 14y ago

richardjordan 14y ago
Regulatory capture. Used to be a term we used to criticize corrupt developing countries. Nowadays it appears to be at the root of so many problems facing America.
- cynicalkane 14y ago
  What are you talking about? Regulatory capture assumes a working rule of law, and describes a process by which people subject to the rules of the game try to rewrite them. In corrupt developing countries the power owns the rules outright, and regulatory capture is no longer a process. I guess you could call it "trivial regulatory capture" but it won't get you any Nobel Prizes. The theory was developed by Chicago School economists who liked to criticize big government. Regulatory capture has always been a bug of advanced, sufficiently honest societies. This is important because it tells us that well-intentioned rule of law is insufficient for running a democracy if you're not paying attention. I don't think anyone at the FDA wakes up and says, "I think I'll spend today getting in the way of good medical care and helping big pharma make money."
  - mschuster91 14y ago
    The problem is the privatization of critical infrastructure, a cancer spreading all over the world. To quote the last sentence of the article: "Many academic physicians are required by their universities to generate a substantial portion of their salaries by obtaining research grants." Healthcare, basic transportation, energy and water distribution as well as basic telecommunication infrastructure must be in the hands of the state. The free market has massively failed in each of these areas, everywhere in the world. Just look at the US electricity grid, the Berlin water privatization disaster (less service, more expensive), the US (and partially, German too) broadband Internet service, the Berlin S-Bahn problems, the (recently HN frontpaged) problems in the US healthcare system... Why can't we learn from the countless mistakes made all over the world?!
    - refurb 14y ago
      I think we have learned from the mistake made all around the world, particularly the ones where government run organizations lose billions of dollars through their shoddy operation of industries.
lemming 14y ago
For anyone interested in the generally fucked up state of clinical drug trials, I highly recommend Ben Goldacre's excellent book Bad Pharma. It'll make you angry.
- Create 14y ago
  Bad Science also does the job. The Emperor of All Maladies gives a candid and almost frank overview on how clinical drug trials are performed "from the inside". Aspirin the Story of a Wonder Drug is also a nice account of "drug development" in practice.
mschuster91 14y ago
This makes me feel so sad. I think the biggest lesson to learn from this case is that there has to be a independent entity in double-blind studies which knows the relation between test subject and the substance and has the power to interrupt or terminate a study when people's lifes are at risk. Again, another fine example of how capitalism does not take unforeseen issues into account - to refuse to drop out Dan of the study program is "manslaughter by neglicence" for me (not by malice, as the responsible actors could not have definite knowledge that Dan would kill himself).
- protomyth 14y ago
  You site capitalism, when all rules governing a clinical trial are strictly enforced by the FDA which is a government entity. The FDA has study rules and the entity conducting the study is not the drug company. It is an independent entity (for example http://clinicaltrials.mayo.edu http://clinicaltrials.mayo.edu) paid by the drug company under the supervision of the FDA (in the US).
  - logjam 14y ago
    That is some really utterly fascinating spin right there. Next up: criminals are not really responsible for crime, since "government entities" like municipal police forces supposedly "strictly enforce" criminal conduct. I'm going to take a wild guess here and assume you've never had anything whatsoever to do with a clinical trial, because the modern FDA has almost nothing to do with the way a clinical trial is conducted. A pharma company (the capitalists) writes the protocol the way they want, and hires a university/researcher to run the study. They file a simple two page form with the FDA to apply for an "indication" to study a treatment. The university/researcher gets the university IRB to sign off on the study protocol. Sometimes, if the IRB is any good at all, they'll tweak the protocol to try to protect the research subjects. But guess who picks the IRB members? The university/researcher making money off the capitalist pharma company. Guess how that influences an IRB. The FDA has no input. There is no supervision in any meaningful sense by the FDA. The capitalists - researcher and sponsor - run every aspect of the study. Each decision is theirs. The decisions spelled out in the article were made by the capitalists - the pharma company and researchers. Barring some complete misstep like all the active arm study patients suddenly dropping dead, the FDA will eventually (read: years after the study is completed) examine the data (typically highly massaged by the capitalists) from the study to make a determination on whether to approve a medication for sale. But pretending that the FDA (especially after years of being stripped of regulatory powers by, you guessed it, capitalists who buy and sell legislators) has much of anything to do with the way a clinical trial is conducted is complete bullshit.
xijuan 14y ago
This article has left me speechless.. It reminds me of what my stats prof said about this kind of research..Researchers doing this kind of research have to believe that they will find the results the pharmaceutical company wants before they conduct the study. In other words, they are not testing if the drug works; they are just showing that the drug works.
- tomjen3 14y ago
  Thats bad enough, but they essentially forced him (at the threat of imprisonment) to participate in a trial.
- refurb 14y ago
  Most of these clinical trials are triple-blinded. The doctor, patient and person analyzing the data don't know who took the drug and who took a placebo. How do they bias the results in that case?
  - DanBC 14y ago
    Not all studies are effectively blinded, nor effectively randomised. Merck-funded researchers trivialised the problems of Vioxx. Those researchers were not evil or stupid. Subtle influences are hard to spot in yourself. When you look at the data there's a remarkable gap between independently funded studies and company funded studies. It's very clear that studies funded by companies more often show benefits and less often show harm. (There are several published studies on this.) And even if the blinding and randomising is effective there's no requirement to publish. Thus, a company can run ten trials and only publish the one that's positive.
  - ChristinaM 14y ago
    By asking the right questions. In this case by aiming to find no significant difference between the 3 drugs under study then making sure the sample size was too small to find a difference no matter what.
  - ScottBurson 14y ago
    In some cases, according to the article, by manipulating the dosages. The drugs they don't want to do well are given at either too low a dose, so as to be ineffective, or too high, so as to have an excess of side effects.
nodata 14y ago
Ben Goldacre has been pushing to fix this. See his campaign site: http://www.alltrials.net/ http://www.alltrials.net/ His blog/news site: http://www.badscience.net/ http://www.badscience.net/ His Ted talk: http://www.ted.com/talks/ben_goldacre_what_doctors_don_t_know_about_the_drugs_they_prescribe.html http://www.ted.com/talks/ben_goldacre_what_doctors_don_t_kno...
- pygy_ 14y ago
  The problem goes deeper than that. "Healthcare industry" is an oxymoron. The incentives of pharmaceutical firms and patients are not aligned, and will not for the foreseeable future (not that I have a better model to offer :-/).
  - danpalmer 14y ago
    One suggested solution is to remove all patenting on drugs. That way, competition will force prices down, and result in two things. In countries with national healthcare such as the UK, the NHS bill could be drastically reduced, and the government could give grants to drug companies for R&D purposes. In countries with no healthcare system like the US, the general population will have much more money, and so taxes could be increased (hopefully not all the way back up) and grants could be given again. For those who think that drug companies need high prices for recouping R&D costs, their marketing budgets are far greater than R&D budgets, so this isn't really true. In the UK they aren't allowed to market to end-users, so the advertising costs are generally much lower here.
    - grandinj 14y ago
      Their R&D budgets are still pretty huge. And how exactly do you compete when your competitors can clone your product with ease? And the NHS spends < 10% of drugs, most of which are already available as generics, so there is not a lot of scope for cost reduction. I'm not saying that aren't improvements that can be made, but people need to wake up and realise that there are no simple answers.
      - 14y ago
wpietri 14y ago
You know, I am a big fan of capitalism. Having started 5 companies, I'd have to be. But MBA-style capitalism is a technology for producing short-term wins and substantial profits next quarter. If you also care about long-term wins or negative externalities, it's an insufficient technology to apply on its own. And of course, we hackers know that. Jeff Bezos has been telling the quarterly-profits people to go fuck themselves for 15 years. And he has been right to do so. AWS, Amazon Prime, the Kindle, and now their local delivery stuff: all impossible to achieve if you're more interested in beating quarterly profit predictions than reshaping the world. Health care is another area where it doesn't work so well. Consider, for example, these graphs: http://www.washingtonpost.com/blogs/wonkblog/wp/2013/03/26/21-graphs-that-show-americas-health-care-prices-are-ludicrous/ http://www.washingtonpost.com/blogs/wonkblog/wp/2013/03/26/2... I should note that allowing pharma companies to advertise to the public is a relatively recent change. 1997, I think. And the US is one of only two countries that allow it. We could change it back.
- protomyth 14y ago
  Its not really the advertising that is a problem. That is the symptom of the short time frame most drugs need to pay for themselves. Quite a lot of them only have 3 - 5 years to make the money back. So, either we figure out a way to make drugs cost less or we extend the timeframe for profit. Killing the advertising just means the number sold will be less and much higher for those buying it.
  - wpietri 14y ago
    If they are doing studies to generate marketing claims, then advertising has to be part of the problem, in that it's a major vector for marketing claims. There could be other vectors, and I would be happy to eliminate those as well. The total amount of money spent on pharmaceutical marketing is enormous. We would be much better served if doctors could pay for subscriptions to research companies that only take money from doctors.
    - protomyth 14y ago
      They are doing studies to find drugs that treat a condition. If a drug looks like it can treat something new, then they do another study. They are required to report side effects (that "may cause..." in ever advertisement) found in the study. "We would be much better served if doctors could pay for subscriptions to research companies that only take money from doctors." For what purpose? If your thinking a Forrester or Gartner model, then given how bad those companies advice has been in IT, I can only imagine what would happen in medicine.
tel 14y ago
Note that this is already required by law for FDA regulated drugs and devices: http://clinicaltrials.gov http://clinicaltrials.gov
- capnrefsmmat 14y ago
  These requirements are most often ignored. The law requires that trials be registered and their results published within a year of completion, but most registered trials never have published results: http://www.bmj.com/content/344/bmj.d7373 http://www.bmj.com/content/344/bmj.d7373 The results are simply buried if they're not sufficiently favorable. There have been a number of meta-analyses which dig up unpublished results (which is very difficult) and demonstrate that medications no longer have statistically significant effects when you include the unpublished data. For example: http://www.bmj.com/content/341/bmj.c4737 http://www.bmj.com/content/341/bmj.c4737 (Both these papers are open access, so you can read them if you'd like.)
  - tel 14y ago
    Cool, thanks for that research. I'm very interested in this problem today.
tokenadult 14y ago
"September/October 2010 Issue" The case reported in the Mother Jones article happened at my alma mater, a state university with many biomedical research studies going on at any given time. The article author later published a follow-up piece http://www.thehastingscenter.org/Bioethicsforum/Post.aspx?id=5147&blogid=140 http://www.thehastingscenter.org/Bioethicsforum/Post.aspx?id... in another publication. Two articles by different authors in local publications last year http://blogs.citypages.com/blotter/2012/11/dan_markingsons_suicide_jean_kenney.php http://blogs.citypages.com/blotter/2012/11/dan_markingsons_s... and just last month http://www.minnpost.com/second-opinion/2013/03/controversy-over-u-m-clinical-trial-has-heated-again http://www.minnpost.com/second-opinion/2013/03/controversy-o... provide further updates about this particular local case. A group blog post about medical research from 6 January 2012, "Why Do We Really Need Clinical Trials?" http://www.sciencebasedmedicine.org/index.php/why-do-we-really-need-clinical-trials/ http://www.sciencebasedmedicine.org/index.php/why-do-we-real... examines some of the trade-offs involved in the still flawed, but still necessary, process of conducting clinical trials of proposed new treatments. Human beings who propose new treatments often have a bias to suppose that the new treaments are more safe and more effective than previous treatments--especially if they have a financial interest in the new treatment. People who have a financial interest in other treatments, or in not seeking effective treatment at all (e.g., faith healers) may have a bias that runs in the other direction. The way to check the safety and effectiveness of a treatment is to run an honest clinical trial with adequate justification from known science and sufficient statistical power. But keeping the trial honest involves human beings disciplining themselves not to jump to conclusions. As Richard Feynman said to graduates of Caltech: "The first principle is that you must not fool yourself--and you are the easiest person to fool."

111 points by kumarski 14y ago

Regulatory capture. Used to be a term we used to criticize corrupt developing countries. Nowadays it appears to be at the root of so many problems facing America.

cynicalkane 14y ago
What are you talking about? Regulatory capture assumes a working rule of law, and describes a process by which people subject to the rules of the game try to rewrite them. In corrupt developing countries the power owns the rules outright, and regulatory capture is no longer a process. I guess you could call it "trivial regulatory capture" but it won't get you any Nobel Prizes. The theory was developed by Chicago School economists who liked to criticize big government. Regulatory capture has always been a bug of advanced, sufficiently honest societies. This is important because it tells us that well-intentioned rule of law is insufficient for running a democracy if you're not paying attention. I don't think anyone at the FDA wakes up and says, "I think I'll spend today getting in the way of good medical care and helping big pharma make money."
- mschuster91 14y ago
  The problem is the privatization of critical infrastructure, a cancer spreading all over the world. To quote the last sentence of the article: "Many academic physicians are required by their universities to generate a substantial portion of their salaries by obtaining research grants." Healthcare, basic transportation, energy and water distribution as well as basic telecommunication infrastructure must be in the hands of the state. The free market has massively failed in each of these areas, everywhere in the world. Just look at the US electricity grid, the Berlin water privatization disaster (less service, more expensive), the US (and partially, German too) broadband Internet service, the Berlin S-Bahn problems, the (recently HN frontpaged) problems in the US healthcare system... Why can't we learn from the countless mistakes made all over the world?!
  - refurb 14y ago
    I think we have learned from the mistake made all around the world, particularly the ones where government run organizations lose billions of dollars through their shoddy operation of industries.

lemming 14y ago

For anyone interested in the generally fucked up state of clinical drug trials, I highly recommend Ben Goldacre's excellent book Bad Pharma. It'll make you angry.

Create 14y ago
Bad Science also does the job. The Emperor of All Maladies gives a candid and almost frank overview on how clinical drug trials are performed "from the inside". Aspirin the Story of a Wonder Drug is also a nice account of "drug development" in practice.

mschuster91 14y ago

This makes me feel so sad. I think the biggest lesson to learn from this case is that there has to be a independent entity in double-blind studies which knows the relation between test subject and the substance and has the power to interrupt or terminate a study when people's lifes are at risk. Again, another fine example of how capitalism does not take unforeseen issues into account - to refuse to drop out Dan of the study program is "manslaughter by neglicence" for me (not by malice, as the responsible actors could not have definite knowledge that Dan would kill himself).

protomyth 14y ago
You site capitalism, when all rules governing a clinical trial are strictly enforced by the FDA which is a government entity. The FDA has study rules and the entity conducting the study is not the drug company. It is an independent entity (for example http://clinicaltrials.mayo.edu http://clinicaltrials.mayo.edu) paid by the drug company under the supervision of the FDA (in the US).
- logjam 14y ago
  That is some really utterly fascinating spin right there. Next up: criminals are not really responsible for crime, since "government entities" like municipal police forces supposedly "strictly enforce" criminal conduct. I'm going to take a wild guess here and assume you've never had anything whatsoever to do with a clinical trial, because the modern FDA has almost nothing to do with the way a clinical trial is conducted. A pharma company (the capitalists) writes the protocol the way they want, and hires a university/researcher to run the study. They file a simple two page form with the FDA to apply for an "indication" to study a treatment. The university/researcher gets the university IRB to sign off on the study protocol. Sometimes, if the IRB is any good at all, they'll tweak the protocol to try to protect the research subjects. But guess who picks the IRB members? The university/researcher making money off the capitalist pharma company. Guess how that influences an IRB. The FDA has no input. There is no supervision in any meaningful sense by the FDA. The capitalists - researcher and sponsor - run every aspect of the study. Each decision is theirs. The decisions spelled out in the article were made by the capitalists - the pharma company and researchers. Barring some complete misstep like all the active arm study patients suddenly dropping dead, the FDA will eventually (read: years after the study is completed) examine the data (typically highly massaged by the capitalists) from the study to make a determination on whether to approve a medication for sale. But pretending that the FDA (especially after years of being stripped of regulatory powers by, you guessed it, capitalists who buy and sell legislators) has much of anything to do with the way a clinical trial is conducted is complete bullshit.

xijuan 14y ago

This article has left me speechless.. It reminds me of what my stats prof said about this kind of research..Researchers doing this kind of research have to believe that they will find the results the pharmaceutical company wants before they conduct the study. In other words, they are not testing if the drug works; they are just showing that the drug works.

tomjen3 14y ago
Thats bad enough, but they essentially forced him (at the threat of imprisonment) to participate in a trial.
refurb 14y ago
Most of these clinical trials are triple-blinded. The doctor, patient and person analyzing the data don't know who took the drug and who took a placebo. How do they bias the results in that case?
- DanBC 14y ago
  Not all studies are effectively blinded, nor effectively randomised. Merck-funded researchers trivialised the problems of Vioxx. Those researchers were not evil or stupid. Subtle influences are hard to spot in yourself. When you look at the data there's a remarkable gap between independently funded studies and company funded studies. It's very clear that studies funded by companies more often show benefits and less often show harm. (There are several published studies on this.) And even if the blinding and randomising is effective there's no requirement to publish. Thus, a company can run ten trials and only publish the one that's positive.
- ChristinaM 14y ago
  By asking the right questions. In this case by aiming to find no significant difference between the 3 drugs under study then making sure the sample size was too small to find a difference no matter what.
- ScottBurson 14y ago
  In some cases, according to the article, by manipulating the dosages. The drugs they don't want to do well are given at either too low a dose, so as to be ineffective, or too high, so as to have an excess of side effects.

nodata 14y ago

Ben Goldacre has been pushing to fix this. See his campaign site: http://www.alltrials.net/ http://www.alltrials.net/ His blog/news site: http://www.badscience.net/ http://www.badscience.net/ His Ted talk: http://www.ted.com/talks/ben_goldacre_what_doctors_don_t_know_about_the_drugs_they_prescribe.html http://www.ted.com/talks/ben_goldacre_what_doctors_don_t_kno...

pygy_ 14y ago
The problem goes deeper than that. "Healthcare industry" is an oxymoron. The incentives of pharmaceutical firms and patients are not aligned, and will not for the foreseeable future (not that I have a better model to offer :-/).
- danpalmer 14y ago
  One suggested solution is to remove all patenting on drugs. That way, competition will force prices down, and result in two things. In countries with national healthcare such as the UK, the NHS bill could be drastically reduced, and the government could give grants to drug companies for R&D purposes. In countries with no healthcare system like the US, the general population will have much more money, and so taxes could be increased (hopefully not all the way back up) and grants could be given again. For those who think that drug companies need high prices for recouping R&D costs, their marketing budgets are far greater than R&D budgets, so this isn't really true. In the UK they aren't allowed to market to end-users, so the advertising costs are generally much lower here.
  - grandinj 14y ago
    Their R&D budgets are still pretty huge. And how exactly do you compete when your competitors can clone your product with ease? And the NHS spends < 10% of drugs, most of which are already available as generics, so there is not a lot of scope for cost reduction. I'm not saying that aren't improvements that can be made, but people need to wake up and realise that there are no simple answers.
    - 14y ago

wpietri 14y ago

You know, I am a big fan of capitalism. Having started 5 companies, I'd have to be. But MBA-style capitalism is a technology for producing short-term wins and substantial profits next quarter. If you also care about long-term wins or negative externalities, it's an insufficient technology to apply on its own. And of course, we hackers know that. Jeff Bezos has been telling the quarterly-profits people to go fuck themselves for 15 years. And he has been right to do so. AWS, Amazon Prime, the Kindle, and now their local delivery stuff: all impossible to achieve if you're more interested in beating quarterly profit predictions than reshaping the world. Health care is another area where it doesn't work so well. Consider, for example, these graphs: http://www.washingtonpost.com/blogs/wonkblog/wp/2013/03/26/21-graphs-that-show-americas-health-care-prices-are-ludicrous/ http://www.washingtonpost.com/blogs/wonkblog/wp/2013/03/26/2... I should note that allowing pharma companies to advertise to the public is a relatively recent change. 1997, I think. And the US is one of only two countries that allow it. We could change it back.

protomyth 14y ago
Its not really the advertising that is a problem. That is the symptom of the short time frame most drugs need to pay for themselves. Quite a lot of them only have 3 - 5 years to make the money back. So, either we figure out a way to make drugs cost less or we extend the timeframe for profit. Killing the advertising just means the number sold will be less and much higher for those buying it.
- wpietri 14y ago
  If they are doing studies to generate marketing claims, then advertising has to be part of the problem, in that it's a major vector for marketing claims. There could be other vectors, and I would be happy to eliminate those as well. The total amount of money spent on pharmaceutical marketing is enormous. We would be much better served if doctors could pay for subscriptions to research companies that only take money from doctors.
  - protomyth 14y ago
    They are doing studies to find drugs that treat a condition. If a drug looks like it can treat something new, then they do another study. They are required to report side effects (that "may cause..." in ever advertisement) found in the study. "We would be much better served if doctors could pay for subscriptions to research companies that only take money from doctors." For what purpose? If your thinking a Forrester or Gartner model, then given how bad those companies advice has been in IT, I can only imagine what would happen in medicine.

tel 14y ago

Note that this is already required by law for FDA regulated drugs and devices: http://clinicaltrials.gov http://clinicaltrials.gov

capnrefsmmat 14y ago
These requirements are most often ignored. The law requires that trials be registered and their results published within a year of completion, but most registered trials never have published results: http://www.bmj.com/content/344/bmj.d7373 http://www.bmj.com/content/344/bmj.d7373 The results are simply buried if they're not sufficiently favorable. There have been a number of meta-analyses which dig up unpublished results (which is very difficult) and demonstrate that medications no longer have statistically significant effects when you include the unpublished data. For example: http://www.bmj.com/content/341/bmj.c4737 http://www.bmj.com/content/341/bmj.c4737 (Both these papers are open access, so you can read them if you'd like.)
- tel 14y ago
  Cool, thanks for that research. I'm very interested in this problem today.

tokenadult 14y ago

"September/October 2010 Issue" The case reported in the Mother Jones article happened at my alma mater, a state university with many biomedical research studies going on at any given time. The article author later published a follow-up piece http://www.thehastingscenter.org/Bioethicsforum/Post.aspx?id=5147&blogid=140 http://www.thehastingscenter.org/Bioethicsforum/Post.aspx?id... in another publication. Two articles by different authors in local publications last year http://blogs.citypages.com/blotter/2012/11/dan_markingsons_suicide_jean_kenney.php http://blogs.citypages.com/blotter/2012/11/dan_markingsons_s... and just last month http://www.minnpost.com/second-opinion/2013/03/controversy-over-u-m-clinical-trial-has-heated-again http://www.minnpost.com/second-opinion/2013/03/controversy-o... provide further updates about this particular local case. A group blog post about medical research from 6 January 2012, "Why Do We Really Need Clinical Trials?" http://www.sciencebasedmedicine.org/index.php/why-do-we-really-need-clinical-trials/ http://www.sciencebasedmedicine.org/index.php/why-do-we-real... examines some of the trade-offs involved in the still flawed, but still necessary, process of conducting clinical trials of proposed new treatments. Human beings who propose new treatments often have a bias to suppose that the new treaments are more safe and more effective than previous treatments--especially if they have a financial interest in the new treatment. People who have a financial interest in other treatments, or in not seeking effective treatment at all (e.g., faith healers) may have a bias that runs in the other direction. The way to check the safety and effectiveness of a treatment is to run an honest clinical trial with adequate justification from known science and sufficient statistical power. But keeping the trial honest involves human beings disciplining themselves not to jump to conclusions. As Richard Feynman said to graduates of Caltech: "The first principle is that you must not fool yourself--and you are the easiest person to fool."