7 ms·

FDA and NIH let clinical trial sponsors keep results secret and break the law

289 points by barry-cotter 7y ago

Expez 7y ago
I just listened to the book Bad Pharma, by Ben Goldacre, which deals with this problem. The author works as a doctor and so has a bit of an inside view of how the industry does 'science' and how this affects doctors and patients. The lack of published negative results isn't a problem unique to pharma but the drive for profits and the actual life and death outcomes makes it particular severe in this area of science, I think. I highly recommend the book, it was both hilarious and terrifying.
- Angostura 7y ago
  Don’t forget to support Ben’s campaign to change this. https://www.alltrials.net/ https://www.alltrials.net/
- ekianjo 7y ago
  There is also an opportunity for public databases of patient outcomes (post marketing) that we would all greatly benefit from. A lot of that data is still discussed behind closed doors and authorities are also to blame because it is their prerogative to force data to be open if they wish to.
  - pintxo 7y ago
    We are working on it, but financing seems to be tricky for such an endeavor.
  - jcims 7y ago
    Sounds like a FUD magnet but I agree there would be major upside. Edit: To clarify, publishing individual patient outcomes publicly could pretty easily be spun as a privacy risk to trial participants and/or have a chilling effect on enrollment. I’m personally invested in seeing something like this happen but if not approached defensively it could be vulnerable to getting shot down and made an example of.
- sanxiyn 7y ago
  I must note that according to the linked article, "pharma’s record has been markedly better than that of academia and the federal government". It's academia that is not reporting results, not industry.
  - hannob 7y ago
    That's an illogical conclusion. Both don't always report results. Academia is worse. That doesn't make the industry good in any way.
    - Amygaz 7y ago
      I don’t know what problem that would solve. Academia rarely does trials that end up killing people. Unlike the whole CAR-T cell therapy. Companies like Juno killed almost 50% of their patient, before the FDA halted their trial, which was allowed to resume later, and kill more people. Granted they where going to die anyway. Still the like of Hans Bishop who us held as a hero in the industry because he sold his company for billions, while the product will never be approved. Even if were to be, it would sit on a shelf because it’s too expensive. So much for transparency... (Edited some typos)
      - sanxiyn 7y ago
        > while the product will never be approved I must point out that some CAR-T cell therapy is already approved. I am not sure about Juno's particular candidate, but it isn't obviously insane that it was sold for billions.
      - jcims 7y ago
        I’m standing right now near the window on the 20th floor of a cancer center, typing this on my phone while my wife sleeps 10 feet from me. This is high stakes medicine, yet the current environment has the docs on this floor terrified of doing anything that isn’t in the standard of care runbook. My wife has had elevated blood glucose levels for six months, which is not good when you’re fighting cancer, yet i can’t get anyone to treat it because there’s no clinical evidence that managing glucose levels for her specific type of cancer has any benefit. Guess what. Nobody’s doing that trial any time soon. Meanwhile there are thousands of diabetics managing their blood glucose while fighting cancer and there is endless material available demonstrating that cancer cells thrive in hyperglycemic environments. So, i don’t know anything about Juno, they could be sociopaths. But i have no issues with high risk attempts as long as the patients are informed and there are ethical standards for transparency. At least they are trying.
- rjmunro 7y ago
  The article states "based on data collected from the TrialsTracker website of the University of Oxford", which is Ben Goldacre's project.
- avip 7y ago
  I'd also recommend his TED talks which are really amusing and educating. https://www.ted.com/talks/ben_goldacre_battling_bad_science https://www.ted.com/talks/ben_goldacre_battling_bad_science https://www.ted.com/talks/ben_goldacre_what_doctors_don_t_know_about_the_drugs_they_prescribe?language=en https://www.ted.com/talks/ben_goldacre_what_doctors_don_t_kn...
jMyles 7y ago
I think this is the primary purpose of the FDA (though not the NIH). Providing a shield from transparency, and even against criminal liability, is just another part of regulatory capture.
rajman187 7y ago
Perhaps unbeknownst to some folks, the FDA as a regulatory body is funded in large part now by Congress but by GDUFA and PDUFA which allows the agency to collect user fees for both brand and generic drugs from the drug makers themselves, in essence being funded by the industry it is meant to regulate. So it is then understandable why a lot of the guidelines become flexible (note that the FDA cannot enact laws, that is the duty of Congress)
- conistonwater 7y ago
  This is a very common complaint, but the other side of the coin is that FDA needs resources to do its job and any time the number of submitted drugs changes the FDA would need less/more funding from congress if it was funded only by congress. Collecting fees means the size of FDA's funding is correlated with the amount of work it has to do. The proper way to determine whether FDA is unduly influenced by this is to look at the actual decisions it makes, which is something I remember reading medical researchers already tried to do.
  - rajman187 7y ago
    GDUFA guarantees a decision within 10 weeks, which is considerably better than the running average before and even extreme cases such as 13 years before addressing applications. There are unfortunately many cases that have been decided on the basis of finances rather than clinical justifications. I don’t envy the agency, stuck between the disdain of the public, pressure from industry, and continually having to justify their existence to Congressional committees.
- loceng 7y ago
  This is one place where I would like to see an immutable blockchain database used, as a custody of records for decisions made.
air7 7y ago
Coincidentally, I've just heard Milton Friedman talk about "Health Care in a Free Market". Fascinating and eloquent. He argues that the FDA did more harm than good, and that society would be better off without a governmental supervision for health care in general. -- https://www.youtube.com/watch?v=-6t-R3pWrRw https://www.youtube.com/watch?v=-6t-R3pWrRw btw, in the end he makes a strong argument for negative income tax. This surprised me, as my model of his world-view was apparently more simplistic that it ought to be.
- anthony_doan 7y ago
  I disagree. FDA have set guidelines for clinical trials and moral standards to it. For an example you cannot experiment on a person without their consent and you have to state out what your trials is doing. It all lay out. It's because of Tuskegee syphilis experiment that we have IRB and FDA. FDA also love stating that they prevented a drug that causes birth defect a while back unlike Britain. Another example of government regulation working is cars. Death relating to car crashes have fallen every year with safety regulations. It's not perfect but there are many cases that it works and there are laws and rules set in place. The free market is not the panacea for everything.
  - will4274 7y ago
    Falling car crash rates don't prove regulation is successful. Consumers shop for safety too. It's difficult to compare the counterfactual, but to make this claim, you need to at least compare the rate of change of crashes in unregulated countries or times to regulated countries or times.
    - kube-system 7y ago
      For both auto safety and pharmaceutical safety there is a large degree of consumer information disparity, information unavailability, and disincentive for the individual to protect the public interest. It takes a lot of resources to test those products, and consumers don't really have the resources to do it themselves. In some cases, insurers could possibly foot the bill (eg. IIHS), but that model still has a couple of big weaknesses: consumers may not be informed enough to parse the results, consumers might prioritize short-term needs over public health, and an insurance body may still have some incentives working against public benefit. Things like truck under-ride guards likely wouldn't be on commercial trucks if they weren't regulated. Consumers aren't going to all voluntarily coordinate to buy vehicles which have crash-compatible bumper heights. Antibiotic misuse would be even more out of control if it weren't regulated. Mexico is a good example for road regulations. They do not have the same crash regulations as the US, and many old model vehicles are still legal for sale there. Things like the 1950's VW Beetle were still sold there into the 2000s. They have the highest death rate per distance traveled in the world.
chiefalchemist 7y ago
> Few trial sponsors have consistently done so, even after a 2007 law made posting mandatory for many trials registered in the database. Then the reality is, it wasn't mandatory. It's not reasonable to expect citizens to trust these entities when their conduct is not trustworthy.
qrbLPHiKpiux 7y ago
The FDA operates as a corrupt organization and, if wasn’t a government agency, would be prosecuted under RICO.
walterkrankheit 7y ago
I'm kind of shocked that logging these results isn't already common practice.
Ididntdothis 7y ago
If there ever has been a sector where mandatory full transparency is useful then medicine would be a good candidate. The outcome of trials could be very useful for other researchers. Same should apply for pricing. The fact that prices are completely opaque in the US makes it hard for patients to make good decisions.
calibas 7y ago
Hiding research results should be seen as this scandalous, shameful, and fundamentally anti-scientific practice. A heavy bias towards business profits, which is exactly what this is, goes against objective science. What if you kept repeating experiments, and only published when you got the results you wanted? That goes against my personal ethics and my understanding of how science is supposed to work.
- jcranmer 7y ago
  > A heavy bias towards business profits, which is exactly what this is, goes against objective science. Except the data shows that it's not the industry who's failing to report results, it's academia and the government who's failing.
  - calibas 7y ago
    When people like Michael Taylor run the FDA, that's clearly agency capture. The industry has infiltrated government positions, I don't know how anybody can act like they're innocent in all this.
- Spivak 7y ago
  > What if you kept repeating experiments, and only published when you got the results you wanted? This is pretty much how it's currently done. Science is far less exploratory than you make it seem. When it comes to publishing you should think of it more like law. You make the strongest possible case for your conclusion.
  - ouid 7y ago
    This is very backwards. In science, the evidence for a claim is precisely how hard you tried to disprove it. This doesn't mean that you're wrong, of course, but it's a massive problem.
    - Retric 7y ago
      That’s only a subset of experiments, the ideal experiment makes any result publishable. For example the search for a new particle, don’t find it that’s fine. Alternatively, any result that’s in doubt like salt’s impact on cardiovascular heath. Even better is really cheap experiments like analysis of existing data. That you can keep trying until you find something, sadly this means publishing datasets is a bad idea.
tracker1 7y ago
The pharmaceutical industry is full of unscrupulous bastards, news at 11.
andrewla 7y ago
I don't care about whether they report results or if they miss deadlines for reporting. That's all this article is talking about. But only studies that are pre-registered should be considered when approving a drug, treatment, or device, and any incomplete studies (that is pre-registered but not reported for any reasons) should be treated as "all the patients died", and all individual researchers on the preregistration should be appropriately tainted for future consideration of studies in which they participate. The related story in the article, "Gaming the system" does report on this: > .. a recent study ... examined 19 drugs approved by the FDA in 2015. In only 10 cases were all the trials provided to FDA for its decisions registered on ClinicalTrials.gov. This is unacceptable.
- mjevans 7y ago
  All of the punishments you suggested and also any involved patents released to the public domain since the data wasn't.

289 points by barry-cotter 7y ago

I just listened to the book Bad Pharma, by Ben Goldacre, which deals with this problem. The author works as a doctor and so has a bit of an inside view of how the industry does 'science' and how this affects doctors and patients. The lack of published negative results isn't a problem unique to pharma but the drive for profits and the actual life and death outcomes makes it particular severe in this area of science, I think. I highly recommend the book, it was both hilarious and terrifying.

Angostura 7y ago
Don’t forget to support Ben’s campaign to change this. https://www.alltrials.net/ https://www.alltrials.net/
ekianjo 7y ago
There is also an opportunity for public databases of patient outcomes (post marketing) that we would all greatly benefit from. A lot of that data is still discussed behind closed doors and authorities are also to blame because it is their prerogative to force data to be open if they wish to.
- pintxo 7y ago
  We are working on it, but financing seems to be tricky for such an endeavor.
- jcims 7y ago
  Sounds like a FUD magnet but I agree there would be major upside. Edit: To clarify, publishing individual patient outcomes publicly could pretty easily be spun as a privacy risk to trial participants and/or have a chilling effect on enrollment. I’m personally invested in seeing something like this happen but if not approached defensively it could be vulnerable to getting shot down and made an example of.
sanxiyn 7y ago
I must note that according to the linked article, "pharma’s record has been markedly better than that of academia and the federal government". It's academia that is not reporting results, not industry.
- hannob 7y ago
  That's an illogical conclusion. Both don't always report results. Academia is worse. That doesn't make the industry good in any way.
  - Amygaz 7y ago
    I don’t know what problem that would solve. Academia rarely does trials that end up killing people. Unlike the whole CAR-T cell therapy. Companies like Juno killed almost 50% of their patient, before the FDA halted their trial, which was allowed to resume later, and kill more people. Granted they where going to die anyway. Still the like of Hans Bishop who us held as a hero in the industry because he sold his company for billions, while the product will never be approved. Even if were to be, it would sit on a shelf because it’s too expensive. So much for transparency... (Edited some typos)
    - sanxiyn 7y ago
      > while the product will never be approved I must point out that some CAR-T cell therapy is already approved. I am not sure about Juno's particular candidate, but it isn't obviously insane that it was sold for billions.
    - jcims 7y ago
      I’m standing right now near the window on the 20th floor of a cancer center, typing this on my phone while my wife sleeps 10 feet from me. This is high stakes medicine, yet the current environment has the docs on this floor terrified of doing anything that isn’t in the standard of care runbook. My wife has had elevated blood glucose levels for six months, which is not good when you’re fighting cancer, yet i can’t get anyone to treat it because there’s no clinical evidence that managing glucose levels for her specific type of cancer has any benefit. Guess what. Nobody’s doing that trial any time soon. Meanwhile there are thousands of diabetics managing their blood glucose while fighting cancer and there is endless material available demonstrating that cancer cells thrive in hyperglycemic environments. So, i don’t know anything about Juno, they could be sociopaths. But i have no issues with high risk attempts as long as the patients are informed and there are ethical standards for transparency. At least they are trying.
rjmunro 7y ago
The article states "based on data collected from the TrialsTracker website of the University of Oxford", which is Ben Goldacre's project.
avip 7y ago
I'd also recommend his TED talks which are really amusing and educating. https://www.ted.com/talks/ben_goldacre_battling_bad_science https://www.ted.com/talks/ben_goldacre_battling_bad_science https://www.ted.com/talks/ben_goldacre_what_doctors_don_t_know_about_the_drugs_they_prescribe?language=en https://www.ted.com/talks/ben_goldacre_what_doctors_don_t_kn...

jMyles 7y ago

I think this is the primary purpose of the FDA (though not the NIH). Providing a shield from transparency, and even against criminal liability, is just another part of regulatory capture.

rajman187 7y ago

Perhaps unbeknownst to some folks, the FDA as a regulatory body is funded in large part now by Congress but by GDUFA and PDUFA which allows the agency to collect user fees for both brand and generic drugs from the drug makers themselves, in essence being funded by the industry it is meant to regulate. So it is then understandable why a lot of the guidelines become flexible (note that the FDA cannot enact laws, that is the duty of Congress)

conistonwater 7y ago
This is a very common complaint, but the other side of the coin is that FDA needs resources to do its job and any time the number of submitted drugs changes the FDA would need less/more funding from congress if it was funded only by congress. Collecting fees means the size of FDA's funding is correlated with the amount of work it has to do. The proper way to determine whether FDA is unduly influenced by this is to look at the actual decisions it makes, which is something I remember reading medical researchers already tried to do.
- rajman187 7y ago
  GDUFA guarantees a decision within 10 weeks, which is considerably better than the running average before and even extreme cases such as 13 years before addressing applications. There are unfortunately many cases that have been decided on the basis of finances rather than clinical justifications. I don’t envy the agency, stuck between the disdain of the public, pressure from industry, and continually having to justify their existence to Congressional committees.
loceng 7y ago
This is one place where I would like to see an immutable blockchain database used, as a custody of records for decisions made.

air7 7y ago

Coincidentally, I've just heard Milton Friedman talk about "Health Care in a Free Market". Fascinating and eloquent. He argues that the FDA did more harm than good, and that society would be better off without a governmental supervision for health care in general. -- https://www.youtube.com/watch?v=-6t-R3pWrRw https://www.youtube.com/watch?v=-6t-R3pWrRw btw, in the end he makes a strong argument for negative income tax. This surprised me, as my model of his world-view was apparently more simplistic that it ought to be.

anthony_doan 7y ago
I disagree. FDA have set guidelines for clinical trials and moral standards to it. For an example you cannot experiment on a person without their consent and you have to state out what your trials is doing. It all lay out. It's because of Tuskegee syphilis experiment that we have IRB and FDA. FDA also love stating that they prevented a drug that causes birth defect a while back unlike Britain. Another example of government regulation working is cars. Death relating to car crashes have fallen every year with safety regulations. It's not perfect but there are many cases that it works and there are laws and rules set in place. The free market is not the panacea for everything.
- will4274 7y ago
  Falling car crash rates don't prove regulation is successful. Consumers shop for safety too. It's difficult to compare the counterfactual, but to make this claim, you need to at least compare the rate of change of crashes in unregulated countries or times to regulated countries or times.
  - kube-system 7y ago
    For both auto safety and pharmaceutical safety there is a large degree of consumer information disparity, information unavailability, and disincentive for the individual to protect the public interest. It takes a lot of resources to test those products, and consumers don't really have the resources to do it themselves. In some cases, insurers could possibly foot the bill (eg. IIHS), but that model still has a couple of big weaknesses: consumers may not be informed enough to parse the results, consumers might prioritize short-term needs over public health, and an insurance body may still have some incentives working against public benefit. Things like truck under-ride guards likely wouldn't be on commercial trucks if they weren't regulated. Consumers aren't going to all voluntarily coordinate to buy vehicles which have crash-compatible bumper heights. Antibiotic misuse would be even more out of control if it weren't regulated. Mexico is a good example for road regulations. They do not have the same crash regulations as the US, and many old model vehicles are still legal for sale there. Things like the 1950's VW Beetle were still sold there into the 2000s. They have the highest death rate per distance traveled in the world.

chiefalchemist 7y ago

> Few trial sponsors have consistently done so, even after a 2007 law made posting mandatory for many trials registered in the database. Then the reality is, it wasn't mandatory. It's not reasonable to expect citizens to trust these entities when their conduct is not trustworthy.

qrbLPHiKpiux 7y ago

The FDA operates as a corrupt organization and, if wasn’t a government agency, would be prosecuted under RICO.

walterkrankheit 7y ago

I'm kind of shocked that logging these results isn't already common practice.

Ididntdothis 7y ago

If there ever has been a sector where mandatory full transparency is useful then medicine would be a good candidate. The outcome of trials could be very useful for other researchers. Same should apply for pricing. The fact that prices are completely opaque in the US makes it hard for patients to make good decisions.

calibas 7y ago

Hiding research results should be seen as this scandalous, shameful, and fundamentally anti-scientific practice. A heavy bias towards business profits, which is exactly what this is, goes against objective science. What if you kept repeating experiments, and only published when you got the results you wanted? That goes against my personal ethics and my understanding of how science is supposed to work.

jcranmer 7y ago
> A heavy bias towards business profits, which is exactly what this is, goes against objective science. Except the data shows that it's not the industry who's failing to report results, it's academia and the government who's failing.
- calibas 7y ago
  When people like Michael Taylor run the FDA, that's clearly agency capture. The industry has infiltrated government positions, I don't know how anybody can act like they're innocent in all this.
Spivak 7y ago
> What if you kept repeating experiments, and only published when you got the results you wanted? This is pretty much how it's currently done. Science is far less exploratory than you make it seem. When it comes to publishing you should think of it more like law. You make the strongest possible case for your conclusion.
- ouid 7y ago
  This is very backwards. In science, the evidence for a claim is precisely how hard you tried to disprove it. This doesn't mean that you're wrong, of course, but it's a massive problem.
  - Retric 7y ago
    That’s only a subset of experiments, the ideal experiment makes any result publishable. For example the search for a new particle, don’t find it that’s fine. Alternatively, any result that’s in doubt like salt’s impact on cardiovascular heath. Even better is really cheap experiments like analysis of existing data. That you can keep trying until you find something, sadly this means publishing datasets is a bad idea.

tracker1 7y ago

The pharmaceutical industry is full of unscrupulous bastards, news at 11.

andrewla 7y ago

I don't care about whether they report results or if they miss deadlines for reporting. That's all this article is talking about. But only studies that are pre-registered should be considered when approving a drug, treatment, or device, and any incomplete studies (that is pre-registered but not reported for any reasons) should be treated as "all the patients died", and all individual researchers on the preregistration should be appropriately tainted for future consideration of studies in which they participate. The related story in the article, "Gaming the system" does report on this: > .. a recent study ... examined 19 drugs approved by the FDA in 2015. In only 10 cases were all the trials provided to FDA for its decisions registered on ClinicalTrials.gov. This is unacceptable.

mjevans 7y ago
All of the punishments you suggested and also any involved patents released to the public domain since the data wasn't.