4 ms·
> "Phase 3 double-blind randomized clinical trials are needed to provide definitive data." Let me LOL at this statement. If you think Phase 3 studies can't be
by ekianjo 5y ago
> "Phase 3 double-blind randomized clinical trials are needed to provide definitive data."
Let me LOL at this statement. If you think Phase 3 studies can't be hacked in one way or another, well please meet the Pharma industry with its hordes of 1) cherry picking patients for phase 3 recruitment, 2) cherry picking patients who should be removed from the analysis as "dropouts", 3) p-hacking and multiple other practices well known under the sun.
Believing that randomized Phase 3s remove all bias is incredibly naive.
- gumby 5y agoBased on my experience (I designed pharmaceutical clinical trials in an earlier life) this is much harder todo than it sounds. You have to start early (Phase 1) and what you end up with goes on your label (basically the package insert). You don’t always end up where you want to. The real shenanigans appear once your drug is approved.
- ekianjo 5y ago> this is much harder todo than it sounds. Cherry picking patients is not hard, since this is entirely up to the trial's recruitment criteria. This is also what makes comparing drugs virtually impossible since no 2 drugs in the same market apply the same recruitment criteria. This is how manufacturers invite the discussion about "superiority" while competitive drugs are almost never compared directly in the same trials.
- srean 5y agoI wonder if there are cryptography based secure protocols that can ensure that the studies have been carried out the proper way. Given your experience would love to hear your thoughts. My second question to you is to understand what apart from deep pockets prevent a trial to be repeated until an outcome favorable to the drug company is obtained. I understand pharma companies are encouraged to make all their trials public, but to me that seems like a generic guidance with no teeth.
- gumby 5y ago> I wonder if there are cryptography based secure protocols that can ensure that the studies have been carried out the proper way. Given your experience would love to hear your thoughts. Cryptographic methods would be overkill and yet couldn't be applied to the places where such fraud might occur. When you submit trial results the documentation is immense -- you submit pretty much everything. There is a long, highly documented chain of custody and a lot of people (and independent entities) involved so it would be very difficult to pull off a conspiracy. That's not where the problems lie. For example if the agency thinks something is irregular they can go back and look at the underling data and the instructions on how those data were collected. The amount of data supplied is so large that before the days of electronic filing every drug company had a group whose job was simply to manage the logistics of organizing the trucks needed to transport a single submission! That sounds therefore like there would be a needle-in-a-haystack problem, and of course there always is, but you'd be surprised by the tiny details you get asked about as someone sees something they don't understand or are suspicious about and just follow the trail backwards. > My second question to you is to understand what apart from deep pockets prevent a trial to be repeated until an outcome favorable to the drug company is obtained. I understand pharma companies are encouraged to make all their trials public, but to me that seems like a generic guidance with no teeth. So this is the place to look: at a high level. However what you ask is hard. You can't legally give humans unapproved drugs (the agency doesn't give you permission to do so; they say "you proposed to use X on people who meet criteria C, following a specify procedure P, in order to answer questions Q, R and S. As long as you do precisely as you asked, only on the population C and on no more people than discussed, we will not prosecute you for using an unlicensed drug". And you submit an enormous amount of data you have already collected that explains why it's worth giving it a try and why it isn't a risk to human life. So you do specific pre-clinical (animal) work just to get it into humans at all (phase 1), then use the safety data you collected to justify some kind of limited study of efficacy (Phase 2) and based on that you do a large, representative (of the people in the US who might need this drug) population of subjects. If you tried to repeat a trial the agency would say "but you already did this and didn't like the results, so no". And in fast why would you? You'd be desperately trying to figure out why it failed and try to do something different.
- missedthecue 5y agoPerhaps phase 3s aren't perfectly free of bias, but they are essentially impossible to effectively manipulate by the drugmaker. Even if we assume all the physicians involved in the trial are willing to be bribed, the pharma company doesn't know which ones are handing out placebos.
- ekianjo 5y ago> Perhaps phase 3s aren't perfectly free of bias, but they are essentially impossible to effectively manipulate by the drugmaker. Not completely accurate. Studies with negative outcomes are not required to be published by the drug maker. You can virtually keep trying the same drug and "get lucky" with a single trial, and just register that one with the authorities since the standards for replication are still very low.
- kuhewa 5y agoNo because you have to preregister all of them.
- ekianjo 5y agoPreregistering does not mean anything about sharing actual results. the large majority of negative outcomes are NEVER published. Please don't spread misinformation.
- kuhewa 5y agoPretty sure FDAAA has mandated reporting results for a while now (although some sponsors choose to break the law). Do you have an example of your scenario of an approved drug with many phase 3 trials pre-registered but only one with reported results?
- forcry 5y ago> pharma company doesn't know which ones are handing out placebos.. Not sure I get you. If all the physicians are bribed, then why would the company need to know anything? Everyone can cook up some data and go along with the result that the company needs, right? Also, about this >If all the physicians are bribed I think you should assume that any physician that ends up in a trial is there because they are known to be complaisant. Same as politicians and authority figures are often planted by powerful people with vested interests.
- unishark 5y agoBy phase 3 they should have to preregister the study and identify the outcomes they will measure. There isn't much room for p-hacking at this point. There will certainly always be bias, but it's still a gold standard for good reasons.
- forcry 5y ago>a gold standard for good reasons. What does this mean exactly? Is it the best possible? Or the best available? If it is the latter, then it does not mean much. If it is the former, then your statement is not true.
- unishark 5y agoI mean "good reasons" as in objectively good reasons. It's not a comparison. Preregistration is one of those reasons.
- ekianjo 5y ago> By phase 3 they should have to preregister the study and identify the outcomes they will measure. There isn't much room for p-hacking at this point. Not all Phase 3 with negative outcomes are published. It's actually the opposite. Most of them are not. So preregistration achieves nothing in terms of transparency. And p-hacking still happens far more often than you think - I have personally seen trials where they identified "sub-groups" of patients in order to prove that there were patients for which said drugs worked. That's a thing, no matter what you decided to set in the first place as objective. And then there's stuff like Biogen's alzheimer drug for which no efficacy was proven but still received FDA commercialization approval. Everything is working just peachy, right?
- kuhewa 5y agoNecessary but not sufficient. Those are issues that can plague all studies, but at least some sources of bias and error can be mitigated against with randomisation and blinding.