3 ms·

It is quite possible to succeed and still fail your trial. Which is one of the many reasons these things should be thrown out in favor of patient choice. You co

It is quite possible to succeed and still fail your trial. Which is one of the many reasons these things should be thrown out in favor of patient choice. You could spend 1/10 of the cost of later trials on publishing risk assessments and still be far ahead of the game of risk management. Another problem is that too many of today's medical technologies are horribly marginal in their benefits. Marginal benefits have a way of smoothing out to nothing as the patient population broadens. The industry isn't approaching medicine for age-related disease in the right way. If you are actually addressing a useful mechanism, the effect won't be marginal. Science managed that shift from marginal to effective for infectious disease. There is now a pending transition for the diseases of aging - from marginal messing with the disease state to actually addressing the root causes. Here is a phase 3 that succeeded and failed. Clearly it works. But the study design is such that they had an unexpected result: https://www.gensight-biologics.com/2018/04/03/gensight-biologics-announces-topline-results-from-reverse-phase-iii-clinical-trial-of-gs010-in-patients-with-leber-hereditary-optic-neuropathy-lhon/ https://www.gensight-biologics.com/2018/04/03/gensight-biolo... They show a good effect in the treated eye, and their gene therapy also produces benefits in the untreated eye. Since they don't know how that could happen, no approval.

derefr 8y ago
> If you are actually addressing a useful mechanism, the effect won't be marginal. If there are 10,000 different mechanisms that each only affect 0.01% of the population, because the actual "problem" is just gradual DNA damage over time leading to one of 10,000 different problems... then what else is there to do but to come up with 10,000 solutions? (I mean, you could figure out how to entirely replace DNA with something that isn't affected by radiation, or has cryptographically-strong checksums, or something. But in terms of realistic solutions...?)
- reasonattlm 8y ago
  In the case of infectious disease, clearly your scenario isn't what happens in the real world. Otherwise no disease could be cured. In the case of aging, everyone ages for the same reasons. A few types of underlying molecular damage that are comparatively straightforward to investigate and address, when compared to the enormous complexity of metabolism. Look at senolytic drugs: remove senescent cells, life span increases, aspects of aging are reversed in old individuals. Aging is like rust in a complex metal structure; the failure modes appear varied and complex because the structure is complex. But rust isn't complicated. Aging is the same story. Genetic variation is not particularly meaningful for the vast majority of age-related disease. It isn't important for most of the life span. It only becomes even somewhat influential in very late life where there is variation in resistance to damage and consequences of damage. But why care thing one about that when the right strategy is to repair the underlying damage in order to ensure that people either never enter or are removed from the situation in which genetics start to matter a little?
  - Fomite 8y ago
    "In the case of infectious disease, clearly your scenario isn't what happens in the real world. Otherwise no disease could be cured." Note that your statement is only true for a relatively narrow range of infectious diseases, and mostly in special cases where we have "Nuke the site from orbit" class drugs that don't target things eukaryotic cells use.
aaavl2821 8y ago
You can't have an informed risk assessment without clinical trials because you don't have the data to inform assumptions about risk / benefit. The models currently used to predict risk / benefit are really poor predictors of whether something is safe and effective. If you don't do phase 3 RCTs you'll just end up w a bunch of people gaming the risk assessments and getting drugs that don't work approved Statistics and study powering are designed to address the issue you cite about marginal benefits "smoothing out" as the population gets bigger. That's why you have to do big phase 3 studies with __predefined__ endpoints and statistical analysis plans powered by estimated effect sizes Saying that phase 3 study "clearly worked" is not true. It didn't do better than sham. Yes it could be true that there is some drug related benefit in the nontreated eye, but another interpretation is the sham injection itself could cause the observed improvement. Or the study design was wonky, endpoints inconsistently measured, etc
jcranmer 8y ago
Patient choice is an idea that sounds good in theory but just has no good way of actually working in practice. Most people are going to be uninformed about medical science. (Hell, most medical practitioners struggle to stay informed). Worse, there is a strong bias towards misinformation--look at the health supplement market, which is basically a market the seedy snake oil sellers set up to get around FDA regulation. On top of this, keep in mind that it is very easy to accidentally design bad experiment methodology. With respect to clinical trials, it's all too easy to take a failed trial and say "it actually worked on this subgroup" because there are often enough subgroups to find one with a spurious correlation. And when you have monetary pressure to salvage a failed product, the p-hacking is less likely to be internally questioned. The current regulatory regime requires that you declare how you're going to define acceptance before you run the trial to prevent this kind of p-hacking, and if you truly believe that you're not p-hacking, you're welcome to submit another trial to show correlation on that subgroup (unsurprisingly, many of these end up coming negative).
- teslabox 8y ago
  > Worse, there is a strong bias towards misinformation--look at the health supplement market, which is basically a market the seedy snake oil sellers set up to get around FDA regulation. The pure food and drug act was passed to deal with the snake oil industry, which was in the process of reinventing and legitimizing itself into the pharmaceutical industry. The supplement industry was later exempted from the requirements for patent medicine producers because there is no way to monopolize the profits of many treatments based on supplements. Some companies have patented certain aspects of their supplement's production or delivery mechanisms, but (I believe) the natural biological molecule itself can't be patented. https://en.wikipedia.org/wiki/Pure_Food_and_Drug_Act https://en.wikipedia.org/wiki/Pure_Food_and_Drug_Act (1908) https://en.wikipedia.org/wiki/Federal_Food%2C_Drug%2C_and_Cosmetic_Act https://en.wikipedia.org/wiki/Federal_Food%2C_Drug%2C_and_Co... (1938)
digitalzombie 8y ago
> Here is a phase 3 that succeeded and failed. Clearly it works. That's not clearly works. Quantifying works/succeed with statistic is how you know it clearly works because it's statistically significant. Other wise your clearly work statement is just your opinion and the opinion of a drug company trying to make money. If it fail and succeed in phase 3 then it's not clear at all if it works.

derefr 8y ago

> If you are actually addressing a useful mechanism, the effect won't be marginal. If there are 10,000 different mechanisms that each only affect 0.01% of the population, because the actual "problem" is just gradual DNA damage over time leading to one of 10,000 different problems... then what else is there to do but to come up with 10,000 solutions? (I mean, you could figure out how to entirely replace DNA with something that isn't affected by radiation, or has cryptographically-strong checksums, or something. But in terms of realistic solutions...?)

reasonattlm 8y ago
In the case of infectious disease, clearly your scenario isn't what happens in the real world. Otherwise no disease could be cured. In the case of aging, everyone ages for the same reasons. A few types of underlying molecular damage that are comparatively straightforward to investigate and address, when compared to the enormous complexity of metabolism. Look at senolytic drugs: remove senescent cells, life span increases, aspects of aging are reversed in old individuals. Aging is like rust in a complex metal structure; the failure modes appear varied and complex because the structure is complex. But rust isn't complicated. Aging is the same story. Genetic variation is not particularly meaningful for the vast majority of age-related disease. It isn't important for most of the life span. It only becomes even somewhat influential in very late life where there is variation in resistance to damage and consequences of damage. But why care thing one about that when the right strategy is to repair the underlying damage in order to ensure that people either never enter or are removed from the situation in which genetics start to matter a little?
- Fomite 8y ago
  "In the case of infectious disease, clearly your scenario isn't what happens in the real world. Otherwise no disease could be cured." Note that your statement is only true for a relatively narrow range of infectious diseases, and mostly in special cases where we have "Nuke the site from orbit" class drugs that don't target things eukaryotic cells use.

aaavl2821 8y ago

You can't have an informed risk assessment without clinical trials because you don't have the data to inform assumptions about risk / benefit. The models currently used to predict risk / benefit are really poor predictors of whether something is safe and effective. If you don't do phase 3 RCTs you'll just end up w a bunch of people gaming the risk assessments and getting drugs that don't work approved Statistics and study powering are designed to address the issue you cite about marginal benefits "smoothing out" as the population gets bigger. That's why you have to do big phase 3 studies with __predefined__ endpoints and statistical analysis plans powered by estimated effect sizes Saying that phase 3 study "clearly worked" is not true. It didn't do better than sham. Yes it could be true that there is some drug related benefit in the nontreated eye, but another interpretation is the sham injection itself could cause the observed improvement. Or the study design was wonky, endpoints inconsistently measured, etc

jcranmer 8y ago

Patient choice is an idea that sounds good in theory but just has no good way of actually working in practice. Most people are going to be uninformed about medical science. (Hell, most medical practitioners struggle to stay informed). Worse, there is a strong bias towards misinformation--look at the health supplement market, which is basically a market the seedy snake oil sellers set up to get around FDA regulation. On top of this, keep in mind that it is very easy to accidentally design bad experiment methodology. With respect to clinical trials, it's all too easy to take a failed trial and say "it actually worked on this subgroup" because there are often enough subgroups to find one with a spurious correlation. And when you have monetary pressure to salvage a failed product, the p-hacking is less likely to be internally questioned. The current regulatory regime requires that you declare how you're going to define acceptance before you run the trial to prevent this kind of p-hacking, and if you truly believe that you're not p-hacking, you're welcome to submit another trial to show correlation on that subgroup (unsurprisingly, many of these end up coming negative).

teslabox 8y ago
> Worse, there is a strong bias towards misinformation--look at the health supplement market, which is basically a market the seedy snake oil sellers set up to get around FDA regulation. The pure food and drug act was passed to deal with the snake oil industry, which was in the process of reinventing and legitimizing itself into the pharmaceutical industry. The supplement industry was later exempted from the requirements for patent medicine producers because there is no way to monopolize the profits of many treatments based on supplements. Some companies have patented certain aspects of their supplement's production or delivery mechanisms, but (I believe) the natural biological molecule itself can't be patented. https://en.wikipedia.org/wiki/Pure_Food_and_Drug_Act https://en.wikipedia.org/wiki/Pure_Food_and_Drug_Act (1908) https://en.wikipedia.org/wiki/Federal_Food%2C_Drug%2C_and_Cosmetic_Act https://en.wikipedia.org/wiki/Federal_Food%2C_Drug%2C_and_Co... (1938)

digitalzombie 8y ago

> Here is a phase 3 that succeeded and failed. Clearly it works. That's not clearly works. Quantifying works/succeed with statistic is how you know it clearly works because it's statistically significant. Other wise your clearly work statement is just your opinion and the opinion of a drug company trying to make money. If it fail and succeed in phase 3 then it's not clear at all if it works.