6 ms·

Scott Alexander's "Adumbrations Of Aducanumab" is essential reading on this subject IMO [0]. He makes an interesting point that when the FDA approves a drug, i

Scott Alexander's "Adumbrations Of Aducanumab" is essential reading on this subject IMO [0]. He makes an interesting point that when the FDA approves a drug, it means that Medicare is automatically on the hook to pay for it, and that a more sensible system might decouple approval and mandated reimbursement by having multiple levels of approval (e.g. 1 star = this is probably safe, 2 stars = it's definitely safe, 3 = there's weak/controversial evidence for efficacy, 4 stars = it is safe and it works, etc.). But it's interesting to me that we hear about FDA scientists resigning when a panel is too permissive, but not when it's too restrictive. Can we chalk this up to something about the kind of people the FDA recruits? I can recall just one instance of an FDA official pushing the institution to go faster, and that was when the doctor's wife died of cancer [1]. [0] https://astralcodexten.substack.com/p/adumbrations-of-aducanumab https://astralcodexten.substack.com/p/adumbrations-of-aducan... [1] https://www.nytimes.com/2016/01/03/us/politics/fda-regulator-widowed-by-cancer-helps-speed-drug-approval.html https://www.nytimes.com/2016/01/03/us/politics/fda-regulator...

Brakenshire 5y ago
We do have to make drug discovery vastly more efficient, but we want to make it more efficient at making useful drugs available, not just making drugs available. If you decouple availability from effectiveness then drug companies will just market unproven drugs to people who are desperate or who don't know any better. Or even if the drug companies do nothing people will just start taking drugs on the basis of recommendations from Facebook. Maybe that means building randomized trials into treatment in a much more regular way (like with the Covid RECOVERY trials, where any doctor could enroll patients with a web form, and source the candidate treatment from the local pharmacy), which vastly reduces cost for treatments already on the market with other indications. And perhaps altering the balance of risk for patients with life-threatening or terminal diseases.
- civilized 5y ago
  One reason your proposal isn't happening today might be because we don't want tons of doctors choosing to enroll in these "field" randomized trials, if they might be experimentally treating people with drugs that aren't safe. We want to know about safety first before unleashing on the world, even in a randomized trial context. And since the FDA bundles verification of safety and efficacy together, so we can't do your proposal without unbundling them. Now, imagine that after safety is verified to a 1 or 2-star level, we don't make the drug approved for general use, but doctors can freely enroll in randomized trials as you proposed? That way we're getting the efficacy data quickly but safely. Maybe a minor point, but we do have to unbundle safety from efficacy in some sense to get off the ground here. The question is, what exactly is allowed to happen after safety is established but efficacy is not yet established?
  - Brakenshire 5y ago
    I agree, that's well put and I think that could work very well. I think the key would be having clear centralized standards for how the trials are conducted, or you could have doctors skirting the system and just using it as de-facto prescription, and patients being put at risk for ineffective treatment without producing any valid evidence.
    - civilized 5y ago
      Yeah, if the existing system is built on trust of doctors who run clinical trials, we have to ask whether that trust scales. If the system were set up so that doctors would have a lot to lose if they didn't follow the system, trust could work here. We already trust doctors as professionals on many other things...
  - ezconnect 5y ago
    His proposal is to treat a new disease with approved drug you can buy and not experimental drugs. This will give us more data on therapies that might work and we can do more detailed studies on therapies with high percentage of recoveries.
- Sanguinaire 5y ago
  The problem here is letting drug companies market to the uneducated general public - there is a good reason why that practice is banned in most countries.
  - irrational 5y ago
    Is it banned when it isn’t the drug companies doing the marketing to the uneducated general public? It wasn’t the drug companies telling uneducated people to use horse dewormer to combat Covid. That seems just as dangerous (probably more so) as drug companies making claims to the general public.
  - wnissen 5y ago
    Banned in every country, except the U.S. and New Zealand.
- themacguffinman 5y ago
  > If you decouple availability from effectiveness then drug companies will just market unproven drugs to people who are desperate or who don't know any better. Or even if the drug companies do nothing people will just start taking drugs on the basis of recommendations from Facebook. But this already happens for certain treatments. The FDA already regulates a separate category with loose efficacy requirements for dietary supplements and herbal remedies. These medically irrelevant treatments are immensely popular, they're just forbidden from marketing specific medical claims and being actually toxic. People still end up taking FDA approved treatments like Ivermectin for non-FDA-approved purposes based on Facebook recommendations. Why do we always have to make people's choices for them? They're grown adults capable of making their own decisions and their own mistakes. If they want to ignore the science and the credibility of the FDA process, they should be free to do so.
  - Brakenshire 5y ago
    One big reason is you're removing the economic incentive to produce good evidence. The herbal remedies industry is a perfect example, there will be valid treatments there but yet the industry is almost entirely medically useless. Some of the treatments might be effective for particular diseases, at particular doses but no one ever knows. These companies have billions in revenue and are vastly profitable but they never look for evidence that treatments work because people buy the remedies anyway, and a company that spent the money to run the trials would be undercut by less scrupulous competition. This is not even a market with asymmetry of knowledge, it's a market where no one knows anything, and the lack of regulation makes sure it continues that way.
    - themacguffinman 5y ago
      There's still an economic incentive for the highest level of FDA approval: Medicare and insurance companies have to cover it and doctors can prescribe it. That ensures a high and consistent level of usage across the whole country, not just among people with more money than sense (usually IMHO; it's possible that they're just smarter & savvier than the notoriously conservative FDA, who knows, this is a big reason why taking away choice is bad on principle). I don't think the lack of credible evidence of efficacy in the supplement market is for lack of trying. If an alternative medicine is actually effective, it just becomes medicine that the majority of people can trust and use.
cameronh90 5y ago
In the UK, as I understand it, we have: MHRA: approval for marketing and sale within the UK NICE: analyse cost/benefit ratio and safety profile for use within NHS (which is then approved by the NHS CCGs)
jeherr 5y ago
In regards to the panel being too restrictive or permissive, I think you have to look at the possible motivations for why. If you have a drug candidate that seems safe and effective, what possible motivation could the panel have for rejecting it? The pharma company gets their drug to market, patients get new and better drugs. The incentives are aligned so the issue with being too restrictive is a debate about how risky we should be approving new drugs. On the other hand for a drug that’s safe but maybe not effective, there’s a lot of money to be made off of people willing to do anything to prolong their time with their loved ones who have been diagnosed with Alzheimer’s. Aside from that, the FDA can hurt it’s reputation by approving drugs that just don’t work too frequently and be accused of being in the pockets of the pharma companies. There’s a stronger possibility of corruption affecting the decision, and whether that corruption is real or perceived doesn’t matter much for public trust.
usefulcat 5y ago
Perhaps we need a third level of FDA approval. In addition to 'approved' and 'not approved', we could have 'safe but not demonstrated to be effective'. Drugs in the third category would be legal to prescribe but Medicare and insurance companies would not be required to pay for them. Although in practice I guess this would probably not be very different from 'not approved', in which case it seems right for the FDA to reject ineffective drugs. In any case, one thing we definitely do not want is to incentivize the creation of ineffective drugs, especially expensive ones.
- dougmwne 5y ago
  What on earth would be the point of that? Why should we encourage the development of new snake oil? Why encourage drug companies, which currently must provide evidence their drugs work, to pivot into snake oil? Do you think we are an enlightened people immune from the snake oil our ancestors used to fall for? Recent pandemic events would beg to differ!
  - s1artibartfast 5y ago
    I think the point would be "safe but efficacy TBD" and be coupled with extensive monitoring until efficacy is either confirmed or denied. I think this is the general goal of the approval, but our reimbursement system is not set up to handle it because it doesn't take the level of benefit into account. Federal reimbursement is set at what the company wants, even if the product provides better or worse treatment than the alternative.
    - dougmwne 5y ago
      "Safe but Efficacy TBD" is a phase 3 clinical trial. And measuring out the level of benefit vs. the cost to medicare is a totally different issue, I think. At that point, we know it works, and it deserves an approval. I think that's where these laws around publishing prices come in. No individual is going to be able to interpret them, and not even doctors were aware of what their treatments cost. But having the prices published gives advocacy organizations a chance to run the numbers of effect size vs treatment cost and point out areas where the cost is extortionate. It could also be pretty awesome if there was a labeling requirement on all FDA approved drugs for effect size, from a red "not very effective" to a yellow "moderately effective" to green "highly effective".
AshamedCaptain 5y ago
If there's weak evidence for the effectiveness of a drug, then most likely the drug has no effect at all. Let's remember that crap such as homeopathy would rank as 3 stars on your proposed system. Our current research/science publishing system is not well suited to evaluating very small effects. Not that there may any system well suited to that.
- dougmwne 5y ago
  Precisely. If I want to create weak evidence for a drug, I can just run a few dozen studies and statistics will take care of the rest. Drugs with real, but very small effects are also not very useful as they aren't going to move the needle much, and certainly not as much as the quite powerful placebo effect.
mchusma 5y ago
Here here. People at the FDA should be resigning over them banning the COVID vaccine for kids under 12, for example. But we don't see behavior like that. It's just a sign that the FDA is so horribly broken, it probably can't be recovered. One note that I think would be important is that we should stop using the term "approved" or "not approved" and instead used "banned" or "not banned", as this is a more clear way to describe the situation. For example, in Sep 2020, it was completely obvious that taking a covid vaccine if you were elderly was dramatically better than waiting for COVID. But you could not, because the government said no. It didn't matter how informed you were, how much your doctor would agree, the answer was no. You go to jail if you try to take it. That should be called "banned".
danzk 5y ago
Government programs need to be able to negotiate drug prices to get a fair deal before subsidising them. Whenever a trade deal comes up between the US and Australia, they always want to kill our PBS system that negotiates lower drug prices with pharmaceutical companies.
s1artibartfast 5y ago
Great read, thanks!
refurb 5y ago
The FDA does have that option and is doing it for this drug. It’s called a “National Coverage Determination” https://www.cms.gov/newsroom/press-releases/cms-opens-national-coverage-determination-analysis-treatment-alzheimers-disease https://www.cms.gov/newsroom/press-releases/cms-opens-nation... "Today, the Centers for Medicare & Medicaid Services (CMS) is opening a National Coverage Determination (NCD) analysis, a process that will allow the agency to carefully review and determine whether Medicare will establish a national Medicare coverage policy for monoclonal antibodies targeting amyloid for the treatment of Alzheimer’s disease."

Brakenshire 5y ago

We do have to make drug discovery vastly more efficient, but we want to make it more efficient at making useful drugs available, not just making drugs available. If you decouple availability from effectiveness then drug companies will just market unproven drugs to people who are desperate or who don't know any better. Or even if the drug companies do nothing people will just start taking drugs on the basis of recommendations from Facebook. Maybe that means building randomized trials into treatment in a much more regular way (like with the Covid RECOVERY trials, where any doctor could enroll patients with a web form, and source the candidate treatment from the local pharmacy), which vastly reduces cost for treatments already on the market with other indications. And perhaps altering the balance of risk for patients with life-threatening or terminal diseases.

civilized 5y ago
One reason your proposal isn't happening today might be because we don't want tons of doctors choosing to enroll in these "field" randomized trials, if they might be experimentally treating people with drugs that aren't safe. We want to know about safety first before unleashing on the world, even in a randomized trial context. And since the FDA bundles verification of safety and efficacy together, so we can't do your proposal without unbundling them. Now, imagine that after safety is verified to a 1 or 2-star level, we don't make the drug approved for general use, but doctors can freely enroll in randomized trials as you proposed? That way we're getting the efficacy data quickly but safely. Maybe a minor point, but we do have to unbundle safety from efficacy in some sense to get off the ground here. The question is, what exactly is allowed to happen after safety is established but efficacy is not yet established?
- Brakenshire 5y ago
  I agree, that's well put and I think that could work very well. I think the key would be having clear centralized standards for how the trials are conducted, or you could have doctors skirting the system and just using it as de-facto prescription, and patients being put at risk for ineffective treatment without producing any valid evidence.
  - civilized 5y ago
    Yeah, if the existing system is built on trust of doctors who run clinical trials, we have to ask whether that trust scales. If the system were set up so that doctors would have a lot to lose if they didn't follow the system, trust could work here. We already trust doctors as professionals on many other things...
- ezconnect 5y ago
  His proposal is to treat a new disease with approved drug you can buy and not experimental drugs. This will give us more data on therapies that might work and we can do more detailed studies on therapies with high percentage of recoveries.
Sanguinaire 5y ago
The problem here is letting drug companies market to the uneducated general public - there is a good reason why that practice is banned in most countries.
- irrational 5y ago
  Is it banned when it isn’t the drug companies doing the marketing to the uneducated general public? It wasn’t the drug companies telling uneducated people to use horse dewormer to combat Covid. That seems just as dangerous (probably more so) as drug companies making claims to the general public.
- wnissen 5y ago
  Banned in every country, except the U.S. and New Zealand.
themacguffinman 5y ago
> If you decouple availability from effectiveness then drug companies will just market unproven drugs to people who are desperate or who don't know any better. Or even if the drug companies do nothing people will just start taking drugs on the basis of recommendations from Facebook. But this already happens for certain treatments. The FDA already regulates a separate category with loose efficacy requirements for dietary supplements and herbal remedies. These medically irrelevant treatments are immensely popular, they're just forbidden from marketing specific medical claims and being actually toxic. People still end up taking FDA approved treatments like Ivermectin for non-FDA-approved purposes based on Facebook recommendations. Why do we always have to make people's choices for them? They're grown adults capable of making their own decisions and their own mistakes. If they want to ignore the science and the credibility of the FDA process, they should be free to do so.
- Brakenshire 5y ago
  One big reason is you're removing the economic incentive to produce good evidence. The herbal remedies industry is a perfect example, there will be valid treatments there but yet the industry is almost entirely medically useless. Some of the treatments might be effective for particular diseases, at particular doses but no one ever knows. These companies have billions in revenue and are vastly profitable but they never look for evidence that treatments work because people buy the remedies anyway, and a company that spent the money to run the trials would be undercut by less scrupulous competition. This is not even a market with asymmetry of knowledge, it's a market where no one knows anything, and the lack of regulation makes sure it continues that way.
  - themacguffinman 5y ago
    There's still an economic incentive for the highest level of FDA approval: Medicare and insurance companies have to cover it and doctors can prescribe it. That ensures a high and consistent level of usage across the whole country, not just among people with more money than sense (usually IMHO; it's possible that they're just smarter & savvier than the notoriously conservative FDA, who knows, this is a big reason why taking away choice is bad on principle). I don't think the lack of credible evidence of efficacy in the supplement market is for lack of trying. If an alternative medicine is actually effective, it just becomes medicine that the majority of people can trust and use.

cameronh90 5y ago

In the UK, as I understand it, we have: MHRA: approval for marketing and sale within the UK NICE: analyse cost/benefit ratio and safety profile for use within NHS (which is then approved by the NHS CCGs)

jeherr 5y ago

In regards to the panel being too restrictive or permissive, I think you have to look at the possible motivations for why. If you have a drug candidate that seems safe and effective, what possible motivation could the panel have for rejecting it? The pharma company gets their drug to market, patients get new and better drugs. The incentives are aligned so the issue with being too restrictive is a debate about how risky we should be approving new drugs. On the other hand for a drug that’s safe but maybe not effective, there’s a lot of money to be made off of people willing to do anything to prolong their time with their loved ones who have been diagnosed with Alzheimer’s. Aside from that, the FDA can hurt it’s reputation by approving drugs that just don’t work too frequently and be accused of being in the pockets of the pharma companies. There’s a stronger possibility of corruption affecting the decision, and whether that corruption is real or perceived doesn’t matter much for public trust.

usefulcat 5y ago

Perhaps we need a third level of FDA approval. In addition to 'approved' and 'not approved', we could have 'safe but not demonstrated to be effective'. Drugs in the third category would be legal to prescribe but Medicare and insurance companies would not be required to pay for them. Although in practice I guess this would probably not be very different from 'not approved', in which case it seems right for the FDA to reject ineffective drugs. In any case, one thing we definitely do not want is to incentivize the creation of ineffective drugs, especially expensive ones.

dougmwne 5y ago
What on earth would be the point of that? Why should we encourage the development of new snake oil? Why encourage drug companies, which currently must provide evidence their drugs work, to pivot into snake oil? Do you think we are an enlightened people immune from the snake oil our ancestors used to fall for? Recent pandemic events would beg to differ!
- s1artibartfast 5y ago
  I think the point would be "safe but efficacy TBD" and be coupled with extensive monitoring until efficacy is either confirmed or denied. I think this is the general goal of the approval, but our reimbursement system is not set up to handle it because it doesn't take the level of benefit into account. Federal reimbursement is set at what the company wants, even if the product provides better or worse treatment than the alternative.
  - dougmwne 5y ago
    "Safe but Efficacy TBD" is a phase 3 clinical trial. And measuring out the level of benefit vs. the cost to medicare is a totally different issue, I think. At that point, we know it works, and it deserves an approval. I think that's where these laws around publishing prices come in. No individual is going to be able to interpret them, and not even doctors were aware of what their treatments cost. But having the prices published gives advocacy organizations a chance to run the numbers of effect size vs treatment cost and point out areas where the cost is extortionate. It could also be pretty awesome if there was a labeling requirement on all FDA approved drugs for effect size, from a red "not very effective" to a yellow "moderately effective" to green "highly effective".

AshamedCaptain 5y ago

If there's weak evidence for the effectiveness of a drug, then most likely the drug has no effect at all. Let's remember that crap such as homeopathy would rank as 3 stars on your proposed system. Our current research/science publishing system is not well suited to evaluating very small effects. Not that there may any system well suited to that.

dougmwne 5y ago
Precisely. If I want to create weak evidence for a drug, I can just run a few dozen studies and statistics will take care of the rest. Drugs with real, but very small effects are also not very useful as they aren't going to move the needle much, and certainly not as much as the quite powerful placebo effect.

mchusma 5y ago

Here here. People at the FDA should be resigning over them banning the COVID vaccine for kids under 12, for example. But we don't see behavior like that. It's just a sign that the FDA is so horribly broken, it probably can't be recovered. One note that I think would be important is that we should stop using the term "approved" or "not approved" and instead used "banned" or "not banned", as this is a more clear way to describe the situation. For example, in Sep 2020, it was completely obvious that taking a covid vaccine if you were elderly was dramatically better than waiting for COVID. But you could not, because the government said no. It didn't matter how informed you were, how much your doctor would agree, the answer was no. You go to jail if you try to take it. That should be called "banned".

danzk 5y ago

Government programs need to be able to negotiate drug prices to get a fair deal before subsidising them. Whenever a trade deal comes up between the US and Australia, they always want to kill our PBS system that negotiates lower drug prices with pharmaceutical companies.

s1artibartfast 5y ago

Great read, thanks!

refurb 5y ago

The FDA does have that option and is doing it for this drug. It’s called a “National Coverage Determination” https://www.cms.gov/newsroom/press-releases/cms-opens-national-coverage-determination-analysis-treatment-alzheimers-disease https://www.cms.gov/newsroom/press-releases/cms-opens-nation... "Today, the Centers for Medicare & Medicaid Services (CMS) is opening a National Coverage Determination (NCD) analysis, a process that will allow the agency to carefully review and determine whether Medicare will establish a national Medicare coverage policy for monoclonal antibodies targeting amyloid for the treatment of Alzheimer’s disease."