4 ms·

It's hard to analyse the data from treatments, as treatments don't have a control group. Trials are designed with control groups, placebos and double-blind pro

It's hard to analyse the data from treatments, as treatments don't have a control group. Trials are designed with control groups, placebos and double-blind process to gather data and then analyse it. We may have loads of data from real people being treated, but good luck trying to analyse it in a scientific way. We need effective vaccines, and we need to be able to trust them. Just treating people with a vaccine doesn't always give that trust and confidence. (I hope I got my medical terminology right, I'm not a clinician, but have been close to covid-19 related medical research)

fabian2k 5y ago
It's much, much harder to understand the data if you don't have a control group. But once you have an effective treatment, you no longer have a control group if you're at all ethical. You cannot keep a working treatment from people just to do more science, that is deeply and fundamentally unethical. So the moment we knew the vaccines worked, the science got harder. Well, that point also save an enormous amount of lives, so I don't think this is something to complain about. It's still possible to do a lot of good science now, even randomized controlled trials. They just can't compare to unvaccinated groups, but you can still do stuff like compare double-vaccinated with triple vaccinated.
- incrudible 5y ago
  A lot of data now shows that double vaccinated are infected with Omikron more often than unvaccinated. I do not know what causes this or what it means, but imagine for a moment there were no unvaccinated left: Your vaccine could be arbitrarily harmful with a new variant and you would not be able to tell.
  - fabian2k 5y ago
    That is simply wrong. There is one study that listed negative effectiveness against Omicron for the vaccines. That study did not control for any confounding factors, it simply was not designed to measure this particular thing, it was focused on a different question. There are some strong fundamental reasons why we would not expect the vaccine to be harmful with future variants. And even then we would still be able to detect if it was.
    - incrudible 5y ago
      It is not one study, national data from Denmark, UK, Germany and Canada shows that double vaccinated are infected at a higher rate. I am not jumping to conclusions that this means efficacy is negative, because that would require a controlled trial with an unvaccinated group. Do you catch my drift?
      - fabian2k 5y ago
        Please show the data that claims that double vaccinated are more likely to be infected than unvaccinated people by Omicron. And raw infection counts are not an answer here as the vaccinated and unvaccinated people are clearly not matched populations you can compare without adjustments.
        incrudible 5y ago
        > And raw infection counts are not an answer here as the vaccinated and unvaccinated people are clearly not matched populations you can compare... That's the point! That's why we need controlled trials. With an unvaccinated group. You know, the thing that you say we can't have, because it's unethical. > ...without adjustments. As usual, there's an XKCD for that: https://xkcd.com/2560/ https://xkcd.com/2560/
      - bigfudge 5y ago
        There’s no way of running a controlled trial of this sort when 80% of the population have already been vaccinated. You can design better or larger observational studies though.
- lurquer 5y ago
  > But once you have an effective treatment, you no longer have a control group if you're at all ethical. You cannot keep a working treatment from people just to do more science, that is deeply and fundamentally unethical. You could easily find millions of potential study participants who don’t want the vaccine. The study wouldn’t be double-blind, but it would be single-blind and better than what they chose to do.
  - mandmandam 5y ago
    It's astounding how rare this rather obvious course of action has been in the global conversation.
- breck 5y ago
  > you no longer have a control group if you're at all ethical. > You cannot keep a working treatment from people just to do more science, that is deeply and fundamentally unethical. I see this position put out there a lot and generally go unchallenged. For the record some people think the opposite: that it was unethical to unblind the placebo group so early. At the time you could estimate that continuing the placebo group might lead to ~60 unnecessary deaths from Covid-19 and a few hundred more serious cases. That is in the lucky scenario where the early success held. But in all scenarios, in return for a few dozen volunteers risking their lives, billions of people would benefit from randomized, placebo-controlled trial data which the NIH itself calls the "gold standard" as to "whether or not a treatment is safe and effective". IMO the harms we have gone through from flying in the dark (people underhyping/overhyping the vaccines), multiplied by the number of people involved, make this a case where unblinding was unethical. It is very easy to imagine that far more life was lost in the general population from being in the dark than was saved by unblinding the volunteer study population. (There's also the issue of whether or not it would have been practical to keep the participants blinded. I think that's a challenging topic of its own orthogonal to the ethical question.)
  - deleted 5y ago
    [deleted]
mike_d 5y ago
Depending on the phase of clinical trials, they don't need to be double blind or controlled. You can absolutely analyze real world mass vaccination data in a scientific way. Just treat pre-vaccination and post-vaccination as two different patients. The numbers are so large that other factors (environmental, genetics, etc) cancel out. So if your post-vaccination population has a rate of heart attacks 80% lower than the established norm, you know that is worth further investigation. (Full disclosure: my family used to own a company that ran clinical trials for pharmaceutical companies)
- timr 5y ago
  > You can absolutely analyze real world mass vaccination data in a scientific way. Just treat pre-vaccination and post-vaccination as two different patients. Yes, you can do this. No, it's not the same. With this kind of uncontrolled, longitudinal study, yes, some things are quasi-controlled: genetics, possibly lifestyle, other drugs, etc. Anything that plausibly doesn't change (much) in a single person from time A to time B. Some things you can't control for, but still matter: changes in behavior due to the event itself. Placebo effect. Sample bias (e.g. the patients receiving treatment X were selected to receive treatment X because it was felt that they would benefit from it. This is subtle, but can really mislead. In the example of vaccination, imagine that the people most likely to vaccinate their young kids are also the ones most likely to keep their kids isolated at home, in a protective bubble...) > So if your post-vaccination population has a rate of heart attacks 80% lower than the established norm, you know that is worth further investigation. The key part of that is the last three words: worth further investigation. To get the final answer, you still need a controlled experiment.
- SomewhatLikely 5y ago
  Different prevalences of variants over time and different rates of infection over time would complicate such a time-based control.
a45a33s 5y ago
the covid vaccine trials had no control groups either
- rsfern 5y ago
  They were all doubly blinded placebo controlled trials… Are you talking about how they unblinded the study after efficacy results started coming in?
  - a45a33s 5y ago
    yes i am talking about how they got rid of the control groups once it appeared to be working

fabian2k 5y ago

It's much, much harder to understand the data if you don't have a control group. But once you have an effective treatment, you no longer have a control group if you're at all ethical. You cannot keep a working treatment from people just to do more science, that is deeply and fundamentally unethical. So the moment we knew the vaccines worked, the science got harder. Well, that point also save an enormous amount of lives, so I don't think this is something to complain about. It's still possible to do a lot of good science now, even randomized controlled trials. They just can't compare to unvaccinated groups, but you can still do stuff like compare double-vaccinated with triple vaccinated.

incrudible 5y ago
A lot of data now shows that double vaccinated are infected with Omikron more often than unvaccinated. I do not know what causes this or what it means, but imagine for a moment there were no unvaccinated left: Your vaccine could be arbitrarily harmful with a new variant and you would not be able to tell.
- fabian2k 5y ago
  That is simply wrong. There is one study that listed negative effectiveness against Omicron for the vaccines. That study did not control for any confounding factors, it simply was not designed to measure this particular thing, it was focused on a different question. There are some strong fundamental reasons why we would not expect the vaccine to be harmful with future variants. And even then we would still be able to detect if it was.
  - incrudible 5y ago
    It is not one study, national data from Denmark, UK, Germany and Canada shows that double vaccinated are infected at a higher rate. I am not jumping to conclusions that this means efficacy is negative, because that would require a controlled trial with an unvaccinated group. Do you catch my drift?
    - fabian2k 5y ago
      Please show the data that claims that double vaccinated are more likely to be infected than unvaccinated people by Omicron. And raw infection counts are not an answer here as the vaccinated and unvaccinated people are clearly not matched populations you can compare without adjustments.
      - incrudible 5y ago
        > And raw infection counts are not an answer here as the vaccinated and unvaccinated people are clearly not matched populations you can compare... That's the point! That's why we need controlled trials. With an unvaccinated group. You know, the thing that you say we can't have, because it's unethical. > ...without adjustments. As usual, there's an XKCD for that: https://xkcd.com/2560/ https://xkcd.com/2560/
    - bigfudge 5y ago
      There’s no way of running a controlled trial of this sort when 80% of the population have already been vaccinated. You can design better or larger observational studies though.
lurquer 5y ago
> But once you have an effective treatment, you no longer have a control group if you're at all ethical. You cannot keep a working treatment from people just to do more science, that is deeply and fundamentally unethical. You could easily find millions of potential study participants who don’t want the vaccine. The study wouldn’t be double-blind, but it would be single-blind and better than what they chose to do.
- mandmandam 5y ago
  It's astounding how rare this rather obvious course of action has been in the global conversation.
breck 5y ago
> you no longer have a control group if you're at all ethical. > You cannot keep a working treatment from people just to do more science, that is deeply and fundamentally unethical. I see this position put out there a lot and generally go unchallenged. For the record some people think the opposite: that it was unethical to unblind the placebo group so early. At the time you could estimate that continuing the placebo group might lead to ~60 unnecessary deaths from Covid-19 and a few hundred more serious cases. That is in the lucky scenario where the early success held. But in all scenarios, in return for a few dozen volunteers risking their lives, billions of people would benefit from randomized, placebo-controlled trial data which the NIH itself calls the "gold standard" as to "whether or not a treatment is safe and effective". IMO the harms we have gone through from flying in the dark (people underhyping/overhyping the vaccines), multiplied by the number of people involved, make this a case where unblinding was unethical. It is very easy to imagine that far more life was lost in the general population from being in the dark than was saved by unblinding the volunteer study population. (There's also the issue of whether or not it would have been practical to keep the participants blinded. I think that's a challenging topic of its own orthogonal to the ethical question.)
- deleted 5y ago
  [deleted]

mike_d 5y ago

Depending on the phase of clinical trials, they don't need to be double blind or controlled. You can absolutely analyze real world mass vaccination data in a scientific way. Just treat pre-vaccination and post-vaccination as two different patients. The numbers are so large that other factors (environmental, genetics, etc) cancel out. So if your post-vaccination population has a rate of heart attacks 80% lower than the established norm, you know that is worth further investigation. (Full disclosure: my family used to own a company that ran clinical trials for pharmaceutical companies)

timr 5y ago
> You can absolutely analyze real world mass vaccination data in a scientific way. Just treat pre-vaccination and post-vaccination as two different patients. Yes, you can do this. No, it's not the same. With this kind of uncontrolled, longitudinal study, yes, some things are quasi-controlled: genetics, possibly lifestyle, other drugs, etc. Anything that plausibly doesn't change (much) in a single person from time A to time B. Some things you can't control for, but still matter: changes in behavior due to the event itself. Placebo effect. Sample bias (e.g. the patients receiving treatment X were selected to receive treatment X because it was felt that they would benefit from it. This is subtle, but can really mislead. In the example of vaccination, imagine that the people most likely to vaccinate their young kids are also the ones most likely to keep their kids isolated at home, in a protective bubble...) > So if your post-vaccination population has a rate of heart attacks 80% lower than the established norm, you know that is worth further investigation. The key part of that is the last three words: worth further investigation. To get the final answer, you still need a controlled experiment.
SomewhatLikely 5y ago
Different prevalences of variants over time and different rates of infection over time would complicate such a time-based control.

a45a33s 5y ago

the covid vaccine trials had no control groups either

rsfern 5y ago
They were all doubly blinded placebo controlled trials… Are you talking about how they unblinded the study after efficacy results started coming in?
- a45a33s 5y ago
  yes i am talking about how they got rid of the control groups once it appeared to be working