6 ms·

A 13% reduction in all-cause mortality is huge. There are other methodological/reliability concerns, but malaria isn't near 100% of death, even among children.

by mlyle 3y ago

CurtisWright 3y ago
[dead]
vmilner 3y ago
I hadn’t grasped the all-cause aspect from a skim read.
lettergram 3y ago
That’s not all that huge when there’s such a small number of deaths in the given group over that time. This is part of the reason a lot of drugs later prove to be useless… I also don’t see a citation for the 13% reduction in all cause mortality. To be honest, that’s an insanely weird metric to review for a malaria vaccine. Why not malaria related deaths? Total in 2015 study [1]: > 8922 children and 6537 young infants were included in the modified intention-to-treat analyses. Total deaths: …? Are we talking 8 deaths in the control group and 7 in those injected? This isn’t clear at all to me [1] https://pubmed.ncbi.nlm.nih.gov/25913272/ https://pubmed.ncbi.nlm.nih.gov/25913272/ On page 20 of the 2021 report we see 1443 deaths in comparison group and 1421 in implementing group [2]. With a pretty wide margin of error I might add. Is that what they’re reporting…? [2] https://www.nitag-resource.org/sites/default/files/2022-05/Milligan-Moore-2021-statistical-report-rtss-pilot-evaluation.pdf https://www.nitag-resource.org/sites/default/files/2022-05/M...
- michaelrpeskin 3y ago
  Actually, all cause is the right metric. Let's say the vaccine stops 100% of malaria but causes severe heart attacks. You'd probably want to know that. I didn't review _this_ study, so I won't comment on the quality of analysis, but what you want to know about a vaccine is "effectivity" does it stop contraction and spread of the illness and "safety" does it not cause any other problems that in aggregate are worse than the illness (all cause mortality is one metric here).
  - Fomite 3y ago
    And indeed there are a number of trials that show a decrease in all cause mortality (mass childhood administration of azithromycin comes to mind) that we'd miss if we were just looking at already known endpoints.
- roywiggins 3y ago
  All-cause mortality is easier to measure, it also takes into account any deaths caused by the vaccine, and any deaths caused by non-malaria proximate causes that would nevertheless have been averted by the vaccine. Malaria is really bad for you even if you survive it, and probably makes you more vulnerable to dying of other things: "For instance, clinical malaria is known to exhaust T cells, he says, and vaccination, by preventing infection, might therefore leave T cells readier to combat other pathogens. Such a general survival benefit has been documented for measles and tuberculosis vaccination. Other leading causes of death in the young children in these areas include pneumonia and diarrhea caused by pathogens including Streptococcus pneumoniae and rotavirus." So, surviving malaria could very easily make it harder to survive your next encounter with rotavirus; vaccinating against malaria would then reduce deaths from rotavirus.
  - timr 3y ago
    Yeah, but...it's also subject to sample bias. Especially in a place like east Africa, having enough resources (not just money -- time, transport, etc.) to get a crazy expensive new vaccine could easily have co-variates that affect all-cause mortality. It's great to report it, but you definitely want to see the cause-specific numbers, too.
    - deleted 3y ago
      [deleted]
    - roywiggins 3y ago
      They used two sets of controls: 1, an entire set of communities where the vaccination wasn't offered at all, and 2, children who were not eligible due to age. They weren't just comparing children who got the vaccine with children who didn't, they were comparing children who were eligible for the vaccine and those who weren't, and that couldn't have been so easily confounded: "79 areas where the RTS,S vaccine was administered and 79 comparator areas where it was not available" "Researchers then compared the death rates of babies whose age made them eligible to receive three doses of the vaccine with those of young children who were not age-eligible for three doses, in both RTS,S areas and unvaccinated areas." This is obviously not foolproof, but it seems like a substantial bulwark against obvious confounders. If you can show that 1) the mortality rate in the vaccinated communities was lower in the age-eligible children than not-age-eligible[0] and 2) this effect was absent in unvaccinated communities that seems like it goes quite far towards proving that it's real and not an artifact. It probably even underestimates how protective it is at an individual level. (communities which had the vaccine offered also had fewer cases of severe malaria recorded in their hospitals, so there's a straightforward reason to think it does something) [1] okay probably age-ineligible children have a different mortality rate regardless. You'd be taking the difference between those two groups of children and comparing it to the same difference in communities without the vaccination offered. This difference-of-difference would tell you whether the vaccine is doing anything. It wouldn't matter if only the richer children received the vaccine if the communities have basically the same distribution of wealth since you're comparing differences between age cohorts, not between vaccinated and unvaccinated.
BurningFrog 3y ago
One way to describe Africa's development in this era is that they're economically 100 years behind the rich world, but catching up 3 years per year. In that environment, you'd expect child mortality to go down year by year, regardless of this vaccine. Maybe the study corrects for that, but the complete absence of details in this report isn't promising.
- regularfry 3y ago
  It's not regardless of the vaccine, though. You'd expect child mortality to go down year by year because of things like that. There used to be malaria in the UK, I'm sure child mortality dropped when it died off here too.
  - BurningFrog 3y ago
    My point is that you need to compare how much the death rates went down in comparable regions, before attributing this 13% decrease to the vaccine.
    - benchaney 3y ago
      That’s not how these kinds of trials work. The 13% figure comes from comparing the control and intervention groups, which were observed over the same period of time. The change in baseline mortality over time of the entire population isn’t relevant.
    - ipqk 3y ago
      It is compared to non-vaccinated regions, if you had bothered to read the article. > To calculate mortality in the three countries, where death registry statistics are unreliable, the researchers employed tens of thousands of community reporters—more than 14,000 of them in Kenya alone—to conduct household surveys of childhood deaths in 79 areas where the RTS,S vaccine was administered and 79 comparator areas where it was not available.
    - mattmaroon 3y ago
      You really don’t if it’s controlled. The control group would have any other factors baked in. If the group that got the vaccine has a 13% lower all cause mortality than the group that didn’t, and it was your standard randomized controlled double blind study, that’s a huge win.

A 13% reduction in all-cause mortality is huge. There are other methodological/reliability concerns, but malaria isn't near 100% of death, even among children.

by mlyle 3y ago

A 13% reduction in all-cause mortality is huge. There are other methodological/reliability concerns, but malaria isn't near 100% of death, even among children.

CurtisWright 3y ago

[dead]

vmilner 3y ago

I hadn’t grasped the all-cause aspect from a skim read.

lettergram 3y ago

That’s not all that huge when there’s such a small number of deaths in the given group over that time. This is part of the reason a lot of drugs later prove to be useless… I also don’t see a citation for the 13% reduction in all cause mortality. To be honest, that’s an insanely weird metric to review for a malaria vaccine. Why not malaria related deaths? Total in 2015 study [1]: > 8922 children and 6537 young infants were included in the modified intention-to-treat analyses. Total deaths: …? Are we talking 8 deaths in the control group and 7 in those injected? This isn’t clear at all to me [1] https://pubmed.ncbi.nlm.nih.gov/25913272/ https://pubmed.ncbi.nlm.nih.gov/25913272/ On page 20 of the 2021 report we see 1443 deaths in comparison group and 1421 in implementing group [2]. With a pretty wide margin of error I might add. Is that what they’re reporting…? [2] https://www.nitag-resource.org/sites/default/files/2022-05/Milligan-Moore-2021-statistical-report-rtss-pilot-evaluation.pdf https://www.nitag-resource.org/sites/default/files/2022-05/M...

michaelrpeskin 3y ago
Actually, all cause is the right metric. Let's say the vaccine stops 100% of malaria but causes severe heart attacks. You'd probably want to know that. I didn't review _this_ study, so I won't comment on the quality of analysis, but what you want to know about a vaccine is "effectivity" does it stop contraction and spread of the illness and "safety" does it not cause any other problems that in aggregate are worse than the illness (all cause mortality is one metric here).
- Fomite 3y ago
  And indeed there are a number of trials that show a decrease in all cause mortality (mass childhood administration of azithromycin comes to mind) that we'd miss if we were just looking at already known endpoints.
roywiggins 3y ago
All-cause mortality is easier to measure, it also takes into account any deaths caused by the vaccine, and any deaths caused by non-malaria proximate causes that would nevertheless have been averted by the vaccine. Malaria is really bad for you even if you survive it, and probably makes you more vulnerable to dying of other things: "For instance, clinical malaria is known to exhaust T cells, he says, and vaccination, by preventing infection, might therefore leave T cells readier to combat other pathogens. Such a general survival benefit has been documented for measles and tuberculosis vaccination. Other leading causes of death in the young children in these areas include pneumonia and diarrhea caused by pathogens including Streptococcus pneumoniae and rotavirus." So, surviving malaria could very easily make it harder to survive your next encounter with rotavirus; vaccinating against malaria would then reduce deaths from rotavirus.
- timr 3y ago
  Yeah, but...it's also subject to sample bias. Especially in a place like east Africa, having enough resources (not just money -- time, transport, etc.) to get a crazy expensive new vaccine could easily have co-variates that affect all-cause mortality. It's great to report it, but you definitely want to see the cause-specific numbers, too.
  - deleted 3y ago
    [deleted]
  - roywiggins 3y ago
    They used two sets of controls: 1, an entire set of communities where the vaccination wasn't offered at all, and 2, children who were not eligible due to age. They weren't just comparing children who got the vaccine with children who didn't, they were comparing children who were eligible for the vaccine and those who weren't, and that couldn't have been so easily confounded: "79 areas where the RTS,S vaccine was administered and 79 comparator areas where it was not available" "Researchers then compared the death rates of babies whose age made them eligible to receive three doses of the vaccine with those of young children who were not age-eligible for three doses, in both RTS,S areas and unvaccinated areas." This is obviously not foolproof, but it seems like a substantial bulwark against obvious confounders. If you can show that 1) the mortality rate in the vaccinated communities was lower in the age-eligible children than not-age-eligible[0] and 2) this effect was absent in unvaccinated communities that seems like it goes quite far towards proving that it's real and not an artifact. It probably even underestimates how protective it is at an individual level. (communities which had the vaccine offered also had fewer cases of severe malaria recorded in their hospitals, so there's a straightforward reason to think it does something) [1] okay probably age-ineligible children have a different mortality rate regardless. You'd be taking the difference between those two groups of children and comparing it to the same difference in communities without the vaccination offered. This difference-of-difference would tell you whether the vaccine is doing anything. It wouldn't matter if only the richer children received the vaccine if the communities have basically the same distribution of wealth since you're comparing differences between age cohorts, not between vaccinated and unvaccinated.

BurningFrog 3y ago

One way to describe Africa's development in this era is that they're economically 100 years behind the rich world, but catching up 3 years per year. In that environment, you'd expect child mortality to go down year by year, regardless of this vaccine. Maybe the study corrects for that, but the complete absence of details in this report isn't promising.

regularfry 3y ago
It's not regardless of the vaccine, though. You'd expect child mortality to go down year by year because of things like that. There used to be malaria in the UK, I'm sure child mortality dropped when it died off here too.
- BurningFrog 3y ago
  My point is that you need to compare how much the death rates went down in comparable regions, before attributing this 13% decrease to the vaccine.
  - benchaney 3y ago
    That’s not how these kinds of trials work. The 13% figure comes from comparing the control and intervention groups, which were observed over the same period of time. The change in baseline mortality over time of the entire population isn’t relevant.
  - ipqk 3y ago
    It is compared to non-vaccinated regions, if you had bothered to read the article. > To calculate mortality in the three countries, where death registry statistics are unreliable, the researchers employed tens of thousands of community reporters—more than 14,000 of them in Kenya alone—to conduct household surveys of childhood deaths in 79 areas where the RTS,S vaccine was administered and 79 comparator areas where it was not available.
  - mattmaroon 3y ago
    You really don’t if it’s controlled. The control group would have any other factors baked in. If the group that got the vaccine has a 13% lower all cause mortality than the group that didn’t, and it was your standard randomized controlled double blind study, that’s a huge win.