4 ms·

Isn't your last paragraph describing a randomized controlled trial? The page lists 31 for Ivermectin [0] and (for comparison) 40 for HCQ [1]. I don't find the e

by 0-_-0 5y ago

Isn't your last paragraph describing a randomized controlled trial? The page lists 31 for Ivermectin [0] and (for comparison) 40 for HCQ [1]. I don't find the evidence for HCQ convincing, but Ivermectin seems solid from what I can tell. It's hard to explain 28 out of 31 randomized controlled trials showing a positive result with bias, since you would expect that bias to affect both HCQ and Ivermectin. Why would Ivermectin show much better results than HCQ? What kind of bias would affect Ivermectin much more than HCQ? [0]: https://ivmmeta.com/#fig_fpr https://ivmmeta.com/#fig_fpr [1]: https://hcqmeta.com/#fcite_rct https://hcqmeta.com/#fcite_rct (Figure 7, scroll down)

gus_massa 5y ago
Nice! The HCQ looks definitively like report+publication bias, but it would be nice to see an analysis by an statistic expert. The Ivermectin looks more interesting. I read the first study on the list Chowdhury et al. https://c19ivermectin.com/chowdhury.html https://c19ivermectin.com/chowdhury.html * If I understand correctly it's no double blind, but as far as I can see both group get the same treatment. Also the criteria are hard (like death, instead of how much pain do you feel in the scale from 1 to 10), so my guess is that it's not a big problem here. * It's strange that one group got Ivermectin and the other got HCQ. So why is this study not in the other site? * I think it's not preregistered, so it's not clear how many similar experiments were made. * The main result is that 0/60 patients with Ivermectin get hospitalized, but 2/56 patients with HCQ. To simplify the discussion, can I round it to 0/60 patients with Ivermectin get hospitalized, but 2/60 patients with HCQ? * This looks like 1±1/60 in both case to me. I'm a Mathematician but I'm not an expert in statistic. Anyway I took to many laboratory classes for Physics, and it definitively looks like 1±1/60 that is very unconvincing. I don't know how they get 80.6% from that, but I can ask one of my friends. * Anyway, let's use the null hypothesis and suppose that have Ivermectin and HCQ.no effects. And for simplicity, let's assume that the hospitalization rate of no treatment is 1/60. So if we repeat these kind of experiments with groups of 60 patients we must use a binomial distribution. From https://www.di-mgt.com.au/binomial-calculator.html https://www.di-mgt.com.au/binomial-calculator.html with n=60 and p=0.01666666666 Probability of 0 cases: 36.5% Probability of 1 case: 37.1% Probability of 2 cases: 18.6% Probability of more than 2 cases: 7.8% So if we make many experiments, and each of them has 60 persons with Ivermectin and 60 persons with HCQ, then in 36.5%18.6%=6.8% of them the we will get only by a fluke a result like the reported. So if there are 16 not pre-registered studies, on of them may get a result like this. Pre-registration is very important. Looking at the other studies, in early interventions there are too many that have 0 cases in the treatment branch. For late intervention treatments, most have not 0 cases in both groups. It's suspicious, and my guess is that is inclining the result.
- 0-_-0 5y ago
  I see your point that it could be P-hacking (as in https://xkcd.com/882/ https://xkcd.com/882/), but why would Ivermectin be so much more P-hacked than HCQ?
  - gus_massa 5y ago
    Let's look at the biggest studies HQC: RECOVERY (RCT) -9% 1.09 [0.97-1.23] death 421/1,561 790/3,155 Mitjà (RCT) 52% 0.48 [0.15-1.57] death 4/1,196 9/1,301 SOLIDARITY (RCT) -19% 1.19 [0.89-1.59] death 104/947 84/906 --- Ivermectin: Seet (CLUS. RCT) 50% 0.50 [0.33-0.76] severe case 32/617 64/619 12mg Elgazzar (RCT) 92% 0.08 [0.02-0.35] death 2/200 24/200 112mg Mahmud (DB RCT) 86% 0.14 [0.01-2.75] death 0/183 3/183 12mg --- My guess is: HCQ was very promising in early 2020 and there has been enough time to make big RCT. Ivermectin got more promising this year and the big RCT didn't finish yet, or the results are still not published. (In a big trial with a few thousand patients, you need a lot of time, coordination and paperwork, so the result is less likely to not get published.)
    - 0-_-0 5y ago
      If you're right, there should be a correlation with stidy size and results (I'll check for this tomorrow if I remember). It would still have to be explained why do small studies show a much worse result for HCQ than Ivermectin.

by 0-_-0 5y ago

gus_massa 5y ago

Nice! The HCQ looks definitively like report+publication bias, but it would be nice to see an analysis by an statistic expert. The Ivermectin looks more interesting. I read the first study on the list Chowdhury et al. https://c19ivermectin.com/chowdhury.html https://c19ivermectin.com/chowdhury.html * If I understand correctly it's no double blind, but as far as I can see both group get the same treatment. Also the criteria are hard (like death, instead of how much pain do you feel in the scale from 1 to 10), so my guess is that it's not a big problem here. * It's strange that one group got Ivermectin and the other got HCQ. So why is this study not in the other site? * I think it's not preregistered, so it's not clear how many similar experiments were made. * The main result is that 0/60 patients with Ivermectin get hospitalized, but 2/56 patients with HCQ. To simplify the discussion, can I round it to 0/60 patients with Ivermectin get hospitalized, but 2/60 patients with HCQ? * This looks like 1±1/60 in both case to me. I'm a Mathematician but I'm not an expert in statistic. Anyway I took to many laboratory classes for Physics, and it definitively looks like 1±1/60 that is very unconvincing. I don't know how they get 80.6% from that, but I can ask one of my friends. * Anyway, let's use the null hypothesis and suppose that have Ivermectin and HCQ.no effects. And for simplicity, let's assume that the hospitalization rate of no treatment is 1/60. So if we repeat these kind of experiments with groups of 60 patients we must use a binomial distribution. From https://www.di-mgt.com.au/binomial-calculator.html https://www.di-mgt.com.au/binomial-calculator.html with n=60 and p=0.01666666666 Probability of 0 cases: 36.5% Probability of 1 case: 37.1% Probability of 2 cases: 18.6% Probability of more than 2 cases: 7.8% So if we make many experiments, and each of them has 60 persons with Ivermectin and 60 persons with HCQ, then in 36.5%18.6%=6.8% of them the we will get only by a fluke a result like the reported. So if there are 16 not pre-registered studies, on of them may get a result like this. Pre-registration is very important. Looking at the other studies, in early interventions there are too many that have 0 cases in the treatment branch. For late intervention treatments, most have not 0 cases in both groups. It's suspicious, and my guess is that is inclining the result.

0-_-0 5y ago
I see your point that it could be P-hacking (as in https://xkcd.com/882/ https://xkcd.com/882/), but why would Ivermectin be so much more P-hacked than HCQ?
- gus_massa 5y ago
  Let's look at the biggest studies HQC: RECOVERY (RCT) -9% 1.09 [0.97-1.23] death 421/1,561 790/3,155 Mitjà (RCT) 52% 0.48 [0.15-1.57] death 4/1,196 9/1,301 SOLIDARITY (RCT) -19% 1.19 [0.89-1.59] death 104/947 84/906 --- Ivermectin: Seet (CLUS. RCT) 50% 0.50 [0.33-0.76] severe case 32/617 64/619 12mg Elgazzar (RCT) 92% 0.08 [0.02-0.35] death 2/200 24/200 112mg Mahmud (DB RCT) 86% 0.14 [0.01-2.75] death 0/183 3/183 12mg --- My guess is: HCQ was very promising in early 2020 and there has been enough time to make big RCT. Ivermectin got more promising this year and the big RCT didn't finish yet, or the results are still not published. (In a big trial with a few thousand patients, you need a lot of time, coordination and paperwork, so the result is less likely to not get published.)
  - 0-_-0 5y ago
    If you're right, there should be a correlation with stidy size and results (I'll check for this tomorrow if I remember). It would still have to be explained why do small studies show a much worse result for HCQ than Ivermectin.