4 ms·

Then you have to explain why are results about Ivermectin more likely to be thrown out than studies about hydroxychloroquine (https://hcqmeta.com/ https://hcqme

by 0-_-0 5y ago

Then you have to explain why are results about Ivermectin more likely to be thrown out than studies about hydroxychloroquine (https://hcqmeta.com/ https://hcqmeta.com/), which has only 73.8% positive studies and shows much smaller improvement. Why are prophylaxis studies more successful than early treatment studies, which are more successful than late treatment studies? Are they more likely to be thrown out? Why are people throwing out studies exactly in a pattern to create a sort of pattern that would indicate a working drug?

gus_massa 5y ago
Which studies are published and which are dropped is not so easy to calculate. Sometimes the team need to publish to get a grant, sometimes one graduate student needs a paper to get the degree, sometimes some of the team member is too famous or a friend of the editor. It vary a lot, papers get published with not good enough but "promising" results. Also, if the main objective of the study (like reducing the number of death) is not successful, you can pick another criteria of successfulness like number of day in the hospital, viral load after a week, ... there are many to choice. In particular in the fists study about hydroxychloroquine by Raoult, he claims that it's a success, but if you read the paper you can notice that there is no one death in the trial group and no deaths in the control group. (And the control group is not a randomized control group, just an unrelated bunch of patients in another city.) About why early interventions with hydroxychloroquine get apparent better results: That's a good question, but the publication bias is not the only source of problems. It's difficult to be sure, but look at the study here in the OP. My guess is that the treatment with ivermectin includes implicitly phones call "Locatel" and that additional care from doctors improve the result. Other studies have similar additional care for the people under treatment. And if you start the phone calls earlier, it's better that starting late. So you may measure it in the trials. You may disagree with my explanation, and it's fine. The real problem is how to be sure if I'm wrong or you are wrong (or both). So the solution is to do double blind studies, were some people that get the medicine and other people that get a placebo that looks identical. But the important part is that both get all the additional health treatment. Both get phone calls that perhaps are useful. Both get automatic weakly nurse checks that are common in other treatments and perhaps are useful. ... So it's not necessary to guess if any of the other part of the treatment is useful, because the only difference is drug vs placebo.
- 0-_-0 5y ago
  Isn't your last paragraph describing a randomized controlled trial? The page lists 31 for Ivermectin [0] and (for comparison) 40 for HCQ [1]. I don't find the evidence for HCQ convincing, but Ivermectin seems solid from what I can tell. It's hard to explain 28 out of 31 randomized controlled trials showing a positive result with bias, since you would expect that bias to affect both HCQ and Ivermectin. Why would Ivermectin show much better results than HCQ? What kind of bias would affect Ivermectin much more than HCQ? [0]: https://ivmmeta.com/#fig_fpr https://ivmmeta.com/#fig_fpr [1]: https://hcqmeta.com/#fcite_rct https://hcqmeta.com/#fcite_rct (Figure 7, scroll down)
  - gus_massa 5y ago
    Nice! The HCQ looks definitively like report+publication bias, but it would be nice to see an analysis by an statistic expert. The Ivermectin looks more interesting. I read the first study on the list Chowdhury et al. https://c19ivermectin.com/chowdhury.html https://c19ivermectin.com/chowdhury.html * If I understand correctly it's no double blind, but as far as I can see both group get the same treatment. Also the criteria are hard (like death, instead of how much pain do you feel in the scale from 1 to 10), so my guess is that it's not a big problem here. * It's strange that one group got Ivermectin and the other got HCQ. So why is this study not in the other site? * I think it's not preregistered, so it's not clear how many similar experiments were made. * The main result is that 0/60 patients with Ivermectin get hospitalized, but 2/56 patients with HCQ. To simplify the discussion, can I round it to 0/60 patients with Ivermectin get hospitalized, but 2/60 patients with HCQ? * This looks like 1±1/60 in both case to me. I'm a Mathematician but I'm not an expert in statistic. Anyway I took to many laboratory classes for Physics, and it definitively looks like 1±1/60 that is very unconvincing. I don't know how they get 80.6% from that, but I can ask one of my friends. * Anyway, let's use the null hypothesis and suppose that have Ivermectin and HCQ.no effects. And for simplicity, let's assume that the hospitalization rate of no treatment is 1/60. So if we repeat these kind of experiments with groups of 60 patients we must use a binomial distribution. From https://www.di-mgt.com.au/binomial-calculator.html https://www.di-mgt.com.au/binomial-calculator.html with n=60 and p=0.01666666666 Probability of 0 cases: 36.5% Probability of 1 case: 37.1% Probability of 2 cases: 18.6% Probability of more than 2 cases: 7.8% So if we make many experiments, and each of them has 60 persons with Ivermectin and 60 persons with HCQ, then in 36.5%18.6%=6.8% of them the we will get only by a fluke a result like the reported. So if there are 16 not pre-registered studies, on of them may get a result like this. Pre-registration is very important. Looking at the other studies, in early interventions there are too many that have 0 cases in the treatment branch. For late intervention treatments, most have not 0 cases in both groups. It's suspicious, and my guess is that is inclining the result.

by 0-_-0 5y ago

gus_massa 5y ago

Which studies are published and which are dropped is not so easy to calculate. Sometimes the team need to publish to get a grant, sometimes one graduate student needs a paper to get the degree, sometimes some of the team member is too famous or a friend of the editor. It vary a lot, papers get published with not good enough but "promising" results. Also, if the main objective of the study (like reducing the number of death) is not successful, you can pick another criteria of successfulness like number of day in the hospital, viral load after a week, ... there are many to choice. In particular in the fists study about hydroxychloroquine by Raoult, he claims that it's a success, but if you read the paper you can notice that there is no one death in the trial group and no deaths in the control group. (And the control group is not a randomized control group, just an unrelated bunch of patients in another city.) About why early interventions with hydroxychloroquine get apparent better results: That's a good question, but the publication bias is not the only source of problems. It's difficult to be sure, but look at the study here in the OP. My guess is that the treatment with ivermectin includes implicitly phones call "Locatel" and that additional care from doctors improve the result. Other studies have similar additional care for the people under treatment. And if you start the phone calls earlier, it's better that starting late. So you may measure it in the trials. You may disagree with my explanation, and it's fine. The real problem is how to be sure if I'm wrong or you are wrong (or both). So the solution is to do double blind studies, were some people that get the medicine and other people that get a placebo that looks identical. But the important part is that both get all the additional health treatment. Both get phone calls that perhaps are useful. Both get automatic weakly nurse checks that are common in other treatments and perhaps are useful. ... So it's not necessary to guess if any of the other part of the treatment is useful, because the only difference is drug vs placebo.

0-_-0 5y ago
Isn't your last paragraph describing a randomized controlled trial? The page lists 31 for Ivermectin [0] and (for comparison) 40 for HCQ [1]. I don't find the evidence for HCQ convincing, but Ivermectin seems solid from what I can tell. It's hard to explain 28 out of 31 randomized controlled trials showing a positive result with bias, since you would expect that bias to affect both HCQ and Ivermectin. Why would Ivermectin show much better results than HCQ? What kind of bias would affect Ivermectin much more than HCQ? [0]: https://ivmmeta.com/#fig_fpr https://ivmmeta.com/#fig_fpr [1]: https://hcqmeta.com/#fcite_rct https://hcqmeta.com/#fcite_rct (Figure 7, scroll down)
- gus_massa 5y ago
  Nice! The HCQ looks definitively like report+publication bias, but it would be nice to see an analysis by an statistic expert. The Ivermectin looks more interesting. I read the first study on the list Chowdhury et al. https://c19ivermectin.com/chowdhury.html https://c19ivermectin.com/chowdhury.html * If I understand correctly it's no double blind, but as far as I can see both group get the same treatment. Also the criteria are hard (like death, instead of how much pain do you feel in the scale from 1 to 10), so my guess is that it's not a big problem here. * It's strange that one group got Ivermectin and the other got HCQ. So why is this study not in the other site? * I think it's not preregistered, so it's not clear how many similar experiments were made. * The main result is that 0/60 patients with Ivermectin get hospitalized, but 2/56 patients with HCQ. To simplify the discussion, can I round it to 0/60 patients with Ivermectin get hospitalized, but 2/60 patients with HCQ? * This looks like 1±1/60 in both case to me. I'm a Mathematician but I'm not an expert in statistic. Anyway I took to many laboratory classes for Physics, and it definitively looks like 1±1/60 that is very unconvincing. I don't know how they get 80.6% from that, but I can ask one of my friends. * Anyway, let's use the null hypothesis and suppose that have Ivermectin and HCQ.no effects. And for simplicity, let's assume that the hospitalization rate of no treatment is 1/60. So if we repeat these kind of experiments with groups of 60 patients we must use a binomial distribution. From https://www.di-mgt.com.au/binomial-calculator.html https://www.di-mgt.com.au/binomial-calculator.html with n=60 and p=0.01666666666 Probability of 0 cases: 36.5% Probability of 1 case: 37.1% Probability of 2 cases: 18.6% Probability of more than 2 cases: 7.8% So if we make many experiments, and each of them has 60 persons with Ivermectin and 60 persons with HCQ, then in 36.5%18.6%=6.8% of them the we will get only by a fluke a result like the reported. So if there are 16 not pre-registered studies, on of them may get a result like this. Pre-registration is very important. Looking at the other studies, in early interventions there are too many that have 0 cases in the treatment branch. For late intervention treatments, most have not 0 cases in both groups. It's suspicious, and my guess is that is inclining the result.