3 ms·
One of the problem with meta analysis is the lack of pre-registration of most of the studies about ivermectin. There are many studies in many places. Imagine j
by gus_massa 5y ago
One of the problem with meta analysis is the lack of pre-registration of most of the studies about ivermectin.
There are many studies in many places. Imagine just for a moment that some new drug has no effect. Anyway, 1/20 of them will get a fake statistical significant result. The problem is that this study with a fake good result has a high chance of been published. The other 19 have a high chance of not been published, because the doctors lost interest, because the journal don't like null results, or other similar reasons. So if you make a review of the published studies, you will only find only/mostly the ones that get a good result due to a fluke.
This is not an imaginary problem, this is a known problem. Imagine Big Pharma has a new drug, and they want to it approved. They can hire 100 universities/hospitals to run a clinical study, suggest that only the ones with a good fluke result get published and then they get about 5 good papers about the drug. Perhaps they don't even have to make a suggestion, because it's boring to write a paper about a drug that does not work so the medic just don't write it, and it's boring to read a paper about a drug that does not work so the journal just publish it. It's the same effect, and nobody is lying ;) .
The solution for this is pre-registration. Before starting the trial, you must register it, explain what drugs you are going to use, what are you going to measure. With pre-registration, now the 100 studies are public. If you later find 5 good and 95 inconclusive or bad or disappeared, you know in the meta analysis the 5 good studies are only flukes.
- 0-_-0 5y agoI understand the P-hacking problem, but if that was the source of the positive result in the meta analysis that would mean that scientists are throwing away 95% of their studies on one of the most promising COVID treatments during a pandemic. In this case there's not really a reason to throw away a negative result. Considering the controversy around Ivermectin a study showing a negative result would get a huge amount of citations and attention. In any case, thank you for trying to use reason in your argument, we need more around this subject.
- gus_massa 5y ago> scientists are throwing away 95% of their studies My guess is that most of these are not studies on purpose. A MD or a Hospital tries a new drug that seam promising for a week/month. If the result is good they get very happy, and try to make as many fuss as possible, that means that they publish their work in some journal. If the result is bad they lose interest in the subject, or try to hide the wrong results under the carpet, or don't want to spend time in a paper that is very difficult to publish. So the good results of the informal tries are published, and the bad result get lost. No conspiracy. Just people doing people things. > one of the most promising COVID treatments during a pandemic I don't think it's promising, but I think it's important to discuss about it and try to understand why it is necessary to get better evidence.
- 0-_-0 5y agoThen you have to explain why are results about Ivermectin more likely to be thrown out than studies about hydroxychloroquine (https://hcqmeta.com/ https://hcqmeta.com/), which has only 73.8% positive studies and shows much smaller improvement. Why are prophylaxis studies more successful than early treatment studies, which are more successful than late treatment studies? Are they more likely to be thrown out? Why are people throwing out studies exactly in a pattern to create a sort of pattern that would indicate a working drug?
- gus_massa 5y agoWhich studies are published and which are dropped is not so easy to calculate. Sometimes the team need to publish to get a grant, sometimes one graduate student needs a paper to get the degree, sometimes some of the team member is too famous or a friend of the editor. It vary a lot, papers get published with not good enough but "promising" results. Also, if the main objective of the study (like reducing the number of death) is not successful, you can pick another criteria of successfulness like number of day in the hospital, viral load after a week, ... there are many to choice. In particular in the fists study about hydroxychloroquine by Raoult, he claims that it's a success, but if you read the paper you can notice that there is no one death in the trial group and no deaths in the control group. (And the control group is not a randomized control group, just an unrelated bunch of patients in another city.) About why early interventions with hydroxychloroquine get apparent better results: That's a good question, but the publication bias is not the only source of problems. It's difficult to be sure, but look at the study here in the OP. My guess is that the treatment with ivermectin includes implicitly phones call "Locatel" and that additional care from doctors improve the result. Other studies have similar additional care for the people under treatment. And if you start the phone calls earlier, it's better that starting late. So you may measure it in the trials. You may disagree with my explanation, and it's fine. The real problem is how to be sure if I'm wrong or you are wrong (or both). So the solution is to do double blind studies, were some people that get the medicine and other people that get a placebo that looks identical. But the important part is that both get all the additional health treatment. Both get phone calls that perhaps are useful. Both get automatic weakly nurse checks that are common in other treatments and perhaps are useful. ... So it's not necessary to guess if any of the other part of the treatment is useful, because the only difference is drug vs placebo.