3 ms·
I wonder if we're in danger of p hacking here. Suppose in the scramble to contain the outbreak, everything is tested for efficacy against the disease. There wil
by analog31 7y ago
I wonder if we're in danger of p hacking here. Suppose in the scramble to contain the outbreak, everything is tested for efficacy against the disease. There will surely be some positive studies just due to luck.
- asdfasgasdgasdg 7y agoAbsolutely correct. This is a small part of why observational evidence is not very strong evidence. People should take each observation with a grain of salt.
- Cyclone_ 7y agoPerhaps that's true with some observations. I'm no expert on the subject here, but this does seem like a scientific study that wouldn't really involve luck.
- vikramkr 7y agoAll scientific studies, including this one, involve uncertainty, which also Involves luck. Wed need to see a lot more data (and human data as well) before prescribing NO or viagra for coronavirus treatment. You also have the ability to select runs with faster or lower replication rates, confounding variables like the temperature that could have changed with each run or across the plate, etc.
- sp332 7y agoThis is from 2004. Here's a quick search for papers which cite this one: https://scholar.google.com/scholar?cites=3174486874880036805&as_sdt=5,30&sciodt=0,30&hl=en https://scholar.google.com/scholar?cites=3174486874880036805... Edit: "Role of epithelial nitric oxide in airway viral infection" https://www.sciencedirect.com/science/article/pii/S0891584906000918 https://www.sciencedirect.com/science/article/pii/S089158490... says NO inhibits both virus replication and latency of viruses, including coxsackievirus, influenza A and B, rhinovirus, cytomegalovirus, vaccinia virus, ectromelia virus, human herpesvirus-1, and HPIV3 and cites 10 papers for these claims.
- Taek 7y agoEasy solution is to run the trial again with 2-3x the sample size. If you think there are a lot of lot of studies we are searching, use 5x the sample size.
- ajross 7y agoExactly. That seems to be what happened with the chloroquine kerfuffle too, where the original paper got picked up and pushed irresponsibly in the media and policy world, but no one in the scientific community has been able to replicate it. To some extent this is self-healing, though, in that in a giant outbreak like this someone's going to try every promising-looking technique in practice and return good data on real world results.
- outlace 7y agoThat wouldn’t be p hacking, it would just be false positives from multiple tests. But yeah that seems a possibility