3 ms·

Wow, those P values are very low. Is this normal in this type of research? Are they calculated directly based on combinatorial arguments?

londons_explore 5y ago
No. P values this low are very rare. I haven't checked any of the analysis with my own data, but the approach seems pretty legitimate. Each is calculated in a different way, using mostly-non-overlapping evidence, so it's almost legitimate to multiply them together to get P<1.3x10^-18... Not that there's any need - each alone is very convincing. The whole lot really should have been written as three separate papers, with better written abstracts, and a metapaper, written by different authors, combining the evidence and comparing to the evidence for other hypothesis.
deleted 5y ago
[deleted]
MontyCarloHall 5y ago
> Are they calculated directly based on combinatorial arguments? Yes. Most of the extremely low p-values come from binomial tests with a very low binomial probability p_bin relative to the fraction of positive outcomes. For example, > We identified 45 point mutations that were introduced in Branch O (hereafter referred to as “pre-outbreak mutations.”) We observed that the pre-outbreak mutations were over-represented in ORF S (P = 1.2 × 10−13, binomial test with the expected probability equal to the length of ORF S relative to the SARS-CoV-2 genome). So p_bin = len(ORF S)/len(genome). The spike protein sequence (ORF S) is roughly 1300 bases and the entire SARS-CoV-2 genome is roughly 30 kilobases long, yielding p_bin ≈ 0.04. If there are 45 total pre-outbreak mutations, and assuming they ought to be uniformly distributed across the genome (big assumption, BTW!), then seeing 17 of them occur in the spike protein sequence would yield a binomial p-value (one-sided test) on the order of 1e-13. The other low p-values either also come from binomial tests with similarly low binomial probabilities relative to positive outcomes, or from Fisher exact tests with gross imbalances between categories (and high counts to back those up).

londons_explore 5y ago

No. P values this low are very rare. I haven't checked any of the analysis with my own data, but the approach seems pretty legitimate. Each is calculated in a different way, using mostly-non-overlapping evidence, so it's almost legitimate to multiply them together to get P<1.3x10^-18... Not that there's any need - each alone is very convincing. The whole lot really should have been written as three separate papers, with better written abstracts, and a metapaper, written by different authors, combining the evidence and comparing to the evidence for other hypothesis.

deleted 5y ago

[deleted]

MontyCarloHall 5y ago

> Are they calculated directly based on combinatorial arguments? Yes. Most of the extremely low p-values come from binomial tests with a very low binomial probability p_bin relative to the fraction of positive outcomes. For example, > We identified 45 point mutations that were introduced in Branch O (hereafter referred to as “pre-outbreak mutations.”) We observed that the pre-outbreak mutations were over-represented in ORF S (P = 1.2 × 10−13, binomial test with the expected probability equal to the length of ORF S relative to the SARS-CoV-2 genome). So p_bin = len(ORF S)/len(genome). The spike protein sequence (ORF S) is roughly 1300 bases and the entire SARS-CoV-2 genome is roughly 30 kilobases long, yielding p_bin ≈ 0.04. If there are 45 total pre-outbreak mutations, and assuming they ought to be uniformly distributed across the genome (big assumption, BTW!), then seeing 17 of them occur in the spike protein sequence would yield a binomial p-value (one-sided test) on the order of 1e-13. The other low p-values either also come from binomial tests with similarly low binomial probabilities relative to positive outcomes, or from Fisher exact tests with gross imbalances between categories (and high counts to back those up).