4 ms·
Punchline - "Although not statistically significant, patients receiving remdesivir had a numerically faster time to clinical improvement than those receiving pl
by refurb 7y ago
Punchline - "Although not statistically significant, patients receiving remdesivir had a numerically faster time to clinical improvement than those receiving placebo among patients with symptom duration of 10 days or less (hazard ratio 1·52 [0·95–2·43])."
That said, this morning Gilead reported that in their own pivotal trial that the drug met it's primary endpoint.[1] What we don't know is how big of an effect it was.
[1]https://endpts.com/gilead-pivotal-covid-19-study-of-remdesivir-met-its-primary-endpoint/ https://endpts.com/gilead-pivotal-covid-19-study-of-remdesiv...
- air7 7y agoAny line that starts with "Although not statistically significant" should be automatically deleted.
- sparkie 7y agoNot if you want to pump and dump a stock.
- downerending 7y agoIn general, yes. In an emergency when there are no other obvious better alternatives at hand, looking at results with relatively poor p-values might still be a pretty reasonable thing to do.
- seemslegit 7y agoNot really.
- QuesnayJr 7y agoBut it doesn't mean that drug doesn't work -- it means that the sample was too small to say that it works. If the sample were 100,000, then that would mean that it doesn't work. This was around 225, which isn't tiny, but given that we are well-endowed with people dying of COVID who would rather not, it makes sense to do more testing. Which they call for.
- seemslegit 7y agoDidn't say it doesn't make sense to do more testing, but relaxing your reasoning standard in emergencies is a bad idea.
- tarboreus 7y agoThey said "looking at more results," not "relax our reasoning standards."
- seemslegit 7y agoYes, you should not look at any result in an emergency you wouldn't otherwise - in fact you should probably only increase scrutiny given how much incentive there is to merely get a headline with your drug mentioned as a potential covid19 cure. The OP is basically arguing for a "something must be done and this is something" reasoning.
- downerending 7y agoThere's always "We've tried nothing, and we're all out of ideas." :-)
- seemslegit 7y agoYes there is. Not having effective options against a new virus is the null hypothesis. So by all means do whatever research has promising leads but don't lower burden of proof for accepting something as clinically effective.
- downerending 7y agoI'm picturing the passengers on the Titanic standing around, refusing to move until presented with an option proven valid to p < 0.95.
- cy_hauser 7y ago
- jakeogh 7y agoalso https://news.ycombinator.com/item?id=23013667 https://news.ycombinator.com/item?id=23013667
- im3w1l 7y agoDepends on how much power the study has.
- klodolph 7y agoNo, that go against the current standards of scientific ethics. “Not statistically significant” does not mean the effect does not exist. It means that the study provides weak evidence, but weak evidence ≠ no evidence. It tags things for further study. This is one of those cases where people get too hung up on P values. You have to look at the bigger picture; look at the entire scientific process.
- gfodor 7y agoI mean, it does provide no evidence for the specific hypothesis. It may provide evidence of other hypotheses. So you're right, but you're also wrong, without making yourself more clear. Poor precision in the use of language around clinical data has been a major problem in all of this, in the press especially, where for the most part they clearly don't give a damn and just have an agenda to spin in most cases. Because of this, scientists publishing ought to try to use language that ensures their work cannot be misinterpreted. The notes in the results section here about non-statistically significant aspects to the data fail that test, since an average reader would likely walk away with a false mental model of what the data is showing. edit: my comment is wrong, i'm tired and haven't had enough coffee. the criticisms below are valid. i do have a point, but won't jam it in here.
- klodolph 7y ago> I mean, it does provide no evidence for the specific hypothesis. Again, this is incorrect. It provides weak evidence, not no evidence. You talk about precise language here but you are conflating “not statistically significant” with “no evidence”. You can’t just handwave the difference between these two things.
- jefftk 7y ago> it does provide no evidence for the specific hypothesis It does provide some evidence: we are more likely to see this result in a world where the hypothesis is true than a world where it is false.
- gus_massa 7y ago
- pps43 7y agoWhat if there are 10 independent studies, all showing positive, but not statistically significant effect?
- scythe 7y agoStatistical significance is not a bright-line test. The p value reported is the probability that the observed effects occur if there are actually no effects. Notably, p is always greater than zero. In general, when less data is available, larger p values are considered more interesting, while when more data is available, p must be smaller to be interesting. In particle physics, p < 0.001 means almost nothing. In the particular quoted line, p = 0.07ish which is above the human-defined, conventional threshold of 0.05 but in this case there are almost no other useful data and so the phenomenon is interesting.
- achenatx 7y agoThe issue with statistical significance is that it might have a strong effect in some people and no effect in most people and statistical significance cant detect that. For example, statistically chocolate doesnt cause acne. Yet I 100% get acne from chocolate. There is enough variance in people that many things that work anecdotally cannot be proven statistically.