OC didn't quite understand what "statistically significant" means in the statistical context. I believe he mixes it with the coloquial meaning of "significance"

OC didn't quite understand what "statistically significant" means in the statistical context. I believe he mixes it with the coloquial meaning of "significance". A "statistically significant" result is a result unlikely to be explained solely by chance or random factors. It has a very low chance of occurring if there were no true effect in a research study. It does not coloquially mean it's wrong, or unimportant. The p value, or probability value, tells you the statistical significance of a finding for a given measurement confidence interval. Than can happen for a variety of reasons. Consequently, a result is significant or not because of all the parameters of the study, for example measurement characteristics, number of samples, etc, but not its results. While not wrong in the absolute, your answer mixed in examples of unknown cofounding variables which could be the cause of the specific study's result, but they are not in the strict sense examples of statistical insignificance, nor examples of causes of it. Their absence wouldn't have made the study any more significant. The study OP referenced had 947 patients over ~50 mo. [1], which ought to be a sufficient sample size, but those were split into two groups with different regimens [2]. A third and half had to be excluded because of how measurement was made or after review of the cases [3]. With ~320 per-group, the sample size is not that great anymore, so result might not be that precise, depending on the chosen confidence interval. Indeed, drilling down [4], we see a 97.5 CI, and probability values for each group of 0.49 and 0.046. That doesn't mean the observed trend is wrong, just that we can't know for sure - hence it's "insignificance" in statistical parlance. The absence of those effects given by in your examples wouldn't make it any more significant. What could make it more significant ? For example more patients, less exclusions, better confidance interval. If we look at the discussion in [3], the researchers themselves believe a problem was the measurement itself : "A limitation of this analysis is the relatively short duration of follow-up. As of the data cutoff date, the median overall survival was not reached in either group; follow-up is ongoing." A note on double blind studies : in this particular case, a double-blind randomized control group study is not practicable, as it would be a gigantic breach of ethics. DBRCT is not an acceptable research technique for all medical scientific endeavors. It has its applications, but this wouldn't be it. You can only measure difference with other cures if such cures exist, not with the absence thereof (i.e. a control, placebo group) if they do exist. A DBRCT'd mean you'd have to refuse care, which is not possible. Moreover, the medicine was experimental, you can't double blind, as the doctors in charge of the patient must be able to react quickly, and those might not be the clinicians in charge of the study. Consequently, the study's author were right in making it a simple CRT. Don't confuse methodology with medicine. [1] https://clinicaltrials.gov/ct2/show/NCT02741570 https://clinicaltrials.gov/ct2/show/NCT02741570 [2] https://www.thelancet.com/journals/lanonc/article/PIIS1470-2045(21)00121-2/fulltext https://www.thelancet.com/journals/lanonc/article/PIIS1470-2... [3] https://www.nejm.org/doi/full/10.1056/NEJMoa2026982 https://www.nejm.org/doi/full/10.1056/NEJMoa2026982 [4] https://www.cancertherapyadvisor.com/home/news/conference-coverage/esmo-2021/head-neck-cancer-nivolumab-survival-benefit-ipilimumab-treatment/ https://www.cancertherapyadvisor.com/home/news/conference-co...