3 ms·
I presume you are right, but this doesn't inspire confidence in the system. Of course it's not the same trial: if you have a weak effect, it's better to modif
by nkurz 16y ago
I presume you are right, but this doesn't inspire confidence in the system. Of course it's not the same trial: if you have a weak effect, it's better to modify a superficial variable and start the trial again. Why try to climb out of the hole when you can roll the dice again? Do this 20 times and you might stumble on something statistically significant. From a 'true science' perspective, the right thing to do is to carry the trial through to conclusion and publish the negative results. But it's hard to make a living doing this.
- protomyth 16y agoGiven that a FDA trial can last years and cost a lot of money, a company can't really afford to "carry on" if it is obvious to themselves and the people they have conducting the trial that something is very different from their predicted results. They had a belief in what the outcome should be before the trial (else you would not go through with / spend money on the trial). A deviation from the expected results really needs to be looked at and you really need to find out why all the prelim testing is not accurately predicting what is actually going on.
- nkurz 16y agoYes, it's not that the drug companies are actually running trials and random hoping for spurious correlations, but that the traditional measures of statistical significance can't be used in cases where trials might be stopped early. Even inspecting the results midway through a trial to see if the effect is being shown is problematic. Yes, the deviation from expectation really needs to be looked at. The problem is that the financial reward for the companies might be greater to just 'move on' and never publish those unexpectedly negative results. But this means that the unexpectedly positive results for the next test are given undue weight. I don't think the problem is really the science, or the publishers, or the drug companies, but the unrealistic weight placed on erroneously calculated statistics. This combined with a lot of money at stake forces the companies to "produce results" whether or not this is of long term public benefit.