2 ms·
"Both of which I hope you'll admit are just as much a problem for not-for-profit researchers as for-profit ones." I agree that genuinely good treatments are ra
by Alex3917 14y ago
"Both of which I hope you'll admit are just as much a problem for not-for-profit researchers as for-profit ones."
I agree that genuinely good treatments are rare. However, the file drawer problem is definitely much worse for industry-sponsored studies than for, say, non-profit- or NIH-sponsored ones. MAPS has a tiny budget of less than a million dollars a year, and if they were to do a study and not publish the results they would get completely discredited. And for an NIH researcher, if they didn't publish their results then it's unlikely they'd ever get more grants in the future. I don't know of any specific stats about how the file drawer effect for NIH-sponsored studies compares to industry sponsored ones, as that would be the closest comparison. But I do have some quotes that address the issue in general, and explain why the problem is much worse with for-profit companies:
"The failure to publish unsuccessful trials presents a problem in many research areas. When a study has produced non-significant results, it is less likely to be submitted for publication; and, if it is submitted, it is less likely to be favourably reviewed or accepted for publication. But although publication bias affects all areas of research to some extent, it is particularly acute when it comes to drug trials. This is because most of the clinical trials evaluating medications are financially sponsored by the companies that produce and stand to profit from them. The companies own the data that come out of trials they sponsor, and they can choose how to present them to the public -- or to withhold them and not present them to the public at all. With widely prescribed medications, billions of dollars are at stake. In this case, it is not reviewers or journal editors who are impeding publication of negative findings. Rather it is the companies themselves that decide to withold negative data from publication. [...]
Most studies showing negative results remain unpublished, and short of making official enquiries to government agencies, their data are unavailable to researchers, doctors and the public at large." Source: The Emperor's New Drugs, p. 39
"The fact that what gets published are the trials with positive results was most convincingly shown by a group of researchers at the Oregon Health and Science University, who followed up on our initial analysis of the FDA data by comparing the conclusions reached by the FDA with those reported by the drug companies in journal articles. Of 38 drug-company clinical trials that the FDA viewed as having positive results, all but one was published. In the same documents, the FDA described 36 trials as having negative or questionable results. Most of these negative trials were not published at all, and of the few that were published, most were described in the journal articles as showing positive results -- despite the fact that the FDA had concluded that they had not." Source: The Emperor's New Drugs, p. 67, describing the journal article<a href="http://www.nejm.org/doi/full/10.1056/NEJMsa065779>Selective http://www.nejm.org/doi/full/10.1056/NEJMsa065779>Select... Publication of Antidepressant Trials and Its Influence on Apparent Efficacy</a>
"When application was made to the Swedish Drug Authority for approval of five new antidepressant drugs, 28 separate clinical trials evaluating the drugs' effectiveness had been published in medical journals. The results were overwhelmingly positive: Twenty-two studies showed that the new drugs were significantly more effective than a placebo, and only 6 showed no difference. In Sweden, drug applications must including <i>all</i> known studies -- published or not -- relevant to the new drug. When researchers from the Swedish Drug Authority went through the new drug applications for the five new antidepressants, they found that a total of 42 studies had been completed. It turned out that exactly half of these showed that the new antidepressants are more effective than placebos and half found that they are not. The 22 postive articles that had been published represented 19 of the positive studies (three were published twice). In contrast, only six of the 21 studies with negative or inconclusive findings had been published. Even the most conscientious doctor could know only the results of the studies that had been published and would reasonably conclude that the weight of the evidence about the new antidepressants was overwhelmingly positive.
The Swedish researchers commented that their finding that 40 percent of the studies that had been completed on these drugs remained unpublished (as independent studies, not pooled with others) was consistent with the findings of other such reviews. In their conculsion, they warned that 'for anyone who relies on published data alone to choose a specific drug, our results should be a cause for concern.... Any attempt to recommend a specific drug is likely to be based on biased evidence.' What else can a practicing physician rely on but the published data? In an understated way, these researchers were telling doctors that they could not trust the published scientific evidence about antidepressants to be complete and unbiased." Source: Overdosed America, p. 115
"Whereas many of the negative trials were not published at all, some of the positive trias were published many times, a practice known as 'salami slicing', and this was often done in ways that would make it difficult for reviewers to know that the studies were based on the same data. In some cases, the authors were different, and references to previous publication of the data were often missing. Sometimes there were minor differences in the data between one publication and another, as well as between the data as presented to regulatory agencies and the data as published. So a reviewer trying to summarize the data would be likely to count the positive data more than once.
Another trick was to publish only some of the data from a clinical trial, a manoeuvre that researchers call cherry-picking the data. Some clinical trials are conducted in more than one location. These are called multi-centre studies. Multi-centre studies make it easier to find sufficient patients to conduct the trial. They also make it easier to cherry-pick the data. For example, one multi-centre study of Prozac was presented to the FDA as showing a drug-placebo difference of three points on the Hamilton scale. When the data from this clinical trial was published, the difference was reported as 15 points -- a five-times increase in effectiveness. How was this magical augmentation of the benefits of Prozac accomplished? The full study was conducted on 245 patients. The published paper reported data from only 27 of these patients. In the published version, the data from the bulk of the patients were left out, making the drug seem much more effective than it really was." Source: The Emperor's New Drugs, p. 41
"Drug companies also publish 'pooled analyses' of the trials they have conducted. That is, they bundle together the results of different trials and analyse the drug-placebo difference across them. This is similar to the meta-analyses my colleagues and I have conducted, but with one important difference. Our meta-analyses, in common with most others reported in the scientific literature, are based on all the studies that we were able to find. In contrast, the drug companies pick and choose which studies they wish to include in their pooled analyses. For example, GlaxoSmithKline submitted 15 clinical trials of Seroxat to Swedish regulators. In addition to being published individualy -- sometimes more than once -- studies with positive results were also included in six different pooled analyses. Most of the studies with negative results were, of course, not included in the pooled analyses.
There is yet another way in which pooled analyses can hide negative data. Rather than not publishing the negative data at all, the companies can bundle them together with data from positive trials, so that the overall result is positive. By so doing, they can truthfully claim that they have published the data from a negative trial, while hiding the fact that those data showed no difference between a drug and placebo. The article by the Swedish regulators showed that the data from about 20 per cent of trials were not published at all. The data from another 20 per cent of trials were bundled together with data from more successful trials, so that their negative results were hidden from view. Taken together, approximately 40 per cent of the data are kept out of sight. Practices like this [...] make it exceptionally difficult for reiewers to establish how effective the drugs really are." Source: The Emperor's New Drugs, p. 42
- Symmetry 14y agoWell, you've assembled a huge pile of evidence there for the idea that publication bias exists, but I notice that none of the evidence you provide shows that publication bias is stronger for commercial drugs.