3 ms·
The reason why this has happened consistently in medicine is because the studies are being funded by for profit companies and they are using extremely biased me
by Symmetry 14y ago
The reason why this has happened consistently in medicine is because the studies are being funded by for profit companies and they are using extremely biased methodology to make their drugs look much better than they actually are. Whereas MAPS is a non-profit, and their methodology is designed to be as neutral as possible. The research may well still not pan out as trial sizes increase and patients are more effectively double blinded, but there is definitely reason for optimism based on the results so far.
If a for-profit company pushes a study through a first stage trial and then loses out at a later stage they lose much more money than if they had given up quickly. What's really necessary for most promising early stage treatments to not pan out is for genuinely good treatments to be rare, plus people being more likely to report impressive results than failures. Both of which I hope you'll admit are just as much a problem for not-for-profit researchers as for-profit ones.
- Alex3917 14y ago"Both of which I hope you'll admit are just as much a problem for not-for-profit researchers as for-profit ones." I agree that genuinely good treatments are rare. However, the file drawer problem is definitely much worse for industry-sponsored studies than for, say, non-profit- or NIH-sponsored ones. MAPS has a tiny budget of less than a million dollars a year, and if they were to do a study and not publish the results they would get completely discredited. And for an NIH researcher, if they didn't publish their results then it's unlikely they'd ever get more grants in the future. I don't know of any specific stats about how the file drawer effect for NIH-sponsored studies compares to industry sponsored ones, as that would be the closest comparison. But I do have some quotes that address the issue in general, and explain why the problem is much worse with for-profit companies: "The failure to publish unsuccessful trials presents a problem in many research areas. When a study has produced non-significant results, it is less likely to be submitted for publication; and, if it is submitted, it is less likely to be favourably reviewed or accepted for publication. But although publication bias affects all areas of research to some extent, it is particularly acute when it comes to drug trials. This is because most of the clinical trials evaluating medications are financially sponsored by the companies that produce and stand to profit from them. The companies own the data that come out of trials they sponsor, and they can choose how to present them to the public -- or to withhold them and not present them to the public at all. With widely prescribed medications, billions of dollars are at stake. In this case, it is not reviewers or journal editors who are impeding publication of negative findings. Rather it is the companies themselves that decide to withold negative data from publication. [...] Most studies showing negative results remain unpublished, and short of making official enquiries to government agencies, their data are unavailable to researchers, doctors and the public at large." Source: The Emperor's New Drugs, p. 39 "The fact that what gets published are the trials with positive results was most convincingly shown by a group of researchers at the Oregon Health and Science University, who followed up on our initial analysis of the FDA data by comparing the conclusions reached by the FDA with those reported by the drug companies in journal articles. Of 38 drug-company clinical trials that the FDA viewed as having positive results, all but one was published. In the same documents, the FDA described 36 trials as having negative or questionable results. Most of these negative trials were not published at all, and of the few that were published, most were described in the journal articles as showing positive results -- despite the fact that the FDA had concluded that they had not." Source: The Emperor's New Drugs, p. 67, describing the journal article<a href="http://www.nejm.org/doi/full/10.1056/NEJMsa065779>Selective http://www.nejm.org/doi/full/10.1056/NEJMsa065779>Select... Publication of Antidepressant Trials and Its Influence on Apparent Efficacy</a> "When application was made to the Swedish Drug Authority for approval of five new antidepressant drugs, 28 separate clinical trials evaluating the drugs' effectiveness had been published in medical journals. The results were overwhelmingly positive: Twenty-two studies showed that the new drugs were significantly more effective than a placebo, and only 6 showed no difference. In Sweden, drug applications must including <i>all</i> known studies -- published or not -- relevant to the new drug. When researchers from the Swedish Drug Authority went through the new drug applications for the five new antidepressants, they found that a total of 42 studies had been completed. It turned out that exactly half of these showed that the new antidepressants are more effective than placebos and half found that they are not. The 22 postive articles that had been published represented 19 of the positive studies (three were published twice). In contrast, only six of the 21 studies with negative or inconclusive findings had been published. Even the most conscientious doctor could know only the results of the studies that had been published and would reasonably conclude that the weight of the evidence about the new antidepressants was overwhelmingly positive. The Swedish researchers commented that their finding that 40 percent of the studies that had been completed on these drugs remained unpublished (as independent studies, not pooled with others) was consistent with the findings of other such reviews. In their conculsion, they warned that 'for anyone who relies on published data alone to choose a specific drug, our results should be a cause for concern.... Any attempt to recommend a specific drug is likely to be based on biased evidence.' What else can a practicing physician rely on but the published data? In an understated way, these researchers were telling doctors that they could not trust the published scientific evidence about antidepressants to be complete and unbiased." Source: Overdosed America, p. 115 "Whereas many of the negative trials were not published at all, some of the positive trias were published many times, a practice known as 'salami slicing', and this was often done in ways that would make it difficult for reviewers to know that the studies were based on the same data. In some cases, the authors were different, and references to previous publication of the data were often missing. Sometimes there were minor differences in the data between one publication and another, as well as between the data as presented to regulatory agencies and the data as published. So a reviewer trying to summarize the data would be likely to count the positive data more than once. Another trick was to publish only some of the data from a clinical trial, a manoeuvre that researchers call cherry-picking the data. Some clinical trials are conducted in more than one location. These are called multi-centre studies. Multi-centre studies make it easier to find sufficient patients to conduct the trial. They also make it easier to cherry-pick the data. For example, one multi-centre study of Prozac was presented to the FDA as showing a drug-placebo difference of three points on the Hamilton scale. When the data from this clinical trial was published, the difference was reported as 15 points -- a five-times increase in effectiveness. How was this magical augmentation of the benefits of Prozac accomplished? The full study was conducted on 245 patients. The published paper reported data from only 27 of these patients. In the published version, the data from the bulk of the patients were left out, making the drug seem much more effective than it really was." Source: The Emperor's New Drugs, p. 41 "Drug companies also publish 'pooled analyses' of the trials they have conducted. That is, they bundle together the results of different trials and analyse the drug-placebo difference across them. This is similar to the meta-analyses my colleagues and I have conducted, but with one important difference. Our meta-analyses, in common with most others reported in the scientific literature, are based on all the studies that we were able to find. In contrast, the drug companies pick and choose which studies they wish to include in their pooled analyses. For example, GlaxoSmithKline submitted 15 clinical trials of Seroxat to Swedish regulators. In addition to being published individualy -- sometimes more than once -- studies with positive results were also included in six different pooled analyses. Most of the studies with negative results were, of course, not included in the pooled analyses. There is yet another way in which pooled analyses can hide negative data. Rather than not publishing the negative data at all, the companies can bundle them together with data from positive trials, so that the overall result is positive. By so doing, they can truthfully claim that they have published the data from a negative trial, while hiding the fact that those data showed no difference between a drug and placebo. The article by the Swedish regulators showed that the data from about 20 per cent of trials were not published at all. The data from another 20 per cent of trials were bundled together with data from more successful trials, so that their negative results were hidden from view. Taken together, approximately 40 per cent of the data are kept out of sight. Practices like this [...] make it exceptionally difficult for reiewers to establish how effective the drugs really are." Source: The Emperor's New Drugs, p. 42
- Symmetry 14y agoWell, you've assembled a huge pile of evidence there for the idea that publication bias exists, but I notice that none of the evidence you provide shows that publication bias is stronger for commercial drugs.
- Alex3917 14y agoAlso, here are some more quotes on the influence of commercial funding, showing indirectly that the file drawer effect combined with the methodology produces results that are much more biased and of poorer quality than non-profit funded studies. "Studies repeatedly document bias in commercially sponsored research, but the medical journals seem powerless to control the scientific integrity of their own pages. In 2003, separate studies were published in JAMA and the <i>British Medical Journal</i> showing that the odds are 3.6 to 4 times greater that commercially sponsored studies will favor the sponsor's products than studies without commercial funding. And in August of 2003 a study published in JAMA found that among the highest-quality clinical trials, the odds that those with commercial sponsorship will recommend the new drug are 5.3 times greater than for studies funded by non-profit organizations. The authors noted that the lopsided results of commercially sponsored research may be 'due to biased interpretation of trial results.' They cautioned that readers should 'carefully evaluate whether conclusions in randomized trials are supported by data.' In other words, doctors are warned that even the best research published in the best journals <i>cannot</i> be taken at face value." Source: Overdose America, p. 97. Citing <a href="http://www.ncbi.nlm.nih.gov/pubmed/12533125>Scope http://www.ncbi.nlm.nih.gov/pubmed/12533125>Scope and Impact of Financial Conflicts of Interest in Biomedical Research: A systematic Review</a>, <a href="http://www.bmj.com/content/326/7400/1167.full>Pharmaceutical http://www.bmj.com/content/326/7400/1167.full>Pharmaceut... Industry Sponsorship and Research Outcome and Quality: A Systematic Review</a>, <a href="http://jama.jamanetwork.com/article.aspx?articleid=197132>Association http://jama.jamanetwork.com/article.aspx?articleid=197132... of Funding and Conclusions in Randomized Drug Trials</a> "In the case of calcium channel blockers like Norvasc, for instance, one survey of seventy articles about their safety found that 96 percent of authors who were supportive of the drugs had financial ties to the companies that made them, whereas only 37 percent of authors who were critical had such ties." Source: The Truth About Drug Companies, p. 107, citing <a href="http://www.nejm.org/doi/full/10.1056/NEJM199801083380206>Conflict http://www.nejm.org/doi/full/10.1056/NEJM199801083380206>... of Interest in the Debate over Calcium Channel Antagonists</a> "A team of reserachers at the Beth Israel Medical Center in New York have examined the outcome of clinical trials as a function of who had sponsored them. They found that approximately 75 per cent of drug-company studies showed favourable results for their own drugs, but only 25 per cent of them showed favourable results for the product of a competing company. In studies that are not sponsored by a drug company, the success rate is approximately 50 per cent." Source: The Emperor's New Drugs, p. 62, describing <a href="http://journals.cambridge.org/action/displayAbstract;jsessionid=8791B742923B6F462F07F977A080DEFB.journals?fromPage=online&aid=522832>Relationship http://journals.cambridge.org/action/displayAbstract;jsessio... between drug company funding and outcomes of clinical psychiatric research</a> See also: http://jama.jamanetwork.com/article.aspx?articleid=202867 http://jama.jamanetwork.com/article.aspx?articleid=202867 http://www.ncbi.nlm.nih.gov/pubmed/17550302 http://www.ncbi.nlm.nih.gov/pubmed/17550302