4 ms·

Breaking the Seal on Drug Research

46 points by gruseom 13y ago

VengefulCynic 13y ago
To my mind, this sort of thing is a lot like crypto research. As a workman programmer, I know next to nothing about crypto algorithms so I can read the documents, but I can't provide much in the way of meaningful research. That said, I know that there are lot of interested crypto researchers out there who can provide meaningful analysis and who have an incentive to speak up if they find something. But on government-funded closed-source crypto algorithms, I have to take the NSA's word for it...
gwintrob 13y ago
"results of only about half of clinical trials make their way into medical journals" Sounds like an incredible problem not only for bringing new drugs to market, but also for all of the medication that's currently prescribed.
- kvb 13y ago
  Journals are typically interested in publishing novel results. Why would it be a problem if studies which find that a drug works as expected are not published, as long as they are still submitted to the FDA and assessed there?
  - gruseom 13y ago
    Because it's bad both for science and for the public interest to keep this data secret. Also, many of the studies that have been suppressed do not "find that a drug works as expected". Nobody's saying that journals have to publish every study, just that the data should be available for review. That's so bedrock a principle of science that it's hard to imagine a credible argument against it.
    - cup 13y ago
      I think the problem is the fact that pharmaeutial companies are trying to turn a profit to ensure their survival. If you invest a billion dollars in a new drug (which is the average cost these days) and its a flop, are you going to adverise it or is it in your inerest to be silent and hope another drug company makes the same mistake as you and suffers the same opportunity costs, therefore minimising your losses. This is a quintessential problem between public and private research.
  - dyno12345 13y ago
    # Fund 20 studies on a placebo # find that 2 or 3 of them beat another placebo by a "statistically significant margin" # publish those 2 or 3 studies, throw away the rest # Get your placebo FDA approved, patent it, get the advertising department into gear and start selling your placebo to the public
- leoc 13y ago
  Ben Goldacre's Bad Pharma paints a really alarming picture of the problem: http://www.amazon.com/Bad-Pharma-companies-mislead-patients/dp/0007350740 http://www.amazon.com/Bad-Pharma-companies-mislead-patients/...
tokenadult 13y ago
The previous submission https://news.ycombinator.com/item?id=5970873 https://news.ycombinator.com/item?id=5970873 received minimal attention from the HN new page earlier today. My thought on the overall implications of this article is that the scope of the new era of transparency will be much broader than discussed in the article. Any product or service claim that purports to be based on research may eventually have to be based on TRANSPARENT research, lest consumer fraud be suspected. That would apply as much to "alternative" treatments http://www.sciencebasedmedicine.org/sense-and-nonsense-about-alternative-medicine-in-usa-today/ http://www.sciencebasedmedicine.org/sense-and-nonsense-about... as to prescribed treatments ordered by licensed physicians. It would apply to the claims of automobile manufacturers and to the claims of ergonomic keyboard manufacturers. It would apply not just to claims by established big companies, but also to claims by Soylent or DropBox or any other startup that promises benefits from a new product or service. And that would be a good thing, on the whole. If the economy continues to operate on a free-enterprise basis, companies new or old, large or small, might continue to be able to puff their products and services, and consumers continue to be able to buy most that aren't actively harmful. People aren't likely to be required to be rational and evidence-based any time soon. But if consumers decide to only buy drugs, transportation vehicles, foodstuffs, or other products or services that have reasonable evidence demonstrating their safety and effectiveness, that is an exercise of freedom too. I'd be glad to have a lot more data about what data are out there about how well products and services I might pay for actually work. All the startup founders who read Hacker News might well devote some thought to how their business model will develop in a world in which nearly all product and service claims can be tested by independently gathered and analyzed evidence.
- Alex3917 13y ago
  Research into alternative medicine is generally less problematic in terms of transparency because most of it is government funded. So you don't have pharma companies hiding data they don't like, bribing journals to publish data they like, etc. Occasionally you'll get supplement makers funding small trials, e.g. with zinc, but most of these you'd already be ignoring for other reasons anyway.
Fomite 13y ago
While as an epidemiologist I wholeheartedly endorse the push for making corporate research data more open, this sentence: "they relied too heavily on the assumption that the articles published in journals accurately represented the results of all clinical trials that had been conducted." Is a little misleading. Meta-analysis, which is what Cochrane reviews are, are very conscious of the "file drawer effect", and looking for evidence of non-publication is a rather fundamental aspect of doing meta-analysis.

46 points by gruseom 13y ago

To my mind, this sort of thing is a lot like crypto research. As a workman programmer, I know next to nothing about crypto algorithms so I can read the documents, but I can't provide much in the way of meaningful research. That said, I know that there are lot of interested crypto researchers out there who can provide meaningful analysis and who have an incentive to speak up if they find something. But on government-funded closed-source crypto algorithms, I have to take the NSA's word for it...

gwintrob 13y ago

"results of only about half of clinical trials make their way into medical journals" Sounds like an incredible problem not only for bringing new drugs to market, but also for all of the medication that's currently prescribed.

kvb 13y ago
Journals are typically interested in publishing novel results. Why would it be a problem if studies which find that a drug works as expected are not published, as long as they are still submitted to the FDA and assessed there?
- gruseom 13y ago
  Because it's bad both for science and for the public interest to keep this data secret. Also, many of the studies that have been suppressed do not "find that a drug works as expected". Nobody's saying that journals have to publish every study, just that the data should be available for review. That's so bedrock a principle of science that it's hard to imagine a credible argument against it.
  - cup 13y ago
    I think the problem is the fact that pharmaeutial companies are trying to turn a profit to ensure their survival. If you invest a billion dollars in a new drug (which is the average cost these days) and its a flop, are you going to adverise it or is it in your inerest to be silent and hope another drug company makes the same mistake as you and suffers the same opportunity costs, therefore minimising your losses. This is a quintessential problem between public and private research.
- dyno12345 13y ago
  # Fund 20 studies on a placebo # find that 2 or 3 of them beat another placebo by a "statistically significant margin" # publish those 2 or 3 studies, throw away the rest # Get your placebo FDA approved, patent it, get the advertising department into gear and start selling your placebo to the public
leoc 13y ago
Ben Goldacre's Bad Pharma paints a really alarming picture of the problem: http://www.amazon.com/Bad-Pharma-companies-mislead-patients/dp/0007350740 http://www.amazon.com/Bad-Pharma-companies-mislead-patients/...

tokenadult 13y ago

The previous submission https://news.ycombinator.com/item?id=5970873 https://news.ycombinator.com/item?id=5970873 received minimal attention from the HN new page earlier today. My thought on the overall implications of this article is that the scope of the new era of transparency will be much broader than discussed in the article. Any product or service claim that purports to be based on research may eventually have to be based on TRANSPARENT research, lest consumer fraud be suspected. That would apply as much to "alternative" treatments http://www.sciencebasedmedicine.org/sense-and-nonsense-about-alternative-medicine-in-usa-today/ http://www.sciencebasedmedicine.org/sense-and-nonsense-about... as to prescribed treatments ordered by licensed physicians. It would apply to the claims of automobile manufacturers and to the claims of ergonomic keyboard manufacturers. It would apply not just to claims by established big companies, but also to claims by Soylent or DropBox or any other startup that promises benefits from a new product or service. And that would be a good thing, on the whole. If the economy continues to operate on a free-enterprise basis, companies new or old, large or small, might continue to be able to puff their products and services, and consumers continue to be able to buy most that aren't actively harmful. People aren't likely to be required to be rational and evidence-based any time soon. But if consumers decide to only buy drugs, transportation vehicles, foodstuffs, or other products or services that have reasonable evidence demonstrating their safety and effectiveness, that is an exercise of freedom too. I'd be glad to have a lot more data about what data are out there about how well products and services I might pay for actually work. All the startup founders who read Hacker News might well devote some thought to how their business model will develop in a world in which nearly all product and service claims can be tested by independently gathered and analyzed evidence.

Alex3917 13y ago
Research into alternative medicine is generally less problematic in terms of transparency because most of it is government funded. So you don't have pharma companies hiding data they don't like, bribing journals to publish data they like, etc. Occasionally you'll get supplement makers funding small trials, e.g. with zinc, but most of these you'd already be ignoring for other reasons anyway.

Fomite 13y ago

While as an epidemiologist I wholeheartedly endorse the push for making corporate research data more open, this sentence: "they relied too heavily on the assumption that the articles published in journals accurately represented the results of all clinical trials that had been conducted." Is a little misleading. Meta-analysis, which is what Cochrane reviews are, are very conscious of the "file drawer effect", and looking for evidence of non-publication is a rather fundamental aspect of doing meta-analysis.