10 ms·

“Evidence-based medicine has been hijacked”: John Ioannidis (2016)

209 points by ajna91 7y ago

ekianjo 7y ago
> The sales and marketing departments in most companies are more powerful than their R&D departments. Yes, this is correct. > Hence, the design, conduct, reporting, and dissemination of this clinical evidence becomes an advertisement tool. This does not follow well logically. The design of clinical trials still belongs to R&D and not Marketing/Sales departments. Even if Marketing/Sales want to target for a specific outcome, there is nothing saying that this particular outcome will be met until you actually try it clinically. And specific outcomes are usually desirable outcomes: for example, showing that your drug is non just superior to placebo, but superior to the current best-in-class treatment. Or measuring patient-related outcomes that are tied to economic value (for example, making a patient recover faster has tangible societal benefits - they can come back to work, be an active family member, etc...) is also relevant these days. So, there is not a strong link between "pushing for an outcome that you can advertise" and "this outcome being clinically meaningless". If it were meaningless, FDAs and other regulatory bodies would not even accept it during the clinical trial design review phase. > As for basic research Basic research should be the responsibility of academics, not companies. And they do. But not many academics focus on "checking whether what we have been using for 20 years really works", there's too much priority on new clinical targets.
- Alex3917 7y ago
  > This does not follow well logically. Instead of trying to reason from first principles about whether or not what he's saying is true, maybe instead go on Amazon and buy one of the dozens of books on the subject.
  - wallace_f 7y ago
    I don't know if what he is saying is true, but I do know from having worked in another industry that the idea of sales/marketing dictating product development is not necessarily illogical, even if at face value it is.
    - marcus_holmes 7y ago
      in a lot of industries, Sales/Marketing dictating product development is the default. Sales and Marketing folks get used to that kind of power... and then they transfer to a new industry.
- RobertRoberts 7y ago
  > If it were meaningless, FDAs and other regulatory bodies would not even accept it during the clinical trial design review phase. Go look up how many corporate stooges are in the place in the FDA over the years. A good starting point is the history of Aspartame, I'd link some articles, but I suspect the sources would be shot down and evidence ignored. The people at the FDA were complicit in its approval, despite scientific evidence showing it was dangerous.
  - abrichr 7y ago
    I would appreciate links to articles on the subject, if you have them handy!
    - Alex3917 7y ago
      The tl;dr is that the same guys who fabricated the evidence about Saddam Hussein having WMDs also fabricated evidence for aspartame being safe. https://www.huffpost.com/entry/donald-rumsfeld-and-the-s_b_805581 https://www.huffpost.com/entry/donald-rumsfeld-and-the-s_b_8...
      - deleted 7y ago
        [deleted]
      - ben_w 7y ago
        Did Donald Rumsfeld also convince the European Union, who have their own interests and a similar sized economy, that it was safe? https://www.efsa.europa.eu/en/press/news/131210 https://www.efsa.europa.eu/en/press/news/131210 or the “more than 90 countries worldwide” [1] that the Canadian government refers to on their own website? https://www.canada.ca/en/health-canada/services/food-nutrition/food-safety/food-additives/sugar-substitutes/aspartame-artificial-sweeteners.html https://www.canada.ca/en/health-canada/services/food-nutriti... There are certainly cases where I don’t trust USA regulations. This just isn’t one of them. [1] number includes EU and US, but that still leaves more than sixty others.
        7y ago
- BurningFrog 7y ago
  > This does not follow well logically I think it's an observation, not a philosophical argument.
  - ekianjo 7y ago
    No, he said "Hence," right at the start of the sentence, as if it was a logical conclusion. It is not.
- PeterStuer 7y ago
  Sales and Marketing do control which outcomes are allowed to be published though. If you allow me selective disclosure of trials I can show you that my roulette strategy is infallible.
  - viraptor 7y ago
    Hasn't this been fixed with the requirement to pre-register trials? https://chrisblattman.com/2016/03/01/13719/ https://chrisblattman.com/2016/03/01/13719/ I'm not sure if every org requires it now or not, but if not then it seems like a question of time only.
    - hannob 7y ago
      > Hasn't this been fixed with the requirement to pre-register trials? If it was that simple... Preregistration is a step in the right direction, but there are still plenty of ways to game the system, particularly as it's not properly enforced: http://compare-trials.org/ http://compare-trials.org/
      - ekianjo 7y ago
        But it is the necessary step to highlight the problem so that actions can be taken. Also, you can safely assume that the large majority of outcomes that were not published were:\ - negative - worse than negative (i.e. safety issues?) - not properly designed So if you compare the number of trials disclosed/not disclosed for each drug/each company that gives you a pretty good picture of who is doing what already. With that data, it's then a lot easier to approach regulators to do something about it.
        tomp 7y ago
        My assumption is that most unpublished trials were "no effect". Be a drug company. Develop a "drug" that is really a placebo (has no effect). Do 10 trials. 8 are "no effect", 1 is positive (by chance), 1 is negative (by cance). Publish the positive drug. Market the drug as having positive effect. Sell the drug, make billions. It would take decades for enough other research (mostly "no effect") to accumulate before it becomes obvious that your drug actually has no effect.
- wazoox 7y ago
  Until recently, Big Pharma CEOs were either medical doctors of pharmacists. Marketing and finance were important, but not a driving force. Nowadays all of Big Pharma is headed by bean counters, marketing drones and finance guys, and are only interested in EBITDA. This is the main problem in all industries were science and engineering once prevailed.
  - shadowtree 7y ago
    Really? Novartis - former head of R&D. Pfizer - PhD in Veterinary Medicine (he joined their AH division in 93) Astra Zeneca - Veterinary too Merck / MSD - Lawyer Roche - Economics & Law Gilead - Biology abbvie - never went to college, 30 years at company Amgen - Biology
- nabla9 7y ago
  > If it were meaningless, FDAs and other regulatory bodies would not even accept it during the clinical trial design review phase. Billions in R&D goes to getting molecular entities approved by FDA that are clinically meaningless. The medical R&D is done just to get a product that provides already existing functionality or already existing treatment with a slightly modified molecule that can be patented. It's like the joke about man who goes to patent office and says that he invented gunpowder. Patent clerk says that unfortunately gunpowder has already been invented. The man says: "But I invented more of it." Approval process is not looking if the new drug is meaningful improvement over existing drugs. "Inventing more of the same" is approved if it can be tweaked a little to be unique. Alternative is also true. There are potentially clinical treatments and medicines that receive no approvals because there is no profit motive to push them trough the expensive process of approval.
  - galangalalgol 7y ago
    how do they get doctors to prescribe the patented versions if they aren't better?
    - mattmcknight 7y ago
      Sometimes they pay other companies not to make the unpatented drug: https://www.ftc.gov/news-events/media-resources/mergers-competition/pay-delay https://www.ftc.gov/news-events/media-resources/mergers-comp... More tricks here: https://hbr.org/2017/04/how-pharma-companies-game-the-system-to-keep-drugs-expensive https://hbr.org/2017/04/how-pharma-companies-game-the-system...
    - kyokpae 7y ago
      Sales reps bribe doctors, plain and simple. https://www.vice.com/en_us/article/pg7ez9/pharma-reps-are-basically-bribing-your-doctor-study-confirms https://www.vice.com/en_us/article/pg7ez9/pharma-reps-are-ba...
    - deleted 7y ago
      [deleted]
    - 7y ago
- rjkennedy98 7y ago
  > The design of clinical trials still belongs to R&D and not Marketing/Sales departments. This is FALSE. It's literally come out in court documents. https://www.motherjones.com/environment/2010/09/dan-markingson-drug-trial-astrazeneca/ https://www.motherjones.com/environment/2010/09/dan-markings... > In 1997, when Dr. Andrew Goudie, a psychopharmacologist at the University of Liverpool, asked AstraZeneca to fund a research study he was planning, a company official replied that “R&D is no longer responsible for Seroquel research—it is now the responsibility of Sales and Marketing.” The official also noted that funding decisions would depend on whether the study was likely to show a “competitive advantage for Seroquel.”
  - ChrisLomont 7y ago
    >This is FALSE. How do you demonstrate that one case implies all cases?
    - rjkennedy98 7y ago
      This isn't a small company nor is it a minor drug. This is one of the largest drug companies running a trial on one of the most lucrative drugs in the world. And secondly, why should ANY drug trial be managed by a PR department? When you ask people to risk their lives to be in a drug trial (in the link above the subject died in the trial), why should it be to improve the PR of a drug for one company. The fact that this exists at all is a major warning sign. The indifference of people to this is frankly amazing to me (its even worse than the disinformation provided by the OP).
dbt00 7y ago
(2016)
outlace 7y ago
There wasn’t a lot of substance in this article, would have to read Dr. Ioannidis’ referenced papers to actually understand his point.
- DoreenMichele 7y ago
  Off the top of my head: He does meta studies -- aka studies of studies -- and finds that most studies are flawed, so most of the evidence we think we have isn't really there. It doesn't exactly make him popular because he calls into question a lot of existing supposedly "proven" medical science. He talks a lot about the biased assumptions that are the foundation of a lot of the framing of studies. If your baseline framing is sufficiently flawed, the entire study is junk. He thinks a lot of studies are, in fact, junk.
  - Nasrudith 7y ago
    Stupid question - what makes us think the meta studies are themselves valid? Also wouldn't junk itself have many subclassifications as how it is junk? Say poor study data quality, wrong assumptions of wider applicability, wrong analysis or conclusions, etc.
    - DoreenMichele 7y ago
      what makes us think the meta studies are themselves valid? A. This is one guy, basically. I mean, I agree with him, generally, but not everyone does. B. It's easier to poke holes in someone else's thing than create a good study. So study A: "I wonder how fat you have to be to not fit through this door. Let's weigh people (instead of measuring them) to see how fat you have to be not fit through the door." Study B: "It's a great question, but it's badly done. Weighing them won't work. Let's just interview people and write down their self reported height!" Meta analyst: "Y'all are both wrong and stupid, geez." Scientists A and B: "Let's take the meta-analyst out back and beat him up!" "Sounds good to me!"
    - icelancer 7y ago
      >> Stupid question - what makes us think the meta studies are themselves valid? This is not a stupid question. Many meta studies are done by people with their own frameworks of deciding what is and isn't a legitimate approach to studying something. Unsurprisingly, their findings are overwhelmingly negative due to their own publication bias. Meta studies are pretty worthless in my opinion. Primary studies in all fields should be forced to be published with open data, and any study that received - directly or indirectly - more than $1 from the public taxpayer should be forced to publish Open Access. Lay bare all the raw data and let the software developers and data analysts get after it, not so-called expert meta reviewers.
nordsieck 7y ago
It looks like he gave a 30 minute talk [0] on this paper. ___ 0. https://www.youtube.com/watch?v=N63skNtYaJw https://www.youtube.com/watch?v=N63skNtYaJw
- acqq 7y ago
  Yes, the aricle we comment here was posted March 2016. The video is from April same year. Also June 2016 "John Ioannidis - Why most clinical research is not useful" (University of Oxford, Centre for Evidence-Based Medicine) https://www.youtube.com/watch?v=Uok-7NPFn4k https://www.youtube.com/watch?v=Uok-7NPFn4k The paper: https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002049 https://journals.plos.org/plosmedicine/article?id=10.1371/jo...
pessimizer 7y ago
> JI: A systematic review that combines biased pieces of evidence may unfortunately give another seal of authority to that biased evidence. Basically the same problem as CDOs.
- wallace_f 7y ago
  CDOs and the subprime mortgage crisis have some analogues to pre-Enlightenment darkness in human intellectual history. The idea that authoritative figures can use maths to give junk debt the midas touch is basically financial alchemy. There was no sound reasoning behind it. It was literally just "maths is really powerful, and these people are the Smartest Guys in the Room, which is not really that much different from "these guys are really Holy, and also the Smartest Guys in the Room" in previous generations of humanity. Just market it with Math, and you can defraud the most powerful and prosperous society in history.
newcrobuzon 7y ago
It is a sad reality that some of the regulatory agencies have been captured by the industry groups. This is especially concerning in healthcare and pharma industry. Take for example Vioxx and Opioid issues: Merck Manipulated the Science about the Drug Vioxx - https://www.ucsusa.org/manipulating-science-about-drug-vioxx https://www.ucsusa.org/manipulating-science-about-drug-vioxx Purdue infiltrated WHO, manipulated opioid policies to boost sales, report finds - https://arstechnica.com/science/2019/05/world-health-organization-parroted-purdues-deceptive-opioid-claims-report-says/ https://arstechnica.com/science/2019/05/world-health-organiz... Yet it is still surprising to see that on HN in general anyone questioning safety of pharmaceutical products get downvoted into oblivion.
- Madmallard 7y ago
  It's way worse than that. Fluoroquinolone antibiotics are well known in research for being seriously dangerous yet they were prescribed like candy for over a decade. So many people develop disabling symptoms overtime that end up not being linked to the drug because dna adduction and mtdna depletion won't necessarily show up quickly. The drugs have likely disabled hundreds of thousands at this point by Dr Bennett's estimations at UNC. There's something like 10,000 reports and those are assumed at being less than 10% and possibly 1% of total incidents.
EGreg 7y ago
There is a much bigger problem in the methodology, which is why meta studies won’t be the last word on it either https://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/ https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
nkurz 7y ago
Here's the underlying paper that the article is about: Evidence-based medicine has been hijacked: a report to David Sackett (2016) http://sci-hub.tw/10.1016/j.jclinepi.2016.02.012 http://sci-hub.tw/10.1016/j.jclinepi.2016.02.012
jshowa3 7y ago
After reading the interview, it sounds like a lot of scare mongering about bias without actually identifying specific things that are wrong with specific examples. Sure, it's nice to be looking for these issues, but people obviously have different opinions on what biases produce poor results and if biases are actually necessary and helpful.
qrbLPHiKpiux 7y ago
Case-in-point: Perdue Pharma, Oxycontin
cneurotic 7y ago
Dubious evidence in medical studies is a big problem. But I'd argue that it's only the core of the issue. The second layer out — and I think the more influential layer — is the JOURNALISM surrounding these medical studies. Even if the underlying science is sound, a lot of outlets and reporters routinely (and sometimes willfully) get the facts wrong. I attended a talk with Retraction Watch's Ivan Oransky, where he spelled out how easy it is for well-intentioned journalists to exaggerate, over-generalize, or misunderstand what a study says. It was eye-opening. And a little frightening.
lez 7y ago
He is a spiritual brother of Dr. Peter Gøtzsche, who is also very clearly describing the corruption going on in the medical sciences. He made simple statistical analysis to prove certain drugs like antidepressants causing side effects / death. His findings ignored, he was removed from his leading position from Cochrane Collaboration. Videos: https://youtube.com/watch?v=dozpAshvtsA https://youtube.com/watch?v=dozpAshvtsA https://youtube.com/watch?v=UVHSpQ9PbSs https://youtube.com/watch?v=UVHSpQ9PbSs https://youtube.com/watch?v=G2mFHHWyTrc https://youtube.com/watch?v=G2mFHHWyTrc

209 points by ajna91 7y ago

> The sales and marketing departments in most companies are more powerful than their R&D departments. Yes, this is correct. > Hence, the design, conduct, reporting, and dissemination of this clinical evidence becomes an advertisement tool. This does not follow well logically. The design of clinical trials still belongs to R&D and not Marketing/Sales departments. Even if Marketing/Sales want to target for a specific outcome, there is nothing saying that this particular outcome will be met until you actually try it clinically. And specific outcomes are usually desirable outcomes: for example, showing that your drug is non just superior to placebo, but superior to the current best-in-class treatment. Or measuring patient-related outcomes that are tied to economic value (for example, making a patient recover faster has tangible societal benefits - they can come back to work, be an active family member, etc...) is also relevant these days. So, there is not a strong link between "pushing for an outcome that you can advertise" and "this outcome being clinically meaningless". If it were meaningless, FDAs and other regulatory bodies would not even accept it during the clinical trial design review phase. > As for basic research Basic research should be the responsibility of academics, not companies. And they do. But not many academics focus on "checking whether what we have been using for 20 years really works", there's too much priority on new clinical targets.

Alex3917 7y ago
> This does not follow well logically. Instead of trying to reason from first principles about whether or not what he's saying is true, maybe instead go on Amazon and buy one of the dozens of books on the subject.
- wallace_f 7y ago
  I don't know if what he is saying is true, but I do know from having worked in another industry that the idea of sales/marketing dictating product development is not necessarily illogical, even if at face value it is.
  - marcus_holmes 7y ago
    in a lot of industries, Sales/Marketing dictating product development is the default. Sales and Marketing folks get used to that kind of power... and then they transfer to a new industry.
RobertRoberts 7y ago
> If it were meaningless, FDAs and other regulatory bodies would not even accept it during the clinical trial design review phase. Go look up how many corporate stooges are in the place in the FDA over the years. A good starting point is the history of Aspartame, I'd link some articles, but I suspect the sources would be shot down and evidence ignored. The people at the FDA were complicit in its approval, despite scientific evidence showing it was dangerous.
- abrichr 7y ago
  I would appreciate links to articles on the subject, if you have them handy!
  - Alex3917 7y ago
    The tl;dr is that the same guys who fabricated the evidence about Saddam Hussein having WMDs also fabricated evidence for aspartame being safe. https://www.huffpost.com/entry/donald-rumsfeld-and-the-s_b_805581 https://www.huffpost.com/entry/donald-rumsfeld-and-the-s_b_8...
    - deleted 7y ago
      [deleted]
    - ben_w 7y ago
      Did Donald Rumsfeld also convince the European Union, who have their own interests and a similar sized economy, that it was safe? https://www.efsa.europa.eu/en/press/news/131210 https://www.efsa.europa.eu/en/press/news/131210 or the “more than 90 countries worldwide” [1] that the Canadian government refers to on their own website? https://www.canada.ca/en/health-canada/services/food-nutrition/food-safety/food-additives/sugar-substitutes/aspartame-artificial-sweeteners.html https://www.canada.ca/en/health-canada/services/food-nutriti... There are certainly cases where I don’t trust USA regulations. This just isn’t one of them. [1] number includes EU and US, but that still leaves more than sixty others.
      - 7y ago
BurningFrog 7y ago
> This does not follow well logically I think it's an observation, not a philosophical argument.
- ekianjo 7y ago
  No, he said "Hence," right at the start of the sentence, as if it was a logical conclusion. It is not.
PeterStuer 7y ago
Sales and Marketing do control which outcomes are allowed to be published though. If you allow me selective disclosure of trials I can show you that my roulette strategy is infallible.
- viraptor 7y ago
  Hasn't this been fixed with the requirement to pre-register trials? https://chrisblattman.com/2016/03/01/13719/ https://chrisblattman.com/2016/03/01/13719/ I'm not sure if every org requires it now or not, but if not then it seems like a question of time only.
  - hannob 7y ago
    > Hasn't this been fixed with the requirement to pre-register trials? If it was that simple... Preregistration is a step in the right direction, but there are still plenty of ways to game the system, particularly as it's not properly enforced: http://compare-trials.org/ http://compare-trials.org/
    - ekianjo 7y ago
      But it is the necessary step to highlight the problem so that actions can be taken. Also, you can safely assume that the large majority of outcomes that were not published were:\ - negative - worse than negative (i.e. safety issues?) - not properly designed So if you compare the number of trials disclosed/not disclosed for each drug/each company that gives you a pretty good picture of who is doing what already. With that data, it's then a lot easier to approach regulators to do something about it.
      - tomp 7y ago
        My assumption is that most unpublished trials were "no effect". Be a drug company. Develop a "drug" that is really a placebo (has no effect). Do 10 trials. 8 are "no effect", 1 is positive (by chance), 1 is negative (by cance). Publish the positive drug. Market the drug as having positive effect. Sell the drug, make billions. It would take decades for enough other research (mostly "no effect") to accumulate before it becomes obvious that your drug actually has no effect.
wazoox 7y ago
Until recently, Big Pharma CEOs were either medical doctors of pharmacists. Marketing and finance were important, but not a driving force. Nowadays all of Big Pharma is headed by bean counters, marketing drones and finance guys, and are only interested in EBITDA. This is the main problem in all industries were science and engineering once prevailed.
- shadowtree 7y ago
  Really? Novartis - former head of R&D. Pfizer - PhD in Veterinary Medicine (he joined their AH division in 93) Astra Zeneca - Veterinary too Merck / MSD - Lawyer Roche - Economics & Law Gilead - Biology abbvie - never went to college, 30 years at company Amgen - Biology
nabla9 7y ago
> If it were meaningless, FDAs and other regulatory bodies would not even accept it during the clinical trial design review phase. Billions in R&D goes to getting molecular entities approved by FDA that are clinically meaningless. The medical R&D is done just to get a product that provides already existing functionality or already existing treatment with a slightly modified molecule that can be patented. It's like the joke about man who goes to patent office and says that he invented gunpowder. Patent clerk says that unfortunately gunpowder has already been invented. The man says: "But I invented more of it." Approval process is not looking if the new drug is meaningful improvement over existing drugs. "Inventing more of the same" is approved if it can be tweaked a little to be unique. Alternative is also true. There are potentially clinical treatments and medicines that receive no approvals because there is no profit motive to push them trough the expensive process of approval.
- galangalalgol 7y ago
  how do they get doctors to prescribe the patented versions if they aren't better?
  - mattmcknight 7y ago
    Sometimes they pay other companies not to make the unpatented drug: https://www.ftc.gov/news-events/media-resources/mergers-competition/pay-delay https://www.ftc.gov/news-events/media-resources/mergers-comp... More tricks here: https://hbr.org/2017/04/how-pharma-companies-game-the-system-to-keep-drugs-expensive https://hbr.org/2017/04/how-pharma-companies-game-the-system...
  - kyokpae 7y ago
    Sales reps bribe doctors, plain and simple. https://www.vice.com/en_us/article/pg7ez9/pharma-reps-are-basically-bribing-your-doctor-study-confirms https://www.vice.com/en_us/article/pg7ez9/pharma-reps-are-ba...
  - deleted 7y ago
    [deleted]
  - 7y ago
rjkennedy98 7y ago
> The design of clinical trials still belongs to R&D and not Marketing/Sales departments. This is FALSE. It's literally come out in court documents. https://www.motherjones.com/environment/2010/09/dan-markingson-drug-trial-astrazeneca/ https://www.motherjones.com/environment/2010/09/dan-markings... > In 1997, when Dr. Andrew Goudie, a psychopharmacologist at the University of Liverpool, asked AstraZeneca to fund a research study he was planning, a company official replied that “R&D is no longer responsible for Seroquel research—it is now the responsibility of Sales and Marketing.” The official also noted that funding decisions would depend on whether the study was likely to show a “competitive advantage for Seroquel.”
- ChrisLomont 7y ago
  >This is FALSE. How do you demonstrate that one case implies all cases?
  - rjkennedy98 7y ago
    This isn't a small company nor is it a minor drug. This is one of the largest drug companies running a trial on one of the most lucrative drugs in the world. And secondly, why should ANY drug trial be managed by a PR department? When you ask people to risk their lives to be in a drug trial (in the link above the subject died in the trial), why should it be to improve the PR of a drug for one company. The fact that this exists at all is a major warning sign. The indifference of people to this is frankly amazing to me (its even worse than the disinformation provided by the OP).

dbt00 7y ago

(2016)

outlace 7y ago

There wasn’t a lot of substance in this article, would have to read Dr. Ioannidis’ referenced papers to actually understand his point.

DoreenMichele 7y ago
Off the top of my head: He does meta studies -- aka studies of studies -- and finds that most studies are flawed, so most of the evidence we think we have isn't really there. It doesn't exactly make him popular because he calls into question a lot of existing supposedly "proven" medical science. He talks a lot about the biased assumptions that are the foundation of a lot of the framing of studies. If your baseline framing is sufficiently flawed, the entire study is junk. He thinks a lot of studies are, in fact, junk.
- Nasrudith 7y ago
  Stupid question - what makes us think the meta studies are themselves valid? Also wouldn't junk itself have many subclassifications as how it is junk? Say poor study data quality, wrong assumptions of wider applicability, wrong analysis or conclusions, etc.
  - DoreenMichele 7y ago
    what makes us think the meta studies are themselves valid? A. This is one guy, basically. I mean, I agree with him, generally, but not everyone does. B. It's easier to poke holes in someone else's thing than create a good study. So study A: "I wonder how fat you have to be to not fit through this door. Let's weigh people (instead of measuring them) to see how fat you have to be not fit through the door." Study B: "It's a great question, but it's badly done. Weighing them won't work. Let's just interview people and write down their self reported height!" Meta analyst: "Y'all are both wrong and stupid, geez." Scientists A and B: "Let's take the meta-analyst out back and beat him up!" "Sounds good to me!"
  - icelancer 7y ago
    >> Stupid question - what makes us think the meta studies are themselves valid? This is not a stupid question. Many meta studies are done by people with their own frameworks of deciding what is and isn't a legitimate approach to studying something. Unsurprisingly, their findings are overwhelmingly negative due to their own publication bias. Meta studies are pretty worthless in my opinion. Primary studies in all fields should be forced to be published with open data, and any study that received - directly or indirectly - more than $1 from the public taxpayer should be forced to publish Open Access. Lay bare all the raw data and let the software developers and data analysts get after it, not so-called expert meta reviewers.

nordsieck 7y ago

It looks like he gave a 30 minute talk [0] on this paper. ___ 0. https://www.youtube.com/watch?v=N63skNtYaJw https://www.youtube.com/watch?v=N63skNtYaJw

acqq 7y ago
Yes, the aricle we comment here was posted March 2016. The video is from April same year. Also June 2016 "John Ioannidis - Why most clinical research is not useful" (University of Oxford, Centre for Evidence-Based Medicine) https://www.youtube.com/watch?v=Uok-7NPFn4k https://www.youtube.com/watch?v=Uok-7NPFn4k The paper: https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002049 https://journals.plos.org/plosmedicine/article?id=10.1371/jo...

pessimizer 7y ago

> JI: A systematic review that combines biased pieces of evidence may unfortunately give another seal of authority to that biased evidence. Basically the same problem as CDOs.

wallace_f 7y ago
CDOs and the subprime mortgage crisis have some analogues to pre-Enlightenment darkness in human intellectual history. The idea that authoritative figures can use maths to give junk debt the midas touch is basically financial alchemy. There was no sound reasoning behind it. It was literally just "maths is really powerful, and these people are the Smartest Guys in the Room, which is not really that much different from "these guys are really Holy, and also the Smartest Guys in the Room" in previous generations of humanity. Just market it with Math, and you can defraud the most powerful and prosperous society in history.

newcrobuzon 7y ago

It is a sad reality that some of the regulatory agencies have been captured by the industry groups. This is especially concerning in healthcare and pharma industry. Take for example Vioxx and Opioid issues: Merck Manipulated the Science about the Drug Vioxx - https://www.ucsusa.org/manipulating-science-about-drug-vioxx https://www.ucsusa.org/manipulating-science-about-drug-vioxx Purdue infiltrated WHO, manipulated opioid policies to boost sales, report finds - https://arstechnica.com/science/2019/05/world-health-organization-parroted-purdues-deceptive-opioid-claims-report-says/ https://arstechnica.com/science/2019/05/world-health-organiz... Yet it is still surprising to see that on HN in general anyone questioning safety of pharmaceutical products get downvoted into oblivion.

Madmallard 7y ago
It's way worse than that. Fluoroquinolone antibiotics are well known in research for being seriously dangerous yet they were prescribed like candy for over a decade. So many people develop disabling symptoms overtime that end up not being linked to the drug because dna adduction and mtdna depletion won't necessarily show up quickly. The drugs have likely disabled hundreds of thousands at this point by Dr Bennett's estimations at UNC. There's something like 10,000 reports and those are assumed at being less than 10% and possibly 1% of total incidents.

EGreg 7y ago

There is a much bigger problem in the methodology, which is why meta studies won’t be the last word on it either https://slatestarcodex.com/2014/04/28/the-control-group-is-out-of-control/ https://slatestarcodex.com/2014/04/28/the-control-group-is-o...

nkurz 7y ago

Here's the underlying paper that the article is about: Evidence-based medicine has been hijacked: a report to David Sackett (2016) http://sci-hub.tw/10.1016/j.jclinepi.2016.02.012 http://sci-hub.tw/10.1016/j.jclinepi.2016.02.012

jshowa3 7y ago

After reading the interview, it sounds like a lot of scare mongering about bias without actually identifying specific things that are wrong with specific examples. Sure, it's nice to be looking for these issues, but people obviously have different opinions on what biases produce poor results and if biases are actually necessary and helpful.

qrbLPHiKpiux 7y ago

Case-in-point: Perdue Pharma, Oxycontin

cneurotic 7y ago

Dubious evidence in medical studies is a big problem. But I'd argue that it's only the core of the issue. The second layer out — and I think the more influential layer — is the JOURNALISM surrounding these medical studies. Even if the underlying science is sound, a lot of outlets and reporters routinely (and sometimes willfully) get the facts wrong. I attended a talk with Retraction Watch's Ivan Oransky, where he spelled out how easy it is for well-intentioned journalists to exaggerate, over-generalize, or misunderstand what a study says. It was eye-opening. And a little frightening.

lez 7y ago

He is a spiritual brother of Dr. Peter Gøtzsche, who is also very clearly describing the corruption going on in the medical sciences. He made simple statistical analysis to prove certain drugs like antidepressants causing side effects / death. His findings ignored, he was removed from his leading position from Cochrane Collaboration. Videos: https://youtube.com/watch?v=dozpAshvtsA https://youtube.com/watch?v=dozpAshvtsA https://youtube.com/watch?v=UVHSpQ9PbSs https://youtube.com/watch?v=UVHSpQ9PbSs https://youtube.com/watch?v=G2mFHHWyTrc https://youtube.com/watch?v=G2mFHHWyTrc