5 ms·

There wasn’t a lot of substance in this article, would have to read Dr. Ioannidis’ referenced papers to actually understand his point.

DoreenMichele 7y ago
Off the top of my head: He does meta studies -- aka studies of studies -- and finds that most studies are flawed, so most of the evidence we think we have isn't really there. It doesn't exactly make him popular because he calls into question a lot of existing supposedly "proven" medical science. He talks a lot about the biased assumptions that are the foundation of a lot of the framing of studies. If your baseline framing is sufficiently flawed, the entire study is junk. He thinks a lot of studies are, in fact, junk.
- Nasrudith 7y ago
  Stupid question - what makes us think the meta studies are themselves valid? Also wouldn't junk itself have many subclassifications as how it is junk? Say poor study data quality, wrong assumptions of wider applicability, wrong analysis or conclusions, etc.
  - DoreenMichele 7y ago
    what makes us think the meta studies are themselves valid? A. This is one guy, basically. I mean, I agree with him, generally, but not everyone does. B. It's easier to poke holes in someone else's thing than create a good study. So study A: "I wonder how fat you have to be to not fit through this door. Let's weigh people (instead of measuring them) to see how fat you have to be not fit through the door." Study B: "It's a great question, but it's badly done. Weighing them won't work. Let's just interview people and write down their self reported height!" Meta analyst: "Y'all are both wrong and stupid, geez." Scientists A and B: "Let's take the meta-analyst out back and beat him up!" "Sounds good to me!"
  - icelancer 7y ago
    >> Stupid question - what makes us think the meta studies are themselves valid? This is not a stupid question. Many meta studies are done by people with their own frameworks of deciding what is and isn't a legitimate approach to studying something. Unsurprisingly, their findings are overwhelmingly negative due to their own publication bias. Meta studies are pretty worthless in my opinion. Primary studies in all fields should be forced to be published with open data, and any study that received - directly or indirectly - more than $1 from the public taxpayer should be forced to publish Open Access. Lay bare all the raw data and let the software developers and data analysts get after it, not so-called expert meta reviewers.
    - Jedd 7y ago
      > Lay bare all the raw data and let the software developers ... Really can't tell if you're being sarcastic here, even in the context of HN. > Meta studies are pretty worthless in my opinion. Primary studies in all fields should be forced to be published with open data ... And until that happens meta studies are both worthwhile and one of the most effective ways to correlate and sensibly interpret the existing published data sets.
  - gilbetron 7y ago
    I'm sure the people doing the meta studies would be delighted to have other, independent people attempt to replicate their studies, but as they did it, and in other ways to verify their findings. Unlike many of the groups that did the studies this meta is studying, who often don't want any group do anything except confirm their findings because of the money and status behind their studies.
dfeliej 7y ago
I read the J Clin Epi paper being discussed when it first came out and it expressed my feelings and thoughts very well, as a professor. The crux of the problem is this: we as a society value science, and the epitome of this in healthcare has been the goal of evidence based medicine (EBM) or evidence based practice (EBP). The idea seems simple enough, that healthcare decisions and procedures should be based on scientific evidence. This clashes, however, with the reality that much of research in the biomedical sciences is unreplicable or completely invalid for other reasons. The reasons for this are complex but you have things like p-hacking, selective reporting of results, confounds, and so forth and so on. Think "how to lie with science" instead of "how to lie with statistics." As a result, EBM gets hijacked by those who are trying to manipulate; appeals to science becomes a form of fallacious argument, sort of a version of appeal to authority error, where one appeals to a scientific finding because it's peer reviewed etc. without invoking the underlying integrity of the research per se. Then you have the unscrupulous making money off of bad science, and authority figures (in medical care and administration) cudgeling people for not using EBM/EBP when what is really happening is people are recognizing it as bad science. Sometimes no science is better than bad science--or sometimes, science means rejecting peer-reviewed research, even with all the right surface features. The paper is part of a slightly personal exchange between Ioannidis and someone he looks up to but is also close to, so it has a kind of poignant conversational style. It's not so much a scientific paper as it is a personal discussion.

DoreenMichele 7y ago

Off the top of my head: He does meta studies -- aka studies of studies -- and finds that most studies are flawed, so most of the evidence we think we have isn't really there. It doesn't exactly make him popular because he calls into question a lot of existing supposedly "proven" medical science. He talks a lot about the biased assumptions that are the foundation of a lot of the framing of studies. If your baseline framing is sufficiently flawed, the entire study is junk. He thinks a lot of studies are, in fact, junk.

Nasrudith 7y ago
Stupid question - what makes us think the meta studies are themselves valid? Also wouldn't junk itself have many subclassifications as how it is junk? Say poor study data quality, wrong assumptions of wider applicability, wrong analysis or conclusions, etc.
- DoreenMichele 7y ago
  what makes us think the meta studies are themselves valid? A. This is one guy, basically. I mean, I agree with him, generally, but not everyone does. B. It's easier to poke holes in someone else's thing than create a good study. So study A: "I wonder how fat you have to be to not fit through this door. Let's weigh people (instead of measuring them) to see how fat you have to be not fit through the door." Study B: "It's a great question, but it's badly done. Weighing them won't work. Let's just interview people and write down their self reported height!" Meta analyst: "Y'all are both wrong and stupid, geez." Scientists A and B: "Let's take the meta-analyst out back and beat him up!" "Sounds good to me!"
- icelancer 7y ago
  >> Stupid question - what makes us think the meta studies are themselves valid? This is not a stupid question. Many meta studies are done by people with their own frameworks of deciding what is and isn't a legitimate approach to studying something. Unsurprisingly, their findings are overwhelmingly negative due to their own publication bias. Meta studies are pretty worthless in my opinion. Primary studies in all fields should be forced to be published with open data, and any study that received - directly or indirectly - more than $1 from the public taxpayer should be forced to publish Open Access. Lay bare all the raw data and let the software developers and data analysts get after it, not so-called expert meta reviewers.
  - Jedd 7y ago
    > Lay bare all the raw data and let the software developers ... Really can't tell if you're being sarcastic here, even in the context of HN. > Meta studies are pretty worthless in my opinion. Primary studies in all fields should be forced to be published with open data ... And until that happens meta studies are both worthwhile and one of the most effective ways to correlate and sensibly interpret the existing published data sets.
- gilbetron 7y ago
  I'm sure the people doing the meta studies would be delighted to have other, independent people attempt to replicate their studies, but as they did it, and in other ways to verify their findings. Unlike many of the groups that did the studies this meta is studying, who often don't want any group do anything except confirm their findings because of the money and status behind their studies.

dfeliej 7y ago

I read the J Clin Epi paper being discussed when it first came out and it expressed my feelings and thoughts very well, as a professor. The crux of the problem is this: we as a society value science, and the epitome of this in healthcare has been the goal of evidence based medicine (EBM) or evidence based practice (EBP). The idea seems simple enough, that healthcare decisions and procedures should be based on scientific evidence. This clashes, however, with the reality that much of research in the biomedical sciences is unreplicable or completely invalid for other reasons. The reasons for this are complex but you have things like p-hacking, selective reporting of results, confounds, and so forth and so on. Think "how to lie with science" instead of "how to lie with statistics." As a result, EBM gets hijacked by those who are trying to manipulate; appeals to science becomes a form of fallacious argument, sort of a version of appeal to authority error, where one appeals to a scientific finding because it's peer reviewed etc. without invoking the underlying integrity of the research per se. Then you have the unscrupulous making money off of bad science, and authority figures (in medical care and administration) cudgeling people for not using EBM/EBP when what is really happening is people are recognizing it as bad science. Sometimes no science is better than bad science--or sometimes, science means rejecting peer-reviewed research, even with all the right surface features. The paper is part of a slightly personal exchange between Ioannidis and someone he looks up to but is also close to, so it has a kind of poignant conversational style. It's not so much a scientific paper as it is a personal discussion.