6 ms·

Clinical trials != scientific studies in general. Motherjones is doing a great disservice implying equivalence.

function_seven 8y ago
Aren't they equivalent in their potential to allow post-hoc data mining? Whether it's a clinical trial, or a psychological study with n participants, or a look at how changes in x soil conditions affects the y tree. Anything where data samples are taken and statistically analyzed is prone to p-hacking and after-the-fact hypothesis changes. This example happens to be for a narrow set of studies: clinical trials for cardiovascular drugs/supplements. The take away from that graph applies broadly.
- L_Rahman 8y ago
  And in fact the replication crisis in psychology suggests that the practice goes beyond just clinical trials.
  - gaius 8y ago
    The replication crisis in every field of science. Two-thirds of reported results cannot be reproduced.
    - dwd 8y ago
      If you haven't read it already, this article on the placebo effect a few days ago was interesting in how we on one hand use it as a baseline for measuring the effectiveness of drug treatments but actually can vary wildly between individuals. It's not surprising that results are so inconsistent. https://news.ycombinator.com/item?id=18398342 https://news.ycombinator.com/item?id=18398342
geggam 8y ago
Explain that to the layman in a way the entire Internet will understand. Until you can ... they are for all intents and purposes.
- gmfawcett 8y ago
  Hardly equivalent "for all intents and purposes." The purposes of the layman are nothing like those of the research community.
  - geggam 8y ago
    If you put it in the public domain it is your responsibility to ensure it is digestible by the audience you publish it to. So yes... public posted information should be layman ready
    - radus 8y ago
      Do you apply this reasoning to articles or comments discussing kernel development? Some topics have higher irreducible complexity than others. More importantly, I can make whatever I want public, and I can do so with whatever audience I have in mind.
      - geggam 8y ago
        Interesting. So inciting panic online is fine but not in a theater ?
        metaphor 8y ago
        Your remark finds difficulty in satisfying clear and present danger.
        geggam 8y ago
        Vaccine studies
        metaphor 8y ago
        ...are of a certain irreducible complexity which undergo a process of peer review prior to publication in scientific journals, and rightfully presume a minimum prerequisite background for proper interpretation? I'm sure the topic and its meta are interesting to the professional audience in which they were intended for.
Udik 8y ago
If anything, clinical trials should be more reliable than the average scientific study: after all they are experiments, with well defined numbers (sample size, effects measurement, controlled conditions). Compare with all non-experimental science- including for example most environmental and climate science, where, if experiments are made at all, the results are wildly extrapolated and generalized.
- dwighttk 8y ago
  If humans are the equivalent of atoms, or molecules, or rockets, or planets or whatever other scientific thing you are studying.

Aren't they equivalent in their potential to allow post-hoc data mining? Whether it's a clinical trial, or a psychological study with n participants, or a look at how changes in x soil conditions affects the y tree. Anything where data samples are taken and statistically analyzed is prone to p-hacking and after-the-fact hypothesis changes. This example happens to be for a narrow set of studies: clinical trials for cardiovascular drugs/supplements. The take away from that graph applies broadly.

L_Rahman 8y ago
And in fact the replication crisis in psychology suggests that the practice goes beyond just clinical trials.
- gaius 8y ago
  The replication crisis in every field of science. Two-thirds of reported results cannot be reproduced.
  - dwd 8y ago
    If you haven't read it already, this article on the placebo effect a few days ago was interesting in how we on one hand use it as a baseline for measuring the effectiveness of drug treatments but actually can vary wildly between individuals. It's not surprising that results are so inconsistent. https://news.ycombinator.com/item?id=18398342 https://news.ycombinator.com/item?id=18398342

geggam 8y ago

Explain that to the layman in a way the entire Internet will understand. Until you can ... they are for all intents and purposes.

gmfawcett 8y ago
Hardly equivalent "for all intents and purposes." The purposes of the layman are nothing like those of the research community.
- geggam 8y ago
  If you put it in the public domain it is your responsibility to ensure it is digestible by the audience you publish it to. So yes... public posted information should be layman ready
  - radus 8y ago
    Do you apply this reasoning to articles or comments discussing kernel development? Some topics have higher irreducible complexity than others. More importantly, I can make whatever I want public, and I can do so with whatever audience I have in mind.
    - geggam 8y ago
      Interesting. So inciting panic online is fine but not in a theater ?
      - metaphor 8y ago
        Your remark finds difficulty in satisfying clear and present danger.
        geggam 8y ago
        Vaccine studies
        metaphor 8y ago
        ...are of a certain irreducible complexity which undergo a process of peer review prior to publication in scientific journals, and rightfully presume a minimum prerequisite background for proper interpretation? I'm sure the topic and its meta are interesting to the professional audience in which they were intended for.

Udik 8y ago

If anything, clinical trials should be more reliable than the average scientific study: after all they are experiments, with well defined numbers (sample size, effects measurement, controlled conditions). Compare with all non-experimental science- including for example most environmental and climate science, where, if experiments are made at all, the results are wildly extrapolated and generalized.

dwighttk 8y ago
If humans are the equivalent of atoms, or molecules, or rockets, or planets or whatever other scientific thing you are studying.