4 ms·

I agree. One scientist does a study, finds a thing. Another group attempts to repeat the study (as one does in science) and finds different results. Sounds l

by bphogan 11y ago

I agree. One scientist does a study, finds a thing. Another group attempts to repeat the study (as one does in science) and finds different results. Sounds like science to me.

detaro 11y ago
But presenting the result of the first study with large effort and bombast as great, newly discovered truth that will help the public solve their problems is not.
semi-extrinsic 11y ago
No. It's bad science. I'd go as far as to say it's fucking crappy science. Which is what TFA is saying. I mean, look at the figure in TFA where they show the original study effect size plus/minus two standard deviations, and the huge error bars (at 95% confidence) are just barely excluding the null hypothesis. With a sample size of 21 people. That is so pitiful even the Mythbusters would've said "We need to do a bigger experiment." But this Harvard professor instead said "Hooray, a significant result!" and went and did TED talks and books and all sorts of publicity stuff. Scott Aaronson recently quipped "(...) there was much discussion around the discovery that most psychology studies fail to replicate. I'd long assumed as much, but apparently this was big news in psychology!" The observation in the hard sciences that psychology has a big problem goes at least back to Feynman's "cargo cult" speech in '74. The fact that it's taken them forty years to catch up to this fact speaks volumes about the field.
- bphogan 11y ago
  Presenting things that are not peer reviewed is generally considered bad science, is it not? This kind of "science" happens here on HN every day and nobody bats an eye. "We used XYZ framework and it's the best thing ever!" Then tons of people use XYZ framework. Blog posts. Videos. Fad happens. Fad is over because "Why I'm switching to ABC framework." With tons of benchmarks of course. My point is that this happens all over the place. Someone makes a "discovery", markets the hell out of it with great speeches, bravado, etc. It gets coverage. It "grows legs" if you will. People repeat it and spread it. The "science" part I was referring to was the part where someone else attempts and fails to repeat the initial findings.
  - semi-extrinsic 11y ago
    > Presenting things that are not peer reviewed is generally considered bad science, is it not? Not really. Most conference presentations present stuff that's not (yet) peer reviewed. A Master's thesis is not peer reviewed. Etc. > This kind of "science" happens here on HN every day and nobody bats an eye. Sure. And bad cooking happens every day in the kitchens of many homes in the US. But when bad cooking happens in the kitchen of a Michelin star restaurant, and the cook still serves it up as gourmet food, it's a really big problem.

by bphogan 11y ago

I agree. One scientist does a study, finds a thing. Another group attempts to repeat the study (as one does in science) and finds different results. Sounds like science to me.

detaro 11y ago

But presenting the result of the first study with large effort and bombast as great, newly discovered truth that will help the public solve their problems is not.

semi-extrinsic 11y ago

No. It's bad science. I'd go as far as to say it's fucking crappy science. Which is what TFA is saying. I mean, look at the figure in TFA where they show the original study effect size plus/minus two standard deviations, and the huge error bars (at 95% confidence) are just barely excluding the null hypothesis. With a sample size of 21 people. That is so pitiful even the Mythbusters would've said "We need to do a bigger experiment." But this Harvard professor instead said "Hooray, a significant result!" and went and did TED talks and books and all sorts of publicity stuff. Scott Aaronson recently quipped "(...) there was much discussion around the discovery that most psychology studies fail to replicate. I'd long assumed as much, but apparently this was big news in psychology!" The observation in the hard sciences that psychology has a big problem goes at least back to Feynman's "cargo cult" speech in '74. The fact that it's taken them forty years to catch up to this fact speaks volumes about the field.

bphogan 11y ago
Presenting things that are not peer reviewed is generally considered bad science, is it not? This kind of "science" happens here on HN every day and nobody bats an eye. "We used XYZ framework and it's the best thing ever!" Then tons of people use XYZ framework. Blog posts. Videos. Fad happens. Fad is over because "Why I'm switching to ABC framework." With tons of benchmarks of course. My point is that this happens all over the place. Someone makes a "discovery", markets the hell out of it with great speeches, bravado, etc. It gets coverage. It "grows legs" if you will. People repeat it and spread it. The "science" part I was referring to was the part where someone else attempts and fails to repeat the initial findings.
- semi-extrinsic 11y ago
  > Presenting things that are not peer reviewed is generally considered bad science, is it not? Not really. Most conference presentations present stuff that's not (yet) peer reviewed. A Master's thesis is not peer reviewed. Etc. > This kind of "science" happens here on HN every day and nobody bats an eye. Sure. And bad cooking happens every day in the kitchens of many homes in the US. But when bad cooking happens in the kitchen of a Michelin star restaurant, and the cook still serves it up as gourmet food, it's a really big problem.