4 ms·

The Truth Wears Off (2010)

72 points by helloworld 9y ago

klenwell 9y ago
Interesting that Jonah Lehrer is the author of this. Here's a more recent New York Times article about him that might be title, "Does the Stigma Wear Off?": https://www.nytimes.com/2016/07/12/books/a-fraud-jonah-lehrer-says-his-remorse-is-real.html https://www.nytimes.com/2016/07/12/books/a-fraud-jonah-lehre... I don't think this article was affected by his disgrace and I still like a lot of what he's written. Still, in light of Lehrer's own subsequent history, it's hard not to read a certain ironic biographical significance into the final paragraph.
- hudibras 9y ago
  A later Radiolab episode on the same topic that he contributed to was found to have errors, and WNYC issued two corrections. https://www.wnyc.org/people/jonah-lehrer/ https://www.wnyc.org/people/jonah-lehrer/
hudibras 9y ago
By Jonah Lehrer, later fired for plagiarism and fabricating quotes. A Wired.com review of articles that he wrote for them said "Lehrer has a cavalier attitude about truth and falsehood. This shows not only in his attitude toward quotations but in some of the other details of his writing. And a journalist who repeatedly fails to correct errors when they're pointed out is, in my opinion, exhibiting reckless disregard for the truth. It is thus my opinion that Lehrer plagiarized others' work, published inaccurate quotations, printed narrative details that were factually incorrect, and failed to address errors when they were pointed out."
- bitumen 9y ago
  I don’t think it’s ad hominem if it’s all completely true, and relevant to the subject at hand. “Consider the source” is different from, “dismiss this, because of the source.” It’s an article that makes a good point, but it’s also good to realize that the author is not reliable, so it would pay to do further research, check claims made, and quotes cited.
- thaumaturgy 9y ago
  It's fine to point out that a controversial piece was written by someone that might not be impartial or, in this case, has had some ethical issues. However, other people contacted the scientists interviewed by Lehrer and asked for their opinions on what he wrote, and they generally agreed that it was accurate: http://www.lastwordonnothing.com/2012/11/05/jonah-lehrer-nature-of-truth/ http://www.lastwordonnothing.com/2012/11/05/jonah-lehrer-nat...
anon1253 9y ago
It was more or less the basis of my PhD in epidemiology that I dropped out of: https://joelkuiper.eu/academic-publishing https://joelkuiper.eu/academic-publishing (Most academic publishing isn't academic (and how to deal with it))
draw_down 9y ago
https://thelastpsychiatrist.com/2011/02/the_decline_effect_is_stupid.html https://thelastpsychiatrist.com/2011/02/the_decline_effect_i...
hammock 9y ago
Ignoring all the ad hominem about Jonah Lehrer here -- I recall a number of other discussions on HN about how the scientific method and science are different, how something is broken, about the irreproducibility of studies, about reversion to the mean, etc. Don't be distracted by who authored this piece.
- ubernostrum 9y ago
  If a particular person has been demonstrated in the past to be an unreliable source, it is not fallacious to say that they have been demonstrated in the past to be an unreliable source. Not every comment about a person is an example of the ad hominem fallacy.
  - darkerside 9y ago
    Well, ad hominem is certainly a logical fallacy, and discrediting the article by discrediting the author is a clear example of it. However, logic is not the only basis in which to challenge an article. I agree the information is relevant, and I think it's worth people understanding that logical arguments are not the only ones that carry water.
    - ubernostrum 9y ago
      Casting doubt on an article by pointing out the documented, verifiable unreliability of its author or primary source is not an example of the ad hominem fallacy. HN has an unfortunate love of the fallacy fallacy.
    - darkerside 9y ago
      I'm not arguing that impugning the author's credibility is fallacious here. Only that it's logically fallacious. Doesn't make it wrong, but a worthy distinction nonetheless.
- conistonwater 9y ago
  I also don't think it's ad hominem to mention his reputation (in this case, because he earned that reputation himself), but I want to mention that this piece completely mangles its description of the whole reproducibility crisis [1] and you would have to look elsewhere for a genuine discussion of the issues. This article isn't it. [1] The key example I think is this: he talks about the "decline effect", that you do experiments and find effects that later disappear ("When the experiments are done, we still have to choose what to believe."), but completely misses the point that when a low-power experiment finds an effect (with a correspondingly large type-M error), it is not the effect that declines, the effect never existed in the first place, it is the experiment that was done poorly and should be discarded. It would be the most boring banal statement possible to say that scientific ideas can only be found by well-conducted experiments, but he doesn't do even that.
avsteele 9y ago
Feynman mentions the Rhine example in his cargo cult science lecture (http://www.lockhaven.edu/~dsimanek/cargocul.htm http://www.lockhaven.edu/~dsimanek/cargocul.htm) I think an even more interesting example, one that shows greater subtlety of these effects, is the trend in experimental measurements taken when performing the Millikan Oil drop experiment (https://en.wikipedia.org/wiki/Oil_drop_experiment https://en.wikipedia.org/wiki/Oil_drop_experiment). This is also mentioned in Feynman's lecture. I had a plot I can't find now of the historical measures. It quite funny how the measurements trend slowly from Millikan's value to the true one over time.
darkerside 9y ago
All of the effects described in the article, despite the mystical tone, can be explained by regression to the mean, publication of outlying results, and confirmation bias in future publishings.
bkohlmann 9y ago
In my last year of business school, I took a class called Consumer behavior. It was easily one of my favorite courses and the professor was incredible. But one day she wanted to prove a psychological insight about rating Coffee - namely that in blind taste tests, people can’t distinguish between $5/lb coffee and $100/lb coffee (she literally bought all types). We set up a blind tasting line and all rated the drinks after a sip or two. Then we submitted an o line poll. Her mistake came in actually showing the results real time - they showed that we COULD tell the difference, even if subtly. She went on to explain, without looking at the screen, that we had just proven the theory right. Now, it’s true that we could’ve been the “outlier” group that randomly aligned with the the ratings. But it shows that if our group had been the first to be tested all those year ago, the researcher may have stopped - who reports on what you expect?

72 points by helloworld 9y ago

Interesting that Jonah Lehrer is the author of this. Here's a more recent New York Times article about him that might be title, "Does the Stigma Wear Off?": https://www.nytimes.com/2016/07/12/books/a-fraud-jonah-lehrer-says-his-remorse-is-real.html https://www.nytimes.com/2016/07/12/books/a-fraud-jonah-lehre... I don't think this article was affected by his disgrace and I still like a lot of what he's written. Still, in light of Lehrer's own subsequent history, it's hard not to read a certain ironic biographical significance into the final paragraph.

hudibras 9y ago
A later Radiolab episode on the same topic that he contributed to was found to have errors, and WNYC issued two corrections. https://www.wnyc.org/people/jonah-lehrer/ https://www.wnyc.org/people/jonah-lehrer/

hudibras 9y ago

By Jonah Lehrer, later fired for plagiarism and fabricating quotes. A Wired.com review of articles that he wrote for them said "Lehrer has a cavalier attitude about truth and falsehood. This shows not only in his attitude toward quotations but in some of the other details of his writing. And a journalist who repeatedly fails to correct errors when they're pointed out is, in my opinion, exhibiting reckless disregard for the truth. It is thus my opinion that Lehrer plagiarized others' work, published inaccurate quotations, printed narrative details that were factually incorrect, and failed to address errors when they were pointed out."

bitumen 9y ago
I don’t think it’s ad hominem if it’s all completely true, and relevant to the subject at hand. “Consider the source” is different from, “dismiss this, because of the source.” It’s an article that makes a good point, but it’s also good to realize that the author is not reliable, so it would pay to do further research, check claims made, and quotes cited.
thaumaturgy 9y ago
It's fine to point out that a controversial piece was written by someone that might not be impartial or, in this case, has had some ethical issues. However, other people contacted the scientists interviewed by Lehrer and asked for their opinions on what he wrote, and they generally agreed that it was accurate: http://www.lastwordonnothing.com/2012/11/05/jonah-lehrer-nature-of-truth/ http://www.lastwordonnothing.com/2012/11/05/jonah-lehrer-nat...

anon1253 9y ago

It was more or less the basis of my PhD in epidemiology that I dropped out of: https://joelkuiper.eu/academic-publishing https://joelkuiper.eu/academic-publishing (Most academic publishing isn't academic (and how to deal with it))

draw_down 9y ago

https://thelastpsychiatrist.com/2011/02/the_decline_effect_is_stupid.html https://thelastpsychiatrist.com/2011/02/the_decline_effect_i...

hammock 9y ago

Ignoring all the ad hominem about Jonah Lehrer here -- I recall a number of other discussions on HN about how the scientific method and science are different, how something is broken, about the irreproducibility of studies, about reversion to the mean, etc. Don't be distracted by who authored this piece.

ubernostrum 9y ago
If a particular person has been demonstrated in the past to be an unreliable source, it is not fallacious to say that they have been demonstrated in the past to be an unreliable source. Not every comment about a person is an example of the ad hominem fallacy.
- darkerside 9y ago
  Well, ad hominem is certainly a logical fallacy, and discrediting the article by discrediting the author is a clear example of it. However, logic is not the only basis in which to challenge an article. I agree the information is relevant, and I think it's worth people understanding that logical arguments are not the only ones that carry water.
  - ubernostrum 9y ago
    Casting doubt on an article by pointing out the documented, verifiable unreliability of its author or primary source is not an example of the ad hominem fallacy. HN has an unfortunate love of the fallacy fallacy.
  - darkerside 9y ago
    I'm not arguing that impugning the author's credibility is fallacious here. Only that it's logically fallacious. Doesn't make it wrong, but a worthy distinction nonetheless.
conistonwater 9y ago
I also don't think it's ad hominem to mention his reputation (in this case, because he earned that reputation himself), but I want to mention that this piece completely mangles its description of the whole reproducibility crisis [1] and you would have to look elsewhere for a genuine discussion of the issues. This article isn't it. [1] The key example I think is this: he talks about the "decline effect", that you do experiments and find effects that later disappear ("When the experiments are done, we still have to choose what to believe."), but completely misses the point that when a low-power experiment finds an effect (with a correspondingly large type-M error), it is not the effect that declines, the effect never existed in the first place, it is the experiment that was done poorly and should be discarded. It would be the most boring banal statement possible to say that scientific ideas can only be found by well-conducted experiments, but he doesn't do even that.

avsteele 9y ago

Feynman mentions the Rhine example in his cargo cult science lecture (http://www.lockhaven.edu/~dsimanek/cargocul.htm http://www.lockhaven.edu/~dsimanek/cargocul.htm) I think an even more interesting example, one that shows greater subtlety of these effects, is the trend in experimental measurements taken when performing the Millikan Oil drop experiment (https://en.wikipedia.org/wiki/Oil_drop_experiment https://en.wikipedia.org/wiki/Oil_drop_experiment). This is also mentioned in Feynman's lecture. I had a plot I can't find now of the historical measures. It quite funny how the measurements trend slowly from Millikan's value to the true one over time.

darkerside 9y ago

All of the effects described in the article, despite the mystical tone, can be explained by regression to the mean, publication of outlying results, and confirmation bias in future publishings.

bkohlmann 9y ago

In my last year of business school, I took a class called Consumer behavior. It was easily one of my favorite courses and the professor was incredible. But one day she wanted to prove a psychological insight about rating Coffee - namely that in blind taste tests, people can’t distinguish between $5/lb coffee and $100/lb coffee (she literally bought all types). We set up a blind tasting line and all rated the drinks after a sip or two. Then we submitted an o line poll. Her mistake came in actually showing the results real time - they showed that we COULD tell the difference, even if subtly. She went on to explain, without looking at the screen, that we had just proven the theory right. Now, it’s true that we could’ve been the “outlier” group that randomly aligned with the the ratings. But it shows that if our group had been the first to be tested all those year ago, the researcher may have stopped - who reports on what you expect?