3 ms·

Reading the article, the various comments and the linked "Why We Sleep" criticism, I feel really disillusioned. Honestly, there will always be people who write

Reading the article, the various comments and the linked "Why We Sleep" criticism, I feel really disillusioned. Honestly, there will always be people who write stuff they have no idea about, or write stuff that are factually wrong. But where does that leave me, and perhaps many other layman readers? I, as a layman in psychology/neuroscience/<insert field>, do not have the academic qualifications or the time to go through and understand the latest/greatest scientific literature of the respective fields, nor am I equipped with the necessary "instincts" to tell the doubtful or suspicious claims apart. Most of the time, all I want is condensed information which is well, from the scientific literature, yet put in one place in a way that laymen in the fields have access to the subject. And to me atleast, books are the best way. But then I find the books I thought were right and did not doubt, had factual errors, or were written by someone who has a possibility of not knowing what they are saying... This is sad. So let me ask the community: what heuristics do you employ when trying to decide whether you can trust a book on what it says or not? Especially if it's not your field of expertise?

ncmncm 7y ago
If it presents psychology as a well-understood and rigorous field, and the author as an authority, it's bunk. Nobody is an expert in psychology, and most of the foundational experiments in the field have been found not to be reproducible.
- smukherjee19 7y ago
  Hm, I was not aware psychology results were not reproducible to this extent. Maybe better to take stuff with a pinch of salt everytime then. Thanks~
jonex 7y ago
Number of citations per paragraph is a pretty good indicator in my experience. Typically this is high (close to 1) for good text books in psychology and low (1 per page or less) in bad ones. Citations isn't everything though, many authors misrepresent the content to fit the narrative and in some areas it's not as important. For instance in math, arguments can be made based on very few axioms and you can evaluate the correctness yourself. In general it helps to have a skeptical mindset, try to figure out the intent of the author by how they treat facts, do they present inconsistent arguments or gloss over relevant fallacies in the claims? Then their intent is probably more about presenting a good argument than correct representation of facts. Another thing to be watchful for is claims that "make sense" or are "obvious", it's easy to fall in the trap of ignoring conflicting evidence when the theory sounds good.
- smukherjee19 7y ago
  Ah, citations and "obvious" claims, and to approach everything with a skeptical mindset, I see! Thanks!
acqq 7y ago
> So let me ask the community: what heuristics do you employ when trying to decide whether you can trust a book on what it says or not? Especially if it's not your field of expertise? If one hasn't studied at the university and read the real scientific papers and learned to compared the good ones to the bad ones in one's own field (and everywhere there are enough bad ones too) my impression one has no chance to be good at recognizing the difference. And even once one learns to recognize the difference in his own area, he's typically has too much confidence to be able to repeat that in other fields. So it's really hard. The best bet is really learning to discover what the limits of he fields are, who are real experts in the field, then read what they write etc. It's never simple. But if you're just watching a TV where something is presented as a "debate" learn to recognize what's "behind" the two only people who are shown as two "sides": behind a real scientists could be the whole scientific community, effectively all the experts in their field, and behind the guy who "sounds" better in the show can be simply... money from some specific interest group which finances an "actor" who looks good. What's however certain is that it was always so: there were always people successfully "selling" as the nonsense as their "knowledge." I'm at the moment reading 1979 book: https://en.wikipedia.org/wiki/Broca%27s_Brain https://en.wikipedia.org/wiki/Broca%27s_Brain and there the same tricks in selling untruths as today were already (unsurprisingly) actual. Earlier, Asimov also attempted to debunk a lot of intentional distortions of achieved human knowledge in popular media. Or even earlier, Mencken 1925: https://web.archive.org/web/20010219062704/http://www.positiveatheism.org/hist/menck01.htm https://web.archive.org/web/20010219062704/http://www.positi... "The inferior man's reasons for hating knowledge are not hard to discern. He hates it because it is complex -- because it puts an unbearable burden upon his meager capacity for taking in ideas. Thus his search is always for short cuts. All superstitions are such short cuts. Their aim is to make the unintelligible simple, and even obvious. So on what seem to be higher levels. No man who has not had a long and arduous education can understand even the most elementary concepts of modern pathology. But even a hind at the plow can grasp the theory of chiropractic in two lessons. Hence the vast popularity of chiropractic among the submerged -- and of osteopathy, Christian Science and other such quackeries with it. They are idiotic, but they are simple -- and every man prefers what he can understand to what puzzles and dismays him." In the article we discuss, it's also a "shortcut" the topic of the book: four colors for people, and "everything is simple, even obvious." The most important part from the article we comment to is therefore: "Despite the use of colours, it turned out that the “Surrounded by …” books were not based on Myers-Brigg. Instead, they built on another personality theory, the so-called DiSC model. The most noteworthy outcome of a search through the academic literature on this model was that, despite the fact that the test had been around for fifty years, there was in principle no research on whether or not it worked." From the books about pseudo-scientific claims presented in media as "real" science I also recommend "The War On Science" by Shawn Otto: https://milkweed.org/book/the-war-on-science https://milkweed.org/book/the-war-on-science and "Merchants of Doubt" by Naomi Oreskes, Erik M. Conway: https://www.bloomsbury.com/uk/merchants-of-doubt-9781596916104/ https://www.bloomsbury.com/uk/merchants-of-doubt-97815969161... In short, there are definitely enough people who earn money by "selling" falsehoods. It was never an accident.

ncmncm 7y ago

If it presents psychology as a well-understood and rigorous field, and the author as an authority, it's bunk. Nobody is an expert in psychology, and most of the foundational experiments in the field have been found not to be reproducible.

smukherjee19 7y ago
Hm, I was not aware psychology results were not reproducible to this extent. Maybe better to take stuff with a pinch of salt everytime then. Thanks~

jonex 7y ago

Number of citations per paragraph is a pretty good indicator in my experience. Typically this is high (close to 1) for good text books in psychology and low (1 per page or less) in bad ones. Citations isn't everything though, many authors misrepresent the content to fit the narrative and in some areas it's not as important. For instance in math, arguments can be made based on very few axioms and you can evaluate the correctness yourself. In general it helps to have a skeptical mindset, try to figure out the intent of the author by how they treat facts, do they present inconsistent arguments or gloss over relevant fallacies in the claims? Then their intent is probably more about presenting a good argument than correct representation of facts. Another thing to be watchful for is claims that "make sense" or are "obvious", it's easy to fall in the trap of ignoring conflicting evidence when the theory sounds good.

smukherjee19 7y ago
Ah, citations and "obvious" claims, and to approach everything with a skeptical mindset, I see! Thanks!

acqq 7y ago

> So let me ask the community: what heuristics do you employ when trying to decide whether you can trust a book on what it says or not? Especially if it's not your field of expertise? If one hasn't studied at the university and read the real scientific papers and learned to compared the good ones to the bad ones in one's own field (and everywhere there are enough bad ones too) my impression one has no chance to be good at recognizing the difference. And even once one learns to recognize the difference in his own area, he's typically has too much confidence to be able to repeat that in other fields. So it's really hard. The best bet is really learning to discover what the limits of he fields are, who are real experts in the field, then read what they write etc. It's never simple. But if you're just watching a TV where something is presented as a "debate" learn to recognize what's "behind" the two only people who are shown as two "sides": behind a real scientists could be the whole scientific community, effectively all the experts in their field, and behind the guy who "sounds" better in the show can be simply... money from some specific interest group which finances an "actor" who looks good. What's however certain is that it was always so: there were always people successfully "selling" as the nonsense as their "knowledge." I'm at the moment reading 1979 book: https://en.wikipedia.org/wiki/Broca%27s_Brain https://en.wikipedia.org/wiki/Broca%27s_Brain and there the same tricks in selling untruths as today were already (unsurprisingly) actual. Earlier, Asimov also attempted to debunk a lot of intentional distortions of achieved human knowledge in popular media. Or even earlier, Mencken 1925: https://web.archive.org/web/20010219062704/http://www.positiveatheism.org/hist/menck01.htm https://web.archive.org/web/20010219062704/http://www.positi... "The inferior man's reasons for hating knowledge are not hard to discern. He hates it because it is complex -- because it puts an unbearable burden upon his meager capacity for taking in ideas. Thus his search is always for short cuts. All superstitions are such short cuts. Their aim is to make the unintelligible simple, and even obvious. So on what seem to be higher levels. No man who has not had a long and arduous education can understand even the most elementary concepts of modern pathology. But even a hind at the plow can grasp the theory of chiropractic in two lessons. Hence the vast popularity of chiropractic among the submerged -- and of osteopathy, Christian Science and other such quackeries with it. They are idiotic, but they are simple -- and every man prefers what he can understand to what puzzles and dismays him." In the article we discuss, it's also a "shortcut" the topic of the book: four colors for people, and "everything is simple, even obvious." The most important part from the article we comment to is therefore: "Despite the use of colours, it turned out that the “Surrounded by …” books were not based on Myers-Brigg. Instead, they built on another personality theory, the so-called DiSC model. The most noteworthy outcome of a search through the academic literature on this model was that, despite the fact that the test had been around for fifty years, there was in principle no research on whether or not it worked." From the books about pseudo-scientific claims presented in media as "real" science I also recommend "The War On Science" by Shawn Otto: https://milkweed.org/book/the-war-on-science https://milkweed.org/book/the-war-on-science and "Merchants of Doubt" by Naomi Oreskes, Erik M. Conway: https://www.bloomsbury.com/uk/merchants-of-doubt-9781596916104/ https://www.bloomsbury.com/uk/merchants-of-doubt-97815969161... In short, there are definitely enough people who earn money by "selling" falsehoods. It was never an accident.