22 ms·

How a middle aged CS major debunked a classic positive psychology finding (2013)

432 points by jvanderbot 5y ago

dekhn 5y ago
Falsifying large parts of the extant literature is trivial.
- dang 5y ago
  "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - dekhn 5y ago
    https://blog.scienceexchange.com/2012/08/the-reproducibility-initiative/ https://blog.scienceexchange.com/2012/08/the-reproducibility... “In the last year, problems in reproducing academic research have drawn a lot of public attention, particularly in the context of translating research into medical advances. Recent studies indicate that up to 70% of research from academic labs cannot be reproduced, representing an enormous waste of money and effort,” said Dr. Elizabeth Iorns, Science Exchange’s co-founder and CEO. “In my experience as a researcher, I found that the problem lay primarily in the lack of incentives and opportunities for validation—the Reproducibility Initiative directly tackles these missing pieces.” https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 https://journals.plos.org/plosmedicine/article?id=10.1371/jo... "Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias." but I didn't really need to add that to make my comment any better.
    - dang 5y ago
      I think it's pretty shallow still, even with that, because it's just a generic dismissal. Most readers are already familiar enough with this phenomenon that your comment amounts to a knee-jerk reaction—literally the most obvious thing one might say in the closest generic category. That's a marker of a bad HN comment, because it points to shallower, more generic discussion. Good comments are reflective, not reflexive: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=reflective%20reflex%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor.... The more generic a discussion gets, the less interesting it is: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=generic%20discussion%20by%3Adang%20-flamewars%20-flamewar&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
      - dekhn 5y ago
        I mean, I could have made the point that this is basically just the base rate fallacy: because 90% of papers are wrong, finding wrong papers is easy. You can just say all papers are wrong, with 90% accuracy. Or, I could have made the point that social sciences as a whole are woefully non-quantitative and the real lesson is: ignore just about anything you hear about psychology studies in the media, because most of them come from innumeracy and invalid experimental techniques. Another great point: when somebody, say an "average Nick", finds that a paper does contain an invalidating flaw, it can be challenging to get the attention of the establishment. Sometimes that's because you're an average Nick with no cred in the field, sometimes it's that the journal doesn't want to admit that one of their board has been publishing total crap for the last 20 years in said journal. Instead, I summed up all those points in a single comment which was in no way shallow, but rather, from decades of experience with our broken scientific system.
        dang 5y ago
        If you have decades of experience with the scientific system that's wonderful—but for a HN good comment you need to show that, not tell it—and you certainly need to make comments that are distinguishable from obvious internet dismissals. If a comment pattern-matches to that (as yours did), people are certain to interpret them that way, and not just moderators. People with significant knowledge in their heads often assume that when they communicate a short message M, the message is substantive even if it doesn't include much information. In one's head, this is precisely true: M automatically connects with the things you already know and thus has a lot of depth to it. But all that is lost when sending M to other people—especially in stateless internet comments. The rest of us don't have your experience and knowledge, and don't have access to what's in your head. All we see is the information you explicitly put into M. In this way, the same M can be both substantive (to the sender) and unsubstantive (to the receiver), because the sender has all the implicit information and the receiver only has the explicit. The 'received M' matters much more than the 'sent M', though, first because there are millions of receivers, and second because it's the received M that determines how people react.
        5y ago
Jeff_Brown 5y ago
- jvanderbot 5y ago
  He traced the math behind it back to a computational fluid dynamics paper, which the debunkees had used equations from with no context. He cold emailed professors asking for feedback of his organized findings, got back a 3000 word article based on it, and with help from a mathematician and psychologist in smoothing the writeup, and ultimately got it published in a top journal and the offending results recognized as flawed. The general crusade continues. The result in question was the legendary 3-1 positive to negative tipping point between flourishing individuals and languishing ones.
  - PaulDavisThe1st 5y ago
    Brown did not write the "3000 word critique". From TFA: > Sokal, “came back with my draft one day,” Brown says, “and had written 3,000 words.”
    - jvanderbot 5y ago
      Ah! My reading comprehension strikes again. Edited, thank you.
300bps 5y ago
Wikipedia has a good tl;dr: https://en.wikipedia.org/wiki/Critical_positivity_ratio https://en.wikipedia.org/wiki/Critical_positivity_ratio Later in 2013, the critical positivity ratio aroused the skepticism of Nick Brown, a graduate student in applied positive psychology, who felt that the paper's mathematical claims underlying the critical positivity ratio were fundamentally flawed.[6] Brown collaborated with physicist Alan Sokal and psychologist Harris Friedman on a re-analysis of the paper's data. They argued that Fredrickson and Losada's paper contained "numerous fundamental conceptual and mathematical errors", as did Losada's earlier work on positive psychology, which completely invalidated their claims.[7] In their response, Hämäläinen and colleagues argued that there were no fundamental errors in the mathematics itself, but the problems relate to the interpretation and justification of the use of the model.[8] Losada declined to respond to the criticism, indicating that he was too busy running his consulting business.[6] Fredrickson wrote a response in which she conceded that the mathematical aspects of the critical positivity ratio were "questionable" and that she had "neither the expertise nor the insight" to defend them, but she maintained that the empirical evidence was solid.[9] Brown and colleagues, whose response was published the next year, maintain that there is no evidence for the critical positivity ratio.[10] In response, American Psychologist formally retracted the mathematical modeling elements of Fredrickson & Losada's paper, including the specific critical positivity ratio of 2.9013, as invalid.
- Joeboy 5y ago
  Since TFA is quite long and has quite an annoying layout, I tried Reader View in Firefox, and it gave me a completely different article. Weird. Thanks for posting the tl;dr.
  - CRConrad 5y ago
    Yup, same here. I think it was because it's one of those "infinite scrolling" pages, where when you've scrolled down to the end of the article another begins (and the URL in the address bar has also magically changed). Yet another reason to hate that shit.
colechristensen 5y ago
I got bored. This is an article style I really don’t like, the interesting information is buried towards the end after thousands of mostly irrelevant bits of narrative which aren’t worth remembering. I can guess the ending though, technical guy sees psychologists appropriating a well defined mathematical structure, doing some hand waving, and claiming that it applies to your everyday life in a shallow way. Guy is hero. Yes, psychology is absolutely full of bad science and bad scientists. Lots of fields are, but particularly psychology.
- BeefWellington 5y ago
  > Yes, psychology is absolutely full of bad science and bad scientists. Lots of fields are, but particularly psychology. Social sciences in general are probably harder than other fields for the simple reason that it requires other people's time and effort. A psychologist friend of mine once dropped the line "we know a lot about how university students view the world" when discussing the replication crisis. I think that while simplifying things quite a bit, it is very apt. Edited for coffee
  - tgv 5y ago
    If the latter were true, I'd be surprised. It's obviously a quip about using students as subjects for credits, but the average psychology study reveals very little about students. After all, it's how they got convinced psi exists (Bem, 2011).
    - mywittyname 5y ago
      Economics is where you really glean insights into university students. For example, they enjoy free food exponentially more than they like very cheap food. Also, they great at playing games for money.
  - colechristensen 5y ago
    It's more than that, people who aren't really interested in science or the math required to do good science are attracted to the "social sciences" in a much higher degree. People who are unqualified to do analyses are publishing papers and those papers are getting published with the approval of their peers who are similarly likely to be unqualified.
Scarblac 5y ago
This is a good quote: > “One could describe a team’s interactions as ‘sparky’ and conﬁdently predict that their emotions would be subject to the same laws that govern the dielectric breakdown of air under the inﬂuence of an electric ﬁeld. Alternatively, the interactions of a team of researchers whose journal articles are characterized by ‘smoke and mirrors’ could be modeled using the physics of airborne particulate combustion residues, combined in some way with classical optics"
- derbOac 5y ago
  To be honest I think this story is overplayed a bit. I've heard of that tipping point idea before, or the positive to negative ratio thing, but I think most in the field didn't take it more seriously than an observation that couples, groups, etc that function well are more positive and supportive. It's good to call this stuff out but part of the reason no one else did is because outside of a certain subsubfield no one took it that seriously.
  - robwwilliams 5y ago
    You may well be right, but this is too casual an approach toward what purports to be science in a nominally good journal. Modern science is amazingly well insulated from “in-house” criticism after the review process. As a result crap can flourish. We need more data/analyst wasps.
  - munificent 5y ago
    I strongly disagree with this take. The whole point of science is to acquire knowledge that eventually gets communicated outside of the scientific body producing it. If the papers never leave the ivory tower, or never lead to other papers that do, it's just navel gazing. When a scientific community has a chunk of their work that everyone in the tower knows is bullshit but never gets corrected, that metadata doesn't leave the ivory tower. What happens instead is that papers like this flutter out the window, get picked up by journalists who don't have access to tribal knowlege, and then get spread wide and far. The information appears legitimate because it did come from a trusted scientific body. It just had the necessary context that it's actually bullshit left off. Every scientific community has an obligation to ensure that the papers they publish and advocate actually mean what they say they mean. If nothing else, because not doing that ultimately tarnishes the brand and undermines the trustworthiness of that community.
oneepic 5y ago
How can you call the guy a CS major? He was a grad student in psychology, right?
- jvanderbot 5y ago
  "The married father of two had graduated from Cambridge University in 1981 with a degree in computer science, and spent most of his career as an IT networks operator at an international organization in Strasbourg, France." Pretty early in the article.
  - ineedasername 5y ago
    Yes, but the context was also with that person as a current student, which was not in CS, and his CS background didn't have much bearing on what he discovered. It would have been just as relevant to say "Father of two debunked..." A better title would have been "New Student in APS debunks core finding in the field."
    - kwhitefoot 5y ago
      But that wouldn't attract so many HNers to comment.
    - jvanderbot 5y ago
      HN readers are predominately CS/IT folks, and would probably relate better to that aspect of his person than others. That's certainly why I read the article. Should I emphasize that he lives in the UK? "UK citizen debunks positive psychology" Or should I emphasize the statehood of the person debunked? "UK citizen debunks US professor with one neat trick" Just trying to summarize the relevant parts of the story without clickbait -- which the article did pretty well I thought.
      - PaulDavisThe1st 5y ago
        Going the full clickbait: "You'll never believe how this aging British empty-nested IT guy took down a US professor with one neat trick!"
duxup 5y ago
It seems weird this theory’s backbone was just some random person who contacted someone and said something like “hey I have math that says you’re right” and it just takes off from there. Until some other rando checks it out of curiosity…
- consumer451 5y ago
  I think you have just described the open source code review model. edit: and science in general?
  - duxup 5y ago
    -PR approved “Rando said it looks good “-
- wheelinsupial 5y ago
  Do you mean this quote: "A Chilean business-consultant named Marcial Losada, 'who had begun to dabble in what had become his passion: mathematical modeling of group behavior,' sent her an out-of-the-blue email." The article seems to ignore it, but Losada had a PhD in social and organizational psychology from the U of Michigan [1]. [1] http://losada.socialpsychology.org/ http://losada.socialpsychology.org/
jvanderbot 5y ago
TL;DR Nick Brown was a 50y/o CS/IT employee who wanted to cross-train to human resources and wanted "evidence-based stuff". He encountered lots of fantastic positive psychology claims that were being accepted wholesale as "quantitative". He traced the math behind a critical one back to a computational fluid dynamics paper, which the debunkees had used equations from with no context. (allegedly because of the use of "Flow" in positive psychology!) He cold emailed professors asking for feedback, wrote a 3000 word critique, got help from a mathematician and psychologist in smoothing the writeup, and ultimately got it published in a top journal and the offending results recognized as flawed. The general crusade continues along with co-crusaders Sokal and Friedman. The result in question was the legendary 3-1 positive to negative tipping point between flourishing individuals and languishing ones.
- readthenotes1 5y ago
  TL;DR: Most psych research results are garbage. Some make millions for the authors. Here's yet another example.
  - AlwaysRock 5y ago
    Some make millions for Authors who take those papers and cite them in popular psych books. The actual authors of the papers are rarely, if ever, making millions.
    - frostburg 5y ago
      The 9 million dollars in grants that the author received probably were also completely wasted, since someone that would randomly apply CFD equations to people is unlikely to have published worthwhile research.
      - CRConrad 5y ago
        Remains to be tallied how many of her research assistants on those projects went on to have careers in the field... Careers which are now more or less tainted by the association. Even harder to tally: How many of those deserve to be more tainted because they saw it for what it was but didn't speak up. (Yeah, I know, hard to do for a lowly assistant.)
dang 5y ago
I remember past threads about this but can't find them. Anyone? Edit: the list so far - thanks caaqil and setgree! I feel like there were still others, if anyone can find them... Nick Brown Smelled Bull (2013) - https://news.ycombinator.com/item?id=9791302 https://news.ycombinator.com/item?id=9791302 - June 2015 (1 comment) The British amateur who debunked the mathematics of happiness (2014) - https://news.ycombinator.com/item?id=16515822 https://news.ycombinator.com/item?id=16515822 - March 2018 (213 comments) The British amateur who debunked the mathematics of happiness (2014) - https://news.ycombinator.com/item?id=9085639 https://news.ycombinator.com/item?id=9085639 - Feb 2015 (15 comments)
- setgree 5y ago
  inter alia: https://news.ycombinator.com/item?id=16515822 https://news.ycombinator.com/item?id=16515822
- caaqil 5y ago
  https://news.ycombinator.com/item?id=9085639 https://news.ycombinator.com/item?id=9085639
- froh 5y ago
  And it's a long detailed read about the genesis of this paper: The Complex Dynamics of Wishful Thinking https://pubmed.ncbi.nlm.nih.gov/23855896/ https://pubmed.ncbi.nlm.nih.gov/23855896/
renewiltord 5y ago
His Twitter account: https://twitter.com/sTeamTraen https://twitter.com/sTeamTraen Seems he does a lot of this data policing stuff. Neat. The funny thing is that a lot of people go around saying "yeah, engineers go around thinking they can just walk into field X and study it without context". Yup. They do, and they can. Another article https://www.theguardian.com/science/2014/jan/19/mathematics-of-happiness-debunked-nick-brown https://www.theguardian.com/science/2014/jan/19/mathematics-...
- derbOac 5y ago
  Sometimes they can and sometimes they can't. Works like that for all kinds of crossovers, and probably should happen more often than it does. I can think of plenty of times CS has benefitted from psychology too, or is reinventing the wheel.
  - renewiltord 5y ago
    Haha, yes. I'm just a big fan of people trying. I'm also a big fan of anti-gatekeeping.
    - derbOac 5y ago
      I surely agree with you on that.
- dogleash 5y ago
  >that a lot of people go around saying "yeah, engineers go around thinking they can just walk into field X and study it without context". That's sorta the point of a degree in the sciences rather than going to a vocational school, right? It's amusing that people find it arrogant. It is that they're showing a level of competence we're all claiming and maybe only pretending to have? Is it "swim in your own lane" union-style isolation? Is it sneering at nerds for not having the good sense to shut it? Is it inflated egos that still think the things they learned in undergrad were hard?
- at-fates-hands 5y ago
  >> The funny thing is that a lot of people go around saying "yeah, engineers go around thinking they can just walk into field X and study it without context" I have a degree in Anthropology and transitioned into UX Research. 90%+ of what you do is observation in both fields. Most of the engineers and math people I've met are horrendously bad at observing behavior. In some fields, its just a different set of skills engineers are not capable of applying very well.
setgree 5y ago
I worked with Nick a few times in a previous job and he was a reviewer on a paper I wrote; he was really nice and professional, and his feedback was constructive and incisive. The movement he's part of is fundamentally disagreeable (in the 'big five' sense [0]), and some of its prominent characters, as you might expect, have reputations for being interpersonally difficult [1]. Collectively they've been labeled "data thugs" [2] and accused of "methodological terrorism" [3], which is why I think Nick's fundamental goodness is an especially valuable asset. We'd gain a ton of social value from certain reforms to the scientific research apparatus, and, IMO, being kind when delivering pointed critiques helps a difficult pill go down. That's different than pulling your punches; it just means staying focused on the thing you actually want to change. [0] https://en.wikipedia.org/wiki/Big_Five_personality_traits https://en.wikipedia.org/wiki/Big_Five_personality_traits [1] https://medium.com/@OmnesRes/the-dumbest-fucking-thing-ive-ever-read-8653e2409ec4 https://medium.com/@OmnesRes/the-dumbest-fucking-thing-ive-e... [2] https://www.science.org/content/article/meet-data-thugs-out-expose-shoddy-and-questionable-research https://www.science.org/content/article/meet-data-thugs-out-... [3] https://statmodeling.stat.columbia.edu/2016/09/21/what-has-happened-down-here-is-the-winds-have-changed/ https://statmodeling.stat.columbia.edu/2016/09/21/what-has-h...
- whimsicalism 5y ago
  I'm unsure what your second citation is for, they don't use the word "data thugs" but the author certainly seems annoying (although likely correct about the points they're making as far as I read). e: the parent comment has now been edited for clarity, keeping my comment for posterity
  - setgree 5y ago
    The piece is written by a member of the posse [0] who can be, as I think the above quoted piece demonstrates, somewhat abrasive [0] https://medium.com/@OmnesRes/the-circle-of-data-thug-life-815289b96c95 https://medium.com/@OmnesRes/the-circle-of-data-thug-life-81...
    - MegaButts 5y ago
okl 5y ago
I think they should have sent in their paper somewhere other than APA. Don't expect courtesy in consequential confrontations. I think similar about responsible disclosure sometimes. If a company played dirty in the past why bother with accomodating them.
- duxup 5y ago
  I think they wanted to give the APA a chance to make the right call / improve, and it also seems they found the peer process valuable and they wanted to get that feedback. Seems like it worked out.
  - CRConrad 5y ago
    > Seems like it worked out. Yeah, kind of... Mainly by exposing how reluctant the APA was to make the right call / improve.
    - duxup 5y ago
      They did the thing and got feedback, worked as far as I can tell.
MetaWhirledPeas 5y ago
I see this as an argument for adding more STEM to every degree plan. There's unavoidable crossover, and people need intuition to properly question unfounded claims.
- wbsss4412 5y ago
  Unfortunately I’m not sure that STEM in and of itself is the antidote. It’s very easy to just routinely apply methods without properly reasoning about the underlying assumptions.
  - MetaWhirledPeas 5y ago
    The less exposure one has to these concepts the more tempting it becomes to skip the reasoning part.
    - wbsss4412 5y ago
      True, but there is an upper bound to the amount of exposure one can receive before they are simply a STEM major, rather than another major with more STEM exposure.
  - klabb3 5y ago
    Be careful what you wish for. Increasing the competence of fraudsters will simply make fraud harder to detect. Had the original theory simply included a "measured" coefficient, the debunking would have been much weaker and never made it into a gotcha story for bystanders to enjoy. This was sloppy from the outset. What we need isn't stronger technical skills, but strong self-correction mechanisms and incentive structures. (Ironically, the management consultant who orchestrated this whole ploy is in the field of group psychology - so perhaps it was performance art all along)
  - FredPret 5y ago
    But you shouldn’t be able to get a degree in a soft “science” without demonstrating that you can apply disciplined thinking by passing some proper STEM courses.
ChrisMarshallNY 5y ago
I've been watching Dopesick, and the part, where they are chasing down the "1%" study, is pretty damn sobering.
- klabb3 5y ago
  Citations are upvotes for academia.
  - eru 5y ago
    Perhaps more like links in the original Google PageRank?
  - mhh__ 5y ago
    Or downvotes, gotta cite it to say it's bullshit.
efsavage 5y ago
IANA data scientist, but I would hope that 4 decimal places in the context of psychology would trigger anyone's BS meter.
- anyfactor 5y ago
  I think there is inverse relation of how frequently a number is written everywhere with it's degree of decimal places. It is a convenience thing. Also if the number is generated on a computer and used there subsequently 4 decimal is plenty for social science and business research/purpose IMO.
  - klabb3 5y ago
    I think parent means the opposite, that 4 decimal places is really high precision in a field with lots of measurement bias and noise.
    - anyfactor 5y ago
      TIL. This is something new I learned. Can you tell me what to google to learn about this bias?
      - waterheater 5y ago
        https://en.wikipedia.org/wiki/Statistical_significance https://en.wikipedia.org/wiki/Statistical_significance When the Higgs boson was declared "discovered," the particle was found to exist with a five-sigma certainty. This brief article does a great job of explaining what that means: https://www.r-bloggers.com/2012/07/the-higgs-boson-5-sigma-and-the-concept-of-p-values/ https://www.r-bloggers.com/2012/07/the-higgs-boson-5-sigma-a... Physicists got to that point (very roughly) by constructing complex mathematical models and building large machines to test those models. In contrast, leading psychology models change with the seasons, and large-scale human experiments cannot be (ethically) conducted with the control as seen in physics. Because psychology experiments have substantial environmental variability, drawing robust conclusions in psychology is exceedingly difficult. This fact should mean that very-high-certainty findings in psychology studies must be subjected to the highest scrutiny.
photochemsyn 5y ago
Taking equations from physics out of context to bolster social science claims isn't unique to psychology, other social sciences like economics have done the same thing with little more justification. The rationale behind this approach is simple marketing, in that 'using more mathy stuff' will sell better to the general public than non-mathy hand-waving claims will. However, the practioners of this approach don't seem to understand that physical equations are constructed by analysis of empirical experimental and observational data on one hand, and theoretical mathematical development on the other, within limited scopes of applicability. Copying equations from one area of science to another willy-nilly like the article describes is just ridiculous - would cosmologists studying general relatively just plug in quantum mechanical equations in the hope of getting a 'mathy explanation'? Here's another example from 'mathematically sound neoclassical economics', which does the same with 19th century physical science equations: > "The strategy the economists used was as simple as it was absurd—they substituted economic variables for physical ones. Utility (a measure of economic well-being) took the place of energy; the sum of utility and expenditure replaced potential and kinetic energy. A number of well-known mathematicians and physicists told the economists that there was absolutely no basis for making these substitutions. But the economists ignored such criticisms and proceeded to claim that they had transformed their field of study into a rigorously mathematical scientific discipline." https://www.scientificamerican.com/article/the-economist-has-no-clothes/ https://www.scientificamerican.com/article/the-economist-has...
- munchler 5y ago
  Based on other people in the field whose responses were quoted at the end of the article, I suspect this phenomenon may still be occurring. Specifically: "David Pincus, a psychologist at Chapman University who specializes in the application of chaos theory to psychology" "Stephen Guastello, a psychologist specializing in nonlinear dynamics at Marquette University"
- notahacker 5y ago
  There are niche heterodox subfields of economics that literally are just a dubious physics analogy (thermoeconomics), but the source's argument that marginal utility theory was copying von Hemholtz is the sort of extraordinary claim that requires extraordinary evidence, not zero citations, baffling confusion of economist Francis Edgeworth with literary writer Maria Edgeworth and a lot of tacked on claims that are laughably false (Jevons certainly didn't believe that "natural resources are inexhaustible: he was actually initially famous for trying to predict when the coal supply would run out!). There are obvious reasons for studies of how money is spent to involve mathematics (though Menger managed to deduce marginal utility theory from first principles without equations) and it's unsurprising some of the equation forms superficially resemble some of the many simple equations of physics. There are much better criticisms of neoclassical economics and the application of neoclassical economics than accusing the early economists of plagiarising another field!
unixhero 5y ago
What is middle aged of any relevance???
deleted 5y ago
[deleted]
TameAntelope 5y ago
I think this is a fantastic example of how a) anyone can call bullshit on science if it doesn't make sense and b) there are more and less correct ways of calling bullshit on established science. You don't need to hop on a podcast or radio show in order to question the orthodoxy of a scientific claim, and in fact you shouldn't if you want to be taken seriously. Nick Brown would have hurt his case had he done that, and instead found a path that actually knocked down a faulty concept in a way that has lasting effects. This is the process working. This is all okay, and welcome, IMO, though possibly not by everyone involved.
- jvanderbot 5y ago
  I agree. The real shining light here is that a determined amateur was able to get free, volunteer help from inside the community to weather the publication process. Those who read the review / revision process in this article and were dismayed / convinced the process was flawed should realize this is precisely how review works in many fields. It takes a year to get a paper reviewed and revised, and a "25 page" (TFA) statement of revision is par. Not "all" will have the stomach for that, but once you are part of that process, it's normal.
  - horsawlarway 5y ago
    You consider having to threaten the reputation of an academic journal by emailing the CEO directly before you even make it to the review for a well written, concise, critical paper "part of that process"? Not even mentioning that it required the publicity and reputation of a well known figure (Sokal) to even open that door for them. If that's the "shining light" of this industry - I sure as fuck don't want to see the dirty alleyway.
    - jvanderbot 5y ago
      I didn't say that. I said the shining light was insiders helping. I also said the peer review process is challenging and that was an accurate representation. Two different things.
deleted 5y ago
[deleted]
phkahler 5y ago
It doesnt seem possible to have a fixed positivity ratio when considering research on extinction. The gist of extinction is that when training using positive feedback, the desired response will cease when the positive feedback does (extinction). In order to maintain the behavior the feedback has to essentially be randomized with a decreasing prevalence (positivity ratio).
j7ake 5y ago
A quote struck me as relevant during covid and work from home times: “because more good things will happen to you than bad on any given day, but nothing will happen to you if you just sit indoors.”
j7ake 5y ago
This misuse and abuse of methods will continue as long as you can make a big splash in the field. The technical correction of the misuse afterwards can then be tacked on to the end of your article later, with absolutely no recourse or penalty to your career.
roywiggins 5y ago
> In a valid fluid dynamics problem, the numbers plugged into the equation must correlate to the properties of the fluid being studied. But in attempting to draw an equivalence between the physical flow of liquids and the emotional “flow” of human beings, Losada had simply lifted the numbers that Lorenz used in 1963 to explain his method in the abstract, numbers used merely for illustrative purposes. Losada’s results, along with the pretty butterfly graphs Brown had been shown in class, were essentially meaningless. Incredible. So it wasn't just a matter of, say, applying an equation beyond its domain, or generating the inputs to the function in a sketchy way, or hacking the inputs to the function until they got an answer they liked, but literally copying example parameters from a 1963 paper. The "Complex Dynamics of Wishful Thinking" paper is very readable: https://physics.nyu.edu/faculty/sokal/complex_dynamics_final_clean.pdf https://physics.nyu.edu/faculty/sokal/complex_dynamics_final...
- FabHK 5y ago
  It is very readable, and it utterly nihilates Losada's papers. It is hard to believe that anyone could have taken them seriously, let alone let them get published. Sigh. Nice touch that Sokal helped debunking it. I wonder to which extent this taints the whole of "Positive Psychology", which I was moderately excited about.
at-fates-hands 5y ago
And through it all, the three met only once in person, when Friedman was in London to give an academic address. It was a day spent scribbling equations in chalk on blackboards and talking strategy – a debunkers’ bonhomie. I truly miss this about academics. Where you get a chance to sit down and think freely and pick someone else's brain in the pursuit of some hypothesis you're chasing down.
throwawaypt 5y ago
Anyone have suggestions of the best way to call BS on clinical studies and their marketing? A publicly traded company's products are all based on BS science. I've pointed it out to them but they are dead silent. Just one example is them comparing (in marketing) the active group of one study to the control group of another study performed in different countries years apart with different inclusions criteria massively different n and insanely different dropout rates. It just blows my mind this stuff happens. If anyone is interested here is the full writeup: https://www.sendbig.com/view-files?Id=c131d2fc-9a4a-0616-f772-2ebc7271efbb https://www.sendbig.com/view-files?Id=c131d2fc-9a4a-0616-f77...
- eru 5y ago
  You could try a shareholder lawsuit?
- toast0 5y ago
  Complain to the agencies that are/should regulate them. FTC coverse false/misleading advertisements generally, but maybe just FDA for mediciney things?
  - throwawaypt 5y ago
    Good idea. Just lodged a complaint with FDA and SEC. They won't provide updates but at least someone might read it. Will look into FTC.
deleted 5y ago
[deleted]
errcorrectcode 5y ago
Toxic positivity and toxic negativity are grating to some, myself included. The issue is that soft, squishy not-STEM "sciences" aren't all that evidence-based or experimentally-verifiable. That doesn't make them completely useless, it makes them prone to fungible, inconsistent guidelines rather than hypotheses, theories, or laws.
davidwritesbugs 5y ago
The whole "Cited 350 times" made me laugh. I used to think this type of number sounded impressive. Recently I just started to do a Masters and was told "1 citation for every hundred words", so dozens of papers need reading and it take for *ever* weeks to read just a small number. Then I recalled articles claiming most citations are never checked so, and you won't believe this, I skim read papers and if vaguely relevent inserted cursory comments from the paper and cited it. Did that go horribly wrong? Go on, guess. So 350? I so believe it.
- ALittleLight 5y ago
  That seems like it's own separate problem which is also bad.
  - davidwritesbugs 5y ago
    It was just a lazy metric to measure effort and engagement, and that relieved the lecturer of the need to read the paper himself to see if you'd absorbed it and analysed its relevance correctly. I can't say I'm unsympathetic - the guy had 50 students to mark. I'm guessing this is standard in academia but if they'd said "read and include relevant papers" I'd have done a good job on half a dozen and the lecturer probably wouldn't have been overburdened either.
- 8K832d7tNmiQ 5y ago
  Ah yes, the classic Goodhart's law [0] working as intended again. [0]:https://en.m.wikipedia.org/wiki/Goodhart%27s_law https://en.m.wikipedia.org/wiki/Goodhart%27s_law
  - mhh__ 5y ago
    It's more of a "it'll do"-ism from bored supervisors IMO, could be imposed upon them but also is an easy way to avoid extra being different.
- PeterisP 5y ago
  Things like your master thesis or anything like that simply aren't included in the citation count at all, generally no one cares how much students or general public are citing your work so that practice has no influence on the metric being used. As the article states, it was cited 350 times in academic journals, i.e. cited in 350 reviewed publications with novel scientific work written by her peers.
sampo 5y ago
> But in attempting to draw an equivalence between the physical flow of liquids and the emotional “flow” of human beings, Losada had simply lifted the numbers that Lorenz used in 1963 to explain his method in the abstract, numbers used merely for illustrative purposes. Losada’s results, along with the pretty butterfly graphs Brown had been shown in class, were essentially meaningless. It was a bit crazy in the late 1990s. Chaos theory had been popularized, and the social sciences copied fashionable buzzwords and buzzconcepts from chaos theory, and used them totally out of context. Maybe it was crazy in the 1920s, too? Einstein's theory of relativity had become popular, and sciences that had nothing to do with gravitation, copied the "everything is relative" slogan.
YaBomm 5y ago
mannerheim 5y ago
'I think the educational and psychological studies I mentioned are examples of what I would like to call Cargo Cult Science. In the South Seas there is a Cargo Cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas—he’s the controller—and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things Cargo Cult Science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.' - Richard Feynman, 'Cargo Cult Science' https://calteches.library.caltech.edu/51/2/CargoCult.htm https://calteches.library.caltech.edu/51/2/CargoCult.htm
georgecmu 5y ago
For those curious: The two nonsense papers are: Losada, Marcial. “The complex dynamics of high performance teams.” Mathematical and Computer Modelling 30 (1999): 179-192. | https://www.sciencedirect.com/science/article/pii/S0895717799001892?via%3Dihub https://www.sciencedirect.com/science/article/pii/S089571779... Fredrickson, B. L., & Losada, M. F. (2005). Positive Affect and the Complex Dynamics of Human Flourishing. American Psychologist, 60(7), 678–686. | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3126111/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3126111/ The debunking paper is Brown, N. J. L., Sokal, A. D., & Friedman, H. L. (2013). The complex dynamics of wishful thinking: The critical positivity ratio. American Psychologist, 68(9), 801–813. | https://arxiv.org/pdf/1307.7006.pdf https://arxiv.org/pdf/1307.7006.pdf
f154hfds 5y ago
Interesting read! The most disturbing part is not fraud scientists, I think there will always be a certain percentage of fraudsters in any field. The most disturbing part is the 300+ citations, scientists who based their work off of an obviously garbage paper. It seems inescapable that the reason why must have been complete mathematical incompetency across the entire field. Say a psychologist partners with perhaps a chemist and uses some organic chemistry to 'prove' some critical point in a paper. I could imagine brushing over the details and citing the paper when any chemist would see the fraud as 'obvious'. Such is the complexity of cross discipline science I guess, and the reality that the sum total of human knowledge is beyond any individual's ability to grasp. That being said, knowing that some papers are utter garbage I should do my due diligence by taking a humble pill and reaching out to my colleague in the chem department across the street to tell me what this critical chemistry is about before I include it in my work.
- justin66 5y ago
  > The most disturbing part is the 300+ citations, scientists who based their work off of an obviously garbage paper. > It seems inescapable that the reason why must have been complete mathematical incompetency across the entire field. I think you won't have to try very hard to find analogous failures to properly test the truth or falsehood of the literature's conventional wisdom in more "mathematically competent" fields.
fluorinerocket 5y ago
richardfey 5y ago
Here's the key passage (SPOILER ahead): > Losada had simply lifted the numbers that Lorenz used in 1963 to explain his method in the abstract, numbers used merely for illustrative purposes. Losada’s results, along with the pretty butterfly graphs Brown had been shown in class, were essentially meaningless.
monksy 5y ago
I can guarantee you there are a lot of people really upset with him. A lot of those kinds of studies are "this makes me feel good and this is the world I want to see" types. No consideration of the negative outcomes of bad info, bad practices, or even externalizes.
tomohawk 5y ago
You have to wonder how many people were harmed by this fraud. And that's exactly what this is. The original paper was a fraud. No honest mistake can explain it. The reviewers committed fraud by not seeking competent input when they lacked the competence to do the math. How many people under psychological care were given treatments and were harmed? How many companies wasted on money on this and should be able to seek compensation? IANAL, but this smells like litigation.
mlindner 5y ago
Psychology in general is full of hucksterism. My sister got her bachelor's in Pyschology and she also found a whole ton of nonsense in it. She says that there's been a split in psychology and the real science was getting done on the "Biopsychology, Cognition, and Neuroscience (BCN)" side of the split while the psychology side is getting further and further away from real science and more into the realm of the made-up.
- cwkoss 5y ago
  I did a psychology undergrad major in the late 00s. There was a shocking amount of bad science in the required courses. Lots of stuff that supported "common sense" but would be impossible to replicate, and the methodology didn't even align with the summary of why the study was important to learn about. Mountains of work built off imprecise self assessment tests. You can find a psych study to support just about any idea, and it seems like that's what many successful psychologists in academia have done. Big part of why I decided not to pursue it as a career.
3flp 5y ago
What I find sad about this is that the effort and skill required to debunk this nonsense is an order of magnitude higher than the effort and skill required to create it in the first place. Aka, the "Gish Gallop" and other analogues, observations and quotes about this effect, made many times.
ensan 5y ago
Reading the title, I expected the debunker to be a middle-aged many _studying_ CS.
mleonhard 5y ago
If there was a collaborative project to review dubious published research papers, which of you would be interested in contributing? I think there are four categories of papers to focus on: 1. Papers with obviously false claims. 2. Papers that omit information that is crucial for verifying their claims. 3. Dubious papers that introduced ideas that attracted a lot of attention and subsequent sketchy papers. 4. Dubious papers published by people who just started in PhD programs.
- mhh__ 5y ago
  Post-publish review is something that should probably be improved. It can't exactly be open to the public to contribute but equally the public should be able to see if scientists have (anonymous) serious doubts about a paper.
etempleton 5y ago
"Lies, damned lies, and statistics." Unfortunately a lot of academic papers throw hopelessly tortured sentences, esoteric words, and complex formulas at you in the hope that no one notices that the whole paper is built on a house of cards: https://www.nature.com/articles/d41586-021-02134-0 https://www.nature.com/articles/d41586-021-02134-0
vzaliva 5y ago
The paper mentioned in the article: https://physics.nyu.edu/faculty/sokal/complex_dynamics_final_clean.pdf https://physics.nyu.edu/faculty/sokal/complex_dynamics_final...
vzaliva 5y ago
This is a fascinating story of how some questionable research slipped through peer review safeguards of academic publishing and flourished. More importantly, it shows how the scientific community eventually caught it and debunked it. The original results were recalled. Sadly, one of the main perpetrators slipped into murky waters of "business consulting" and continue to sell this snake oil: " Dr. Marcial Losada, has a Ph.D. in Social and Organizational Psychology from the University of Michigan. Dr. Losada was invited to present his work at Cambridge University in England, Harvard Business School, Graduate Business School at the University of Michigan, Sloan School of Management at MIT, Kellogg School of Management at Northwestern University, Stanford Research Institute, and Institute for the Future in Menlo Park, California. He presented his findings on applications of nonlinear dynamics to team interaction and productivity at the prestigious Director’s Colloquium in Los Alamos National Laboratory. He has briefed Vice-President Al Gore and the president of MIT, Dr. Charles Vest, on the interaction dynamics of high performance teams." https://www.losadalineconsulting.net/our_team https://www.losadalineconsulting.net/our_team
isitmadeofglass 5y ago
I feel like the field and the journals in psychology have a history of being and currently are quite fragile. I have taken a couple of examples and done some simulations and come to the conclusion that the kinetic fragility index of the field is about the same as amorphous silicon dioxide most the time. And as anyone will tell you if your building a house of that material. It is vital that you ensure no one around is looking critically at it. So you should paint it with a brick pattern, and ensure nothing heavier than a feather is ever thrown towards it. In that regards it seems the journal in this case agrees, as they seriously resisted a scientific rebuttal of a piece until they realized they could not prevent it. Had they had integrity (which I believe we can model analogue to a high Young’s modulus, I haven’t yet finished simulations to give a precise number of how high it should be for the journal) then they would have been thrilled at the chance to adjust their building to not include the original material.
sTeamTraen 5y ago
Hi - I am the subject of this article. I don't know how to prove that other than tweeting a link to this post, which I will do right now (renewiltord already posted my Twitter account here https://news.ycombinator.com/item?id=30247543 https://news.ycombinator.com/item?id=30247543). Thanks for the kind words to those who posted them. For everyone else, feel free to AMA. Edit: Here is the link to the tweet that links to this comment: https://twitter.com/sTeamTraen/status/1491041770217811977?s=20&t=qdC3edEhiZs4g3G7Y-gagQ https://twitter.com/sTeamTraen/status/1491041770217811977?s=...

432 points by jvanderbot 5y ago

dekhn 5y ago

Falsifying large parts of the extant literature is trivial.

dang 5y ago
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- dekhn 5y ago
  https://blog.scienceexchange.com/2012/08/the-reproducibility-initiative/ https://blog.scienceexchange.com/2012/08/the-reproducibility... “In the last year, problems in reproducing academic research have drawn a lot of public attention, particularly in the context of translating research into medical advances. Recent studies indicate that up to 70% of research from academic labs cannot be reproduced, representing an enormous waste of money and effort,” said Dr. Elizabeth Iorns, Science Exchange’s co-founder and CEO. “In my experience as a researcher, I found that the problem lay primarily in the lack of incentives and opportunities for validation—the Reproducibility Initiative directly tackles these missing pieces.” https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 https://journals.plos.org/plosmedicine/article?id=10.1371/jo... "Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias." but I didn't really need to add that to make my comment any better.
  - dang 5y ago
    I think it's pretty shallow still, even with that, because it's just a generic dismissal. Most readers are already familiar enough with this phenomenon that your comment amounts to a knee-jerk reaction—literally the most obvious thing one might say in the closest generic category. That's a marker of a bad HN comment, because it points to shallower, more generic discussion. Good comments are reflective, not reflexive: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=reflective%20reflex%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor.... The more generic a discussion gets, the less interesting it is: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=generic%20discussion%20by%3Adang%20-flamewars%20-flamewar&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
    - dekhn 5y ago
      I mean, I could have made the point that this is basically just the base rate fallacy: because 90% of papers are wrong, finding wrong papers is easy. You can just say all papers are wrong, with 90% accuracy. Or, I could have made the point that social sciences as a whole are woefully non-quantitative and the real lesson is: ignore just about anything you hear about psychology studies in the media, because most of them come from innumeracy and invalid experimental techniques. Another great point: when somebody, say an "average Nick", finds that a paper does contain an invalidating flaw, it can be challenging to get the attention of the establishment. Sometimes that's because you're an average Nick with no cred in the field, sometimes it's that the journal doesn't want to admit that one of their board has been publishing total crap for the last 20 years in said journal. Instead, I summed up all those points in a single comment which was in no way shallow, but rather, from decades of experience with our broken scientific system.
      - dang 5y ago
        If you have decades of experience with the scientific system that's wonderful—but for a HN good comment you need to show that, not tell it—and you certainly need to make comments that are distinguishable from obvious internet dismissals. If a comment pattern-matches to that (as yours did), people are certain to interpret them that way, and not just moderators. People with significant knowledge in their heads often assume that when they communicate a short message M, the message is substantive even if it doesn't include much information. In one's head, this is precisely true: M automatically connects with the things you already know and thus has a lot of depth to it. But all that is lost when sending M to other people—especially in stateless internet comments. The rest of us don't have your experience and knowledge, and don't have access to what's in your head. All we see is the information you explicitly put into M. In this way, the same M can be both substantive (to the sender) and unsubstantive (to the receiver), because the sender has all the implicit information and the receiver only has the explicit. The 'received M' matters much more than the 'sent M', though, first because there are millions of receivers, and second because it's the received M that determines how people react.
        5y ago

Jeff_Brown 5y ago

jvanderbot 5y ago
He traced the math behind it back to a computational fluid dynamics paper, which the debunkees had used equations from with no context. He cold emailed professors asking for feedback of his organized findings, got back a 3000 word article based on it, and with help from a mathematician and psychologist in smoothing the writeup, and ultimately got it published in a top journal and the offending results recognized as flawed. The general crusade continues. The result in question was the legendary 3-1 positive to negative tipping point between flourishing individuals and languishing ones.
- PaulDavisThe1st 5y ago
  Brown did not write the "3000 word critique". From TFA: > Sokal, “came back with my draft one day,” Brown says, “and had written 3,000 words.”
  - jvanderbot 5y ago
    Ah! My reading comprehension strikes again. Edited, thank you.

300bps 5y ago

Wikipedia has a good tl;dr: https://en.wikipedia.org/wiki/Critical_positivity_ratio https://en.wikipedia.org/wiki/Critical_positivity_ratio Later in 2013, the critical positivity ratio aroused the skepticism of Nick Brown, a graduate student in applied positive psychology, who felt that the paper's mathematical claims underlying the critical positivity ratio were fundamentally flawed.[6] Brown collaborated with physicist Alan Sokal and psychologist Harris Friedman on a re-analysis of the paper's data. They argued that Fredrickson and Losada's paper contained "numerous fundamental conceptual and mathematical errors", as did Losada's earlier work on positive psychology, which completely invalidated their claims.[7] In their response, Hämäläinen and colleagues argued that there were no fundamental errors in the mathematics itself, but the problems relate to the interpretation and justification of the use of the model.[8] Losada declined to respond to the criticism, indicating that he was too busy running his consulting business.[6] Fredrickson wrote a response in which she conceded that the mathematical aspects of the critical positivity ratio were "questionable" and that she had "neither the expertise nor the insight" to defend them, but she maintained that the empirical evidence was solid.[9] Brown and colleagues, whose response was published the next year, maintain that there is no evidence for the critical positivity ratio.[10] In response, American Psychologist formally retracted the mathematical modeling elements of Fredrickson & Losada's paper, including the specific critical positivity ratio of 2.9013, as invalid.

Joeboy 5y ago
Since TFA is quite long and has quite an annoying layout, I tried Reader View in Firefox, and it gave me a completely different article. Weird. Thanks for posting the tl;dr.
- CRConrad 5y ago
  Yup, same here. I think it was because it's one of those "infinite scrolling" pages, where when you've scrolled down to the end of the article another begins (and the URL in the address bar has also magically changed). Yet another reason to hate that shit.

colechristensen 5y ago

I got bored. This is an article style I really don’t like, the interesting information is buried towards the end after thousands of mostly irrelevant bits of narrative which aren’t worth remembering. I can guess the ending though, technical guy sees psychologists appropriating a well defined mathematical structure, doing some hand waving, and claiming that it applies to your everyday life in a shallow way. Guy is hero. Yes, psychology is absolutely full of bad science and bad scientists. Lots of fields are, but particularly psychology.

BeefWellington 5y ago
> Yes, psychology is absolutely full of bad science and bad scientists. Lots of fields are, but particularly psychology. Social sciences in general are probably harder than other fields for the simple reason that it requires other people's time and effort. A psychologist friend of mine once dropped the line "we know a lot about how university students view the world" when discussing the replication crisis. I think that while simplifying things quite a bit, it is very apt. Edited for coffee
- tgv 5y ago
  If the latter were true, I'd be surprised. It's obviously a quip about using students as subjects for credits, but the average psychology study reveals very little about students. After all, it's how they got convinced psi exists (Bem, 2011).
  - mywittyname 5y ago
    Economics is where you really glean insights into university students. For example, they enjoy free food exponentially more than they like very cheap food. Also, they great at playing games for money.
- colechristensen 5y ago
  It's more than that, people who aren't really interested in science or the math required to do good science are attracted to the "social sciences" in a much higher degree. People who are unqualified to do analyses are publishing papers and those papers are getting published with the approval of their peers who are similarly likely to be unqualified.

Scarblac 5y ago

This is a good quote: > “One could describe a team’s interactions as ‘sparky’ and conﬁdently predict that their emotions would be subject to the same laws that govern the dielectric breakdown of air under the inﬂuence of an electric ﬁeld. Alternatively, the interactions of a team of researchers whose journal articles are characterized by ‘smoke and mirrors’ could be modeled using the physics of airborne particulate combustion residues, combined in some way with classical optics"

derbOac 5y ago
To be honest I think this story is overplayed a bit. I've heard of that tipping point idea before, or the positive to negative ratio thing, but I think most in the field didn't take it more seriously than an observation that couples, groups, etc that function well are more positive and supportive. It's good to call this stuff out but part of the reason no one else did is because outside of a certain subsubfield no one took it that seriously.
- robwwilliams 5y ago
  You may well be right, but this is too casual an approach toward what purports to be science in a nominally good journal. Modern science is amazingly well insulated from “in-house” criticism after the review process. As a result crap can flourish. We need more data/analyst wasps.
- munificent 5y ago
  I strongly disagree with this take. The whole point of science is to acquire knowledge that eventually gets communicated outside of the scientific body producing it. If the papers never leave the ivory tower, or never lead to other papers that do, it's just navel gazing. When a scientific community has a chunk of their work that everyone in the tower knows is bullshit but never gets corrected, that metadata doesn't leave the ivory tower. What happens instead is that papers like this flutter out the window, get picked up by journalists who don't have access to tribal knowlege, and then get spread wide and far. The information appears legitimate because it did come from a trusted scientific body. It just had the necessary context that it's actually bullshit left off. Every scientific community has an obligation to ensure that the papers they publish and advocate actually mean what they say they mean. If nothing else, because not doing that ultimately tarnishes the brand and undermines the trustworthiness of that community.

oneepic 5y ago

How can you call the guy a CS major? He was a grad student in psychology, right?

jvanderbot 5y ago
"The married father of two had graduated from Cambridge University in 1981 with a degree in computer science, and spent most of his career as an IT networks operator at an international organization in Strasbourg, France." Pretty early in the article.
- ineedasername 5y ago
  Yes, but the context was also with that person as a current student, which was not in CS, and his CS background didn't have much bearing on what he discovered. It would have been just as relevant to say "Father of two debunked..." A better title would have been "New Student in APS debunks core finding in the field."
  - kwhitefoot 5y ago
    But that wouldn't attract so many HNers to comment.
  - jvanderbot 5y ago
    HN readers are predominately CS/IT folks, and would probably relate better to that aspect of his person than others. That's certainly why I read the article. Should I emphasize that he lives in the UK? "UK citizen debunks positive psychology" Or should I emphasize the statehood of the person debunked? "UK citizen debunks US professor with one neat trick" Just trying to summarize the relevant parts of the story without clickbait -- which the article did pretty well I thought.
    - PaulDavisThe1st 5y ago
      Going the full clickbait: "You'll never believe how this aging British empty-nested IT guy took down a US professor with one neat trick!"

duxup 5y ago

It seems weird this theory’s backbone was just some random person who contacted someone and said something like “hey I have math that says you’re right” and it just takes off from there. Until some other rando checks it out of curiosity…

consumer451 5y ago
I think you have just described the open source code review model. edit: and science in general?
- duxup 5y ago
  -PR approved “Rando said it looks good “-
wheelinsupial 5y ago
Do you mean this quote: "A Chilean business-consultant named Marcial Losada, 'who had begun to dabble in what had become his passion: mathematical modeling of group behavior,' sent her an out-of-the-blue email." The article seems to ignore it, but Losada had a PhD in social and organizational psychology from the U of Michigan [1]. [1] http://losada.socialpsychology.org/ http://losada.socialpsychology.org/

jvanderbot 5y ago

TL;DR Nick Brown was a 50y/o CS/IT employee who wanted to cross-train to human resources and wanted "evidence-based stuff". He encountered lots of fantastic positive psychology claims that were being accepted wholesale as "quantitative". He traced the math behind a critical one back to a computational fluid dynamics paper, which the debunkees had used equations from with no context. (allegedly because of the use of "Flow" in positive psychology!) He cold emailed professors asking for feedback, wrote a 3000 word critique, got help from a mathematician and psychologist in smoothing the writeup, and ultimately got it published in a top journal and the offending results recognized as flawed. The general crusade continues along with co-crusaders Sokal and Friedman. The result in question was the legendary 3-1 positive to negative tipping point between flourishing individuals and languishing ones.

readthenotes1 5y ago
TL;DR: Most psych research results are garbage. Some make millions for the authors. Here's yet another example.
- AlwaysRock 5y ago
  Some make millions for Authors who take those papers and cite them in popular psych books. The actual authors of the papers are rarely, if ever, making millions.
  - frostburg 5y ago
    The 9 million dollars in grants that the author received probably were also completely wasted, since someone that would randomly apply CFD equations to people is unlikely to have published worthwhile research.
    - CRConrad 5y ago
      Remains to be tallied how many of her research assistants on those projects went on to have careers in the field... Careers which are now more or less tainted by the association. Even harder to tally: How many of those deserve to be more tainted because they saw it for what it was but didn't speak up. (Yeah, I know, hard to do for a lowly assistant.)

dang 5y ago

I remember past threads about this but can't find them. Anyone? Edit: the list so far - thanks caaqil and setgree! I feel like there were still others, if anyone can find them... Nick Brown Smelled Bull (2013) - https://news.ycombinator.com/item?id=9791302 https://news.ycombinator.com/item?id=9791302 - June 2015 (1 comment) The British amateur who debunked the mathematics of happiness (2014) - https://news.ycombinator.com/item?id=16515822 https://news.ycombinator.com/item?id=16515822 - March 2018 (213 comments) The British amateur who debunked the mathematics of happiness (2014) - https://news.ycombinator.com/item?id=9085639 https://news.ycombinator.com/item?id=9085639 - Feb 2015 (15 comments)

setgree 5y ago
inter alia: https://news.ycombinator.com/item?id=16515822 https://news.ycombinator.com/item?id=16515822
caaqil 5y ago
https://news.ycombinator.com/item?id=9085639 https://news.ycombinator.com/item?id=9085639
froh 5y ago
And it's a long detailed read about the genesis of this paper: The Complex Dynamics of Wishful Thinking https://pubmed.ncbi.nlm.nih.gov/23855896/ https://pubmed.ncbi.nlm.nih.gov/23855896/

renewiltord 5y ago

His Twitter account: https://twitter.com/sTeamTraen https://twitter.com/sTeamTraen Seems he does a lot of this data policing stuff. Neat. The funny thing is that a lot of people go around saying "yeah, engineers go around thinking they can just walk into field X and study it without context". Yup. They do, and they can. Another article https://www.theguardian.com/science/2014/jan/19/mathematics-of-happiness-debunked-nick-brown https://www.theguardian.com/science/2014/jan/19/mathematics-...

derbOac 5y ago
Sometimes they can and sometimes they can't. Works like that for all kinds of crossovers, and probably should happen more often than it does. I can think of plenty of times CS has benefitted from psychology too, or is reinventing the wheel.
- renewiltord 5y ago
  Haha, yes. I'm just a big fan of people trying. I'm also a big fan of anti-gatekeeping.
  - derbOac 5y ago
    I surely agree with you on that.
dogleash 5y ago
>that a lot of people go around saying "yeah, engineers go around thinking they can just walk into field X and study it without context". That's sorta the point of a degree in the sciences rather than going to a vocational school, right? It's amusing that people find it arrogant. It is that they're showing a level of competence we're all claiming and maybe only pretending to have? Is it "swim in your own lane" union-style isolation? Is it sneering at nerds for not having the good sense to shut it? Is it inflated egos that still think the things they learned in undergrad were hard?
at-fates-hands 5y ago
>> The funny thing is that a lot of people go around saying "yeah, engineers go around thinking they can just walk into field X and study it without context" I have a degree in Anthropology and transitioned into UX Research. 90%+ of what you do is observation in both fields. Most of the engineers and math people I've met are horrendously bad at observing behavior. In some fields, its just a different set of skills engineers are not capable of applying very well.

setgree 5y ago

I worked with Nick a few times in a previous job and he was a reviewer on a paper I wrote; he was really nice and professional, and his feedback was constructive and incisive. The movement he's part of is fundamentally disagreeable (in the 'big five' sense [0]), and some of its prominent characters, as you might expect, have reputations for being interpersonally difficult [1]. Collectively they've been labeled "data thugs" [2] and accused of "methodological terrorism" [3], which is why I think Nick's fundamental goodness is an especially valuable asset. We'd gain a ton of social value from certain reforms to the scientific research apparatus, and, IMO, being kind when delivering pointed critiques helps a difficult pill go down. That's different than pulling your punches; it just means staying focused on the thing you actually want to change. [0] https://en.wikipedia.org/wiki/Big_Five_personality_traits https://en.wikipedia.org/wiki/Big_Five_personality_traits [1] https://medium.com/@OmnesRes/the-dumbest-fucking-thing-ive-ever-read-8653e2409ec4 https://medium.com/@OmnesRes/the-dumbest-fucking-thing-ive-e... [2] https://www.science.org/content/article/meet-data-thugs-out-expose-shoddy-and-questionable-research https://www.science.org/content/article/meet-data-thugs-out-... [3] https://statmodeling.stat.columbia.edu/2016/09/21/what-has-happened-down-here-is-the-winds-have-changed/ https://statmodeling.stat.columbia.edu/2016/09/21/what-has-h...

whimsicalism 5y ago
I'm unsure what your second citation is for, they don't use the word "data thugs" but the author certainly seems annoying (although likely correct about the points they're making as far as I read). e: the parent comment has now been edited for clarity, keeping my comment for posterity
- setgree 5y ago
  The piece is written by a member of the posse [0] who can be, as I think the above quoted piece demonstrates, somewhat abrasive [0] https://medium.com/@OmnesRes/the-circle-of-data-thug-life-815289b96c95 https://medium.com/@OmnesRes/the-circle-of-data-thug-life-81...
  - MegaButts 5y ago

okl 5y ago

I think they should have sent in their paper somewhere other than APA. Don't expect courtesy in consequential confrontations. I think similar about responsible disclosure sometimes. If a company played dirty in the past why bother with accomodating them.

duxup 5y ago
I think they wanted to give the APA a chance to make the right call / improve, and it also seems they found the peer process valuable and they wanted to get that feedback. Seems like it worked out.
- CRConrad 5y ago
  > Seems like it worked out. Yeah, kind of... Mainly by exposing how reluctant the APA was to make the right call / improve.
  - duxup 5y ago
    They did the thing and got feedback, worked as far as I can tell.

MetaWhirledPeas 5y ago

I see this as an argument for adding more STEM to every degree plan. There's unavoidable crossover, and people need intuition to properly question unfounded claims.

wbsss4412 5y ago
Unfortunately I’m not sure that STEM in and of itself is the antidote. It’s very easy to just routinely apply methods without properly reasoning about the underlying assumptions.
- MetaWhirledPeas 5y ago
  The less exposure one has to these concepts the more tempting it becomes to skip the reasoning part.
  - wbsss4412 5y ago
    True, but there is an upper bound to the amount of exposure one can receive before they are simply a STEM major, rather than another major with more STEM exposure.
- klabb3 5y ago
  Be careful what you wish for. Increasing the competence of fraudsters will simply make fraud harder to detect. Had the original theory simply included a "measured" coefficient, the debunking would have been much weaker and never made it into a gotcha story for bystanders to enjoy. This was sloppy from the outset. What we need isn't stronger technical skills, but strong self-correction mechanisms and incentive structures. (Ironically, the management consultant who orchestrated this whole ploy is in the field of group psychology - so perhaps it was performance art all along)
- FredPret 5y ago
  But you shouldn’t be able to get a degree in a soft “science” without demonstrating that you can apply disciplined thinking by passing some proper STEM courses.

ChrisMarshallNY 5y ago

I've been watching Dopesick, and the part, where they are chasing down the "1%" study, is pretty damn sobering.

klabb3 5y ago
Citations are upvotes for academia.
- eru 5y ago
  Perhaps more like links in the original Google PageRank?
- mhh__ 5y ago
  Or downvotes, gotta cite it to say it's bullshit.

efsavage 5y ago

IANA data scientist, but I would hope that 4 decimal places in the context of psychology would trigger anyone's BS meter.

anyfactor 5y ago
I think there is inverse relation of how frequently a number is written everywhere with it's degree of decimal places. It is a convenience thing. Also if the number is generated on a computer and used there subsequently 4 decimal is plenty for social science and business research/purpose IMO.
- klabb3 5y ago
  I think parent means the opposite, that 4 decimal places is really high precision in a field with lots of measurement bias and noise.
  - anyfactor 5y ago
    TIL. This is something new I learned. Can you tell me what to google to learn about this bias?
    - waterheater 5y ago
      https://en.wikipedia.org/wiki/Statistical_significance https://en.wikipedia.org/wiki/Statistical_significance When the Higgs boson was declared "discovered," the particle was found to exist with a five-sigma certainty. This brief article does a great job of explaining what that means: https://www.r-bloggers.com/2012/07/the-higgs-boson-5-sigma-and-the-concept-of-p-values/ https://www.r-bloggers.com/2012/07/the-higgs-boson-5-sigma-a... Physicists got to that point (very roughly) by constructing complex mathematical models and building large machines to test those models. In contrast, leading psychology models change with the seasons, and large-scale human experiments cannot be (ethically) conducted with the control as seen in physics. Because psychology experiments have substantial environmental variability, drawing robust conclusions in psychology is exceedingly difficult. This fact should mean that very-high-certainty findings in psychology studies must be subjected to the highest scrutiny.

photochemsyn 5y ago

Taking equations from physics out of context to bolster social science claims isn't unique to psychology, other social sciences like economics have done the same thing with little more justification. The rationale behind this approach is simple marketing, in that 'using more mathy stuff' will sell better to the general public than non-mathy hand-waving claims will. However, the practioners of this approach don't seem to understand that physical equations are constructed by analysis of empirical experimental and observational data on one hand, and theoretical mathematical development on the other, within limited scopes of applicability. Copying equations from one area of science to another willy-nilly like the article describes is just ridiculous - would cosmologists studying general relatively just plug in quantum mechanical equations in the hope of getting a 'mathy explanation'? Here's another example from 'mathematically sound neoclassical economics', which does the same with 19th century physical science equations: > "The strategy the economists used was as simple as it was absurd—they substituted economic variables for physical ones. Utility (a measure of economic well-being) took the place of energy; the sum of utility and expenditure replaced potential and kinetic energy. A number of well-known mathematicians and physicists told the economists that there was absolutely no basis for making these substitutions. But the economists ignored such criticisms and proceeded to claim that they had transformed their field of study into a rigorously mathematical scientific discipline." https://www.scientificamerican.com/article/the-economist-has-no-clothes/ https://www.scientificamerican.com/article/the-economist-has...

munchler 5y ago
Based on other people in the field whose responses were quoted at the end of the article, I suspect this phenomenon may still be occurring. Specifically: "David Pincus, a psychologist at Chapman University who specializes in the application of chaos theory to psychology" "Stephen Guastello, a psychologist specializing in nonlinear dynamics at Marquette University"
notahacker 5y ago
There are niche heterodox subfields of economics that literally are just a dubious physics analogy (thermoeconomics), but the source's argument that marginal utility theory was copying von Hemholtz is the sort of extraordinary claim that requires extraordinary evidence, not zero citations, baffling confusion of economist Francis Edgeworth with literary writer Maria Edgeworth and a lot of tacked on claims that are laughably false (Jevons certainly didn't believe that "natural resources are inexhaustible: he was actually initially famous for trying to predict when the coal supply would run out!). There are obvious reasons for studies of how money is spent to involve mathematics (though Menger managed to deduce marginal utility theory from first principles without equations) and it's unsurprising some of the equation forms superficially resemble some of the many simple equations of physics. There are much better criticisms of neoclassical economics and the application of neoclassical economics than accusing the early economists of plagiarising another field!

unixhero 5y ago

What is middle aged of any relevance???

deleted 5y ago

[deleted]

TameAntelope 5y ago

I think this is a fantastic example of how a) anyone can call bullshit on science if it doesn't make sense and b) there are more and less correct ways of calling bullshit on established science. You don't need to hop on a podcast or radio show in order to question the orthodoxy of a scientific claim, and in fact you shouldn't if you want to be taken seriously. Nick Brown would have hurt his case had he done that, and instead found a path that actually knocked down a faulty concept in a way that has lasting effects. This is the process working. This is all okay, and welcome, IMO, though possibly not by everyone involved.

jvanderbot 5y ago
I agree. The real shining light here is that a determined amateur was able to get free, volunteer help from inside the community to weather the publication process. Those who read the review / revision process in this article and were dismayed / convinced the process was flawed should realize this is precisely how review works in many fields. It takes a year to get a paper reviewed and revised, and a "25 page" (TFA) statement of revision is par. Not "all" will have the stomach for that, but once you are part of that process, it's normal.
- horsawlarway 5y ago
  You consider having to threaten the reputation of an academic journal by emailing the CEO directly before you even make it to the review for a well written, concise, critical paper "part of that process"? Not even mentioning that it required the publicity and reputation of a well known figure (Sokal) to even open that door for them. If that's the "shining light" of this industry - I sure as fuck don't want to see the dirty alleyway.
  - jvanderbot 5y ago
    I didn't say that. I said the shining light was insiders helping. I also said the peer review process is challenging and that was an accurate representation. Two different things.

deleted 5y ago

[deleted]

phkahler 5y ago

It doesnt seem possible to have a fixed positivity ratio when considering research on extinction. The gist of extinction is that when training using positive feedback, the desired response will cease when the positive feedback does (extinction). In order to maintain the behavior the feedback has to essentially be randomized with a decreasing prevalence (positivity ratio).

j7ake 5y ago

A quote struck me as relevant during covid and work from home times: “because more good things will happen to you than bad on any given day, but nothing will happen to you if you just sit indoors.”

j7ake 5y ago

This misuse and abuse of methods will continue as long as you can make a big splash in the field. The technical correction of the misuse afterwards can then be tacked on to the end of your article later, with absolutely no recourse or penalty to your career.

roywiggins 5y ago

> In a valid fluid dynamics problem, the numbers plugged into the equation must correlate to the properties of the fluid being studied. But in attempting to draw an equivalence between the physical flow of liquids and the emotional “flow” of human beings, Losada had simply lifted the numbers that Lorenz used in 1963 to explain his method in the abstract, numbers used merely for illustrative purposes. Losada’s results, along with the pretty butterfly graphs Brown had been shown in class, were essentially meaningless. Incredible. So it wasn't just a matter of, say, applying an equation beyond its domain, or generating the inputs to the function in a sketchy way, or hacking the inputs to the function until they got an answer they liked, but literally copying example parameters from a 1963 paper. The "Complex Dynamics of Wishful Thinking" paper is very readable: https://physics.nyu.edu/faculty/sokal/complex_dynamics_final_clean.pdf https://physics.nyu.edu/faculty/sokal/complex_dynamics_final...

FabHK 5y ago
It is very readable, and it utterly nihilates Losada's papers. It is hard to believe that anyone could have taken them seriously, let alone let them get published. Sigh. Nice touch that Sokal helped debunking it. I wonder to which extent this taints the whole of "Positive Psychology", which I was moderately excited about.

at-fates-hands 5y ago

And through it all, the three met only once in person, when Friedman was in London to give an academic address. It was a day spent scribbling equations in chalk on blackboards and talking strategy – a debunkers’ bonhomie. I truly miss this about academics. Where you get a chance to sit down and think freely and pick someone else's brain in the pursuit of some hypothesis you're chasing down.

throwawaypt 5y ago

Anyone have suggestions of the best way to call BS on clinical studies and their marketing? A publicly traded company's products are all based on BS science. I've pointed it out to them but they are dead silent. Just one example is them comparing (in marketing) the active group of one study to the control group of another study performed in different countries years apart with different inclusions criteria massively different n and insanely different dropout rates. It just blows my mind this stuff happens. If anyone is interested here is the full writeup: https://www.sendbig.com/view-files?Id=c131d2fc-9a4a-0616-f772-2ebc7271efbb https://www.sendbig.com/view-files?Id=c131d2fc-9a4a-0616-f77...

eru 5y ago
You could try a shareholder lawsuit?
toast0 5y ago
Complain to the agencies that are/should regulate them. FTC coverse false/misleading advertisements generally, but maybe just FDA for mediciney things?
- throwawaypt 5y ago
  Good idea. Just lodged a complaint with FDA and SEC. They won't provide updates but at least someone might read it. Will look into FTC.

deleted 5y ago

[deleted]

errcorrectcode 5y ago

Toxic positivity and toxic negativity are grating to some, myself included. The issue is that soft, squishy not-STEM "sciences" aren't all that evidence-based or experimentally-verifiable. That doesn't make them completely useless, it makes them prone to fungible, inconsistent guidelines rather than hypotheses, theories, or laws.

davidwritesbugs 5y ago

The whole "Cited 350 times" made me laugh. I used to think this type of number sounded impressive. Recently I just started to do a Masters and was told "1 citation for every hundred words", so dozens of papers need reading and it take for *ever* weeks to read just a small number. Then I recalled articles claiming most citations are never checked so, and you won't believe this, I skim read papers and if vaguely relevent inserted cursory comments from the paper and cited it. Did that go horribly wrong? Go on, guess. So 350? I so believe it.

ALittleLight 5y ago
That seems like it's own separate problem which is also bad.
- davidwritesbugs 5y ago
  It was just a lazy metric to measure effort and engagement, and that relieved the lecturer of the need to read the paper himself to see if you'd absorbed it and analysed its relevance correctly. I can't say I'm unsympathetic - the guy had 50 students to mark. I'm guessing this is standard in academia but if they'd said "read and include relevant papers" I'd have done a good job on half a dozen and the lecturer probably wouldn't have been overburdened either.
8K832d7tNmiQ 5y ago
Ah yes, the classic Goodhart's law [0] working as intended again. [0]:https://en.m.wikipedia.org/wiki/Goodhart%27s_law https://en.m.wikipedia.org/wiki/Goodhart%27s_law
- mhh__ 5y ago
  It's more of a "it'll do"-ism from bored supervisors IMO, could be imposed upon them but also is an easy way to avoid extra being different.
PeterisP 5y ago
Things like your master thesis or anything like that simply aren't included in the citation count at all, generally no one cares how much students or general public are citing your work so that practice has no influence on the metric being used. As the article states, it was cited 350 times in academic journals, i.e. cited in 350 reviewed publications with novel scientific work written by her peers.

sampo 5y ago

> But in attempting to draw an equivalence between the physical flow of liquids and the emotional “flow” of human beings, Losada had simply lifted the numbers that Lorenz used in 1963 to explain his method in the abstract, numbers used merely for illustrative purposes. Losada’s results, along with the pretty butterfly graphs Brown had been shown in class, were essentially meaningless. It was a bit crazy in the late 1990s. Chaos theory had been popularized, and the social sciences copied fashionable buzzwords and buzzconcepts from chaos theory, and used them totally out of context. Maybe it was crazy in the 1920s, too? Einstein's theory of relativity had become popular, and sciences that had nothing to do with gravitation, copied the "everything is relative" slogan.

YaBomm 5y ago

mannerheim 5y ago

'I think the educational and psychological studies I mentioned are examples of what I would like to call Cargo Cult Science. In the South Seas there is a Cargo Cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas—he’s the controller—and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things Cargo Cult Science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.' - Richard Feynman, 'Cargo Cult Science' https://calteches.library.caltech.edu/51/2/CargoCult.htm https://calteches.library.caltech.edu/51/2/CargoCult.htm

georgecmu 5y ago

For those curious: The two nonsense papers are: Losada, Marcial. “The complex dynamics of high performance teams.” Mathematical and Computer Modelling 30 (1999): 179-192. | https://www.sciencedirect.com/science/article/pii/S0895717799001892?via%3Dihub https://www.sciencedirect.com/science/article/pii/S089571779... Fredrickson, B. L., & Losada, M. F. (2005). Positive Affect and the Complex Dynamics of Human Flourishing. American Psychologist, 60(7), 678–686. | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3126111/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3126111/ The debunking paper is Brown, N. J. L., Sokal, A. D., & Friedman, H. L. (2013). The complex dynamics of wishful thinking: The critical positivity ratio. American Psychologist, 68(9), 801–813. | https://arxiv.org/pdf/1307.7006.pdf https://arxiv.org/pdf/1307.7006.pdf

f154hfds 5y ago

Interesting read! The most disturbing part is not fraud scientists, I think there will always be a certain percentage of fraudsters in any field. The most disturbing part is the 300+ citations, scientists who based their work off of an obviously garbage paper. It seems inescapable that the reason why must have been complete mathematical incompetency across the entire field. Say a psychologist partners with perhaps a chemist and uses some organic chemistry to 'prove' some critical point in a paper. I could imagine brushing over the details and citing the paper when any chemist would see the fraud as 'obvious'. Such is the complexity of cross discipline science I guess, and the reality that the sum total of human knowledge is beyond any individual's ability to grasp. That being said, knowing that some papers are utter garbage I should do my due diligence by taking a humble pill and reaching out to my colleague in the chem department across the street to tell me what this critical chemistry is about before I include it in my work.

justin66 5y ago
> The most disturbing part is the 300+ citations, scientists who based their work off of an obviously garbage paper. > It seems inescapable that the reason why must have been complete mathematical incompetency across the entire field. I think you won't have to try very hard to find analogous failures to properly test the truth or falsehood of the literature's conventional wisdom in more "mathematically competent" fields.

fluorinerocket 5y ago

richardfey 5y ago

Here's the key passage (SPOILER ahead): > Losada had simply lifted the numbers that Lorenz used in 1963 to explain his method in the abstract, numbers used merely for illustrative purposes. Losada’s results, along with the pretty butterfly graphs Brown had been shown in class, were essentially meaningless.

monksy 5y ago

I can guarantee you there are a lot of people really upset with him. A lot of those kinds of studies are "this makes me feel good and this is the world I want to see" types. No consideration of the negative outcomes of bad info, bad practices, or even externalizes.

tomohawk 5y ago

You have to wonder how many people were harmed by this fraud. And that's exactly what this is. The original paper was a fraud. No honest mistake can explain it. The reviewers committed fraud by not seeking competent input when they lacked the competence to do the math. How many people under psychological care were given treatments and were harmed? How many companies wasted on money on this and should be able to seek compensation? IANAL, but this smells like litigation.

mlindner 5y ago

Psychology in general is full of hucksterism. My sister got her bachelor's in Pyschology and she also found a whole ton of nonsense in it. She says that there's been a split in psychology and the real science was getting done on the "Biopsychology, Cognition, and Neuroscience (BCN)" side of the split while the psychology side is getting further and further away from real science and more into the realm of the made-up.

cwkoss 5y ago
I did a psychology undergrad major in the late 00s. There was a shocking amount of bad science in the required courses. Lots of stuff that supported "common sense" but would be impossible to replicate, and the methodology didn't even align with the summary of why the study was important to learn about. Mountains of work built off imprecise self assessment tests. You can find a psych study to support just about any idea, and it seems like that's what many successful psychologists in academia have done. Big part of why I decided not to pursue it as a career.

3flp 5y ago

What I find sad about this is that the effort and skill required to debunk this nonsense is an order of magnitude higher than the effort and skill required to create it in the first place. Aka, the "Gish Gallop" and other analogues, observations and quotes about this effect, made many times.

ensan 5y ago

Reading the title, I expected the debunker to be a middle-aged many _studying_ CS.

mleonhard 5y ago

If there was a collaborative project to review dubious published research papers, which of you would be interested in contributing? I think there are four categories of papers to focus on: 1. Papers with obviously false claims. 2. Papers that omit information that is crucial for verifying their claims. 3. Dubious papers that introduced ideas that attracted a lot of attention and subsequent sketchy papers. 4. Dubious papers published by people who just started in PhD programs.

mhh__ 5y ago
Post-publish review is something that should probably be improved. It can't exactly be open to the public to contribute but equally the public should be able to see if scientists have (anonymous) serious doubts about a paper.

etempleton 5y ago

"Lies, damned lies, and statistics." Unfortunately a lot of academic papers throw hopelessly tortured sentences, esoteric words, and complex formulas at you in the hope that no one notices that the whole paper is built on a house of cards: https://www.nature.com/articles/d41586-021-02134-0 https://www.nature.com/articles/d41586-021-02134-0

vzaliva 5y ago

The paper mentioned in the article: https://physics.nyu.edu/faculty/sokal/complex_dynamics_final_clean.pdf https://physics.nyu.edu/faculty/sokal/complex_dynamics_final...

vzaliva 5y ago

This is a fascinating story of how some questionable research slipped through peer review safeguards of academic publishing and flourished. More importantly, it shows how the scientific community eventually caught it and debunked it. The original results were recalled. Sadly, one of the main perpetrators slipped into murky waters of "business consulting" and continue to sell this snake oil: " Dr. Marcial Losada, has a Ph.D. in Social and Organizational Psychology from the University of Michigan. Dr. Losada was invited to present his work at Cambridge University in England, Harvard Business School, Graduate Business School at the University of Michigan, Sloan School of Management at MIT, Kellogg School of Management at Northwestern University, Stanford Research Institute, and Institute for the Future in Menlo Park, California. He presented his findings on applications of nonlinear dynamics to team interaction and productivity at the prestigious Director’s Colloquium in Los Alamos National Laboratory. He has briefed Vice-President Al Gore and the president of MIT, Dr. Charles Vest, on the interaction dynamics of high performance teams." https://www.losadalineconsulting.net/our_team https://www.losadalineconsulting.net/our_team

isitmadeofglass 5y ago

I feel like the field and the journals in psychology have a history of being and currently are quite fragile. I have taken a couple of examples and done some simulations and come to the conclusion that the kinetic fragility index of the field is about the same as amorphous silicon dioxide most the time. And as anyone will tell you if your building a house of that material. It is vital that you ensure no one around is looking critically at it. So you should paint it with a brick pattern, and ensure nothing heavier than a feather is ever thrown towards it. In that regards it seems the journal in this case agrees, as they seriously resisted a scientific rebuttal of a piece until they realized they could not prevent it. Had they had integrity (which I believe we can model analogue to a high Young’s modulus, I haven’t yet finished simulations to give a precise number of how high it should be for the journal) then they would have been thrilled at the chance to adjust their building to not include the original material.

sTeamTraen 5y ago

Hi - I am the subject of this article. I don't know how to prove that other than tweeting a link to this post, which I will do right now (renewiltord already posted my Twitter account here https://news.ycombinator.com/item?id=30247543 https://news.ycombinator.com/item?id=30247543). Thanks for the kind words to those who posted them. For everyone else, feel free to AMA. Edit: Here is the link to the tweet that links to this comment: https://twitter.com/sTeamTraen/status/1491041770217811977?s=20&t=qdC3edEhiZs4g3G7Y-gagQ https://twitter.com/sTeamTraen/status/1491041770217811977?s=...