7 ms·
Andrew Gelman (of NYU), who is featured in the story and is criticized by Cuddy as being unfair and trollish, wrote a response: http://andrewgelman.com/2017/10
by danso 9y ago
Andrew Gelman (of NYU), who is featured in the story and is criticized by Cuddy as being unfair and trollish, wrote a response:
http://andrewgelman.com/2017/10/18/beyond-power-pose-using-replication-failures-better-understanding-data-collection-analysis-better-science/ http://andrewgelman.com/2017/10/18/beyond-power-pose-using-r...
I thought that Gelman wasn't presented in the best of light, especially his awkward quotes about not wanting to engage in "interpersonal conflict". But he called the piece "excellent and fair to all sides" and was pretty gracious in response to Cuddy's criticisms and how the NYT portrayed him. I honestly wasn't a huge fan of the article but re-reading it after a bit of sleep, and knowing that Gelman thought it was strong, I appreciated the issues it brought up. Certainly has to be one of the longest stories about p-hacking/p-curves in mainstream media.
I sympathize with Cuddy and have no doubt that among the legit attention that Gelman and other critics have directed toward her, she's faced hatemail sparked by dumb sexism, which makes it easier to tar the legitimate criticism as being sexist.
I do think Cuddy needs to face legitimate criticism. She may not have been a malicious fraudster, but ignoring criticism after a long time is not much different than actively being deceptive. I also can believe that she faces hostile criticism that isn't equally distributed among all the other researchers who used weak, but previously acceptable standards. And I don't think sexism can be dismissed outright, if people find it safer to talk shit to a woman compared to a man.
What I wish the story had more of was how scientific criticism is done outside of Gelman's blog. Gelman is the main scientist/research I follow on the topic. Apparently some see his snarky critique as toxic, and Gelman says he is intending to reevaluate the way he writes about things so that his point doesn't get lost in cheap jokes. How do other blogs do it? I'm somewhat sympathetic to Gelman because I perceive that there is significant institutional resistance (and few incentives) to re-examine and correct a peer review study, and being harsh/edgy/rude and beating the drum over time might be the only way to get scientists to care? But that doesn't mean his critique can't be refined for the benefit of his mission.
- skybrian 9y agoStakes matter. A million dollar poker game might be structurally identical to a game played for fun, but its effects on the people involved are very different. We wouldn't expect the players in these two games to think the same way. Talking about the stakes is in direct conflict with the scientific ideal of focusing on the science without talking about its effects on the scientists involved. For example, from Gelman's response: "I just wanted to emphasize that there’s no reason her career, or even her famous Ted talk, has to rely on a particular intriguing idea (on there being a large and predictable effect of a certain pose) that happened not to work out." Well, except that maybe she wouldn't have gotten a Ted talk without it? He's saying, hey, this paper shouldn't have been as important as it was. And according to the scientific ideal, we shouldn't get too attached to any particular results. But in practice, putting bad results aside sometimes requires a significant change of identity. Forgetting about that, even for a moment, is always going to come across as tin-eared. The scientific ideal is more compatible with low stakes than high stakes. But talking about the game as if it's low-stakes (or ignoring the stakes altogether) when actually it's high-stakes is always going to make discussions focused on the science alone come across as unsympathetic.
- dwaltrip 9y agoPerhaps we should be teaching scientists that staking their identities on particular results is to be resisted as much as possible. Science is about finding approximations of truth that are least influenced by the local context. The whims of a particular scientist's ego (especially if you are that scientist!) make for a rather nasty contextual influence. Of course, this is a difficult ask, but constantly striving towards this ideal is absolutely essential.
- projektir 9y ago> Perhaps we should be teaching scientists that staking their identities on particular results is to be resisted as much as possible. But does this make sense? Can a scientist survive without doing this? Can a scientist get very far without results right now? Is this a problem of the scientist, or perhaps the social/economic expectation?
- dwaltrip 9y agoI think this is an institution level failure to adapt to what science is like in the modern age. New results are significantly more difficult and expensive to achieve in modern science, as the easier stuff has already been conquered in centuries and decades past. The harder problems that science now works on means that distinguishing between actual results and incorrect conclusions is also much more difficult. The growth of science as a percentage of the population and economy has also finally slowed down a few decades ago, after centuries of rocket-like growth. It is now a maturing enterprise, instead of a high-growth startup. This means increased competition for resources and mindshare from those wanting to do science. Not surprisingly, all proxy measures and heuristics for evaluating scientific success are thus more likely to be abused. I don't think we have fully adapted to these fundamental changes. Disclaimer: I'm not a historian or researcher, but I've heard aspects of the above from a variety of sources, and it makes a lot of sense to me.
- michaelt 9y agoPersonally if I was going to stake my professional reputation on a set of results, I'd expand from an exploratory study with a 20% chance of being wrong to a more rigorous study with, say, a 0.1% chance of being wrong. Of course, that's much easier for people in STEM than it is for social scientists.
- srtjstjsj 9y agoAndrew Gelman is a rare breed -- he's brilliant, hard working, engages with the public, writes both technical and (semi-)pop articles, and isn't an asshole. And we works in a field -- mathematics -- where there is very little confusion as to whether a result is right or wrong. This makes lesser people working in dirtier subjects very uncomfortable: when he hits them they have nothing of substance to hit back with except generic insults.
- danso 9y agoAs someone outside the scientific-academic community, I find his posts very accessible for learning news/issues about studies. I greatly appreciate his passion in trying to stop junk science from reaching the media. Maybe his rhetoric is too brash and coarse for the respectful standards of scientific discourse, but it's as harsh as it needs to be for scolding media outlets for running with junk science: http://andrewgelman.com/2016/05/05/npr-bites/ http://andrewgelman.com/2016/05/05/npr-bites/ http://andrewgelman.com/2016/11/03/its-not-about-reality-its-all-about-a-salary/ http://andrewgelman.com/2016/11/03/its-not-about-reality-its...
- jmull 9y ago> ...and isn't an asshole... I just read his blog for the first time and thought he’s a huge asshole. It seems to me the primary effect of science-shaming on the blog is to rally the pitchforks, something he can’t possibly be ignorant of. But he apparently keeps on doing it. No doubt it’s good for the hits, but doesn’t contribute to a dynamic that leads to good science. Oh wells.
- danso 9y agoOut of curiosity, what did you read?
- mikebenfield 9y agoI don't know. A lot of the people Gelman complains about are, in my opinion, building their career on fraudulent work. They are undermining science. If there was ever any doubt that they were deliberately deceiving people, that doubt disappeared upon seeing their response to simple and legitimate criticism. A little needling on Gelman's blog is not too harsh, all things considered.
- ryanobjc 9y agoSexism is more than "safer to talk shit to a woman compared to a man" as you say in your article. Sexism comes out in target choice, and the strength of which ones chases their prey, and the vehemence of attacks. I do strongly believe that Cuddy was overly punished for being a woman, and being prominent in the field. Everyone loves when the "high and might" come "crashing down" -> the underdog stories and the desire for retribution are sought after more powerfully than (a) justified and (b) end worthiness of the goal. I don't think anyone can honestly say that this has improved the field and made science better. It just means now that there is money and recognition to be made being attack dogs. That's hardly better. As for Gelman, he wasn't portrayed in a great light, but I think it is fair. He's willing to encourage incivility and outright sexism and hostility as long as he doesnt have to talk to someone he disagrees with. That's the baseline definition of a coward there. And yes, I would discuss that with him in person and call him such a thing to his face.
- rukuu001 9y ago> I do strongly believe that Cuddy was overly punished for being a woman, and being prominent in the field. Theres a pattern of successful women receiving extreme levels of hate (Kathy Sierra et al). I'm not aware of this happening to guys.
- gt_ 9y ago>I'm not aware of this happening to guys. Can you be more specific? Or do you mean this in a general sense? Because isn't this obvious ignorance if so?
- rukuu001 9y agoApologies, I could have been more clear. What I mean is that I haven't heard of guys receiving death, rape etc threats accompanied by doxxing (ie 'this is his address, let's go get him') at the levels I've heard about with women[0]. [0] - https://en.wikipedia.org/wiki/Gamergate_controversy https://en.wikipedia.org/wiki/Gamergate_controversy
- Bartweiss 9y ago> I also can believe that she faces hostile criticism that isn't equally distributed among all the other researchers who used weak, but previously acceptable standards. I agree that this is true, but at a certain point it seems like a justified reaction to unequal importance. Other Gelman targets like the "himmicanes" paper generated a bit of media attention without any real consequence. Power poses, meanwhile, have become a major pop-science hit. They're still being widely read and promoted, even by tech conferences and news orgs like the BBC. And Cuddy in particular has chosen an aggressive non-scientific promotion campaign while her co-authors have made more honest attempts to take on the replication question - if Carvey was getting the same amount of flak I'd be more concerned that something unfair was happening. If power poses don't work, it seems like a much worse indictment of academic social psych than most other failures to replicate - the results were widely touted and are still being raised to public acceptance even as they fall apart. The challenges to stereotype threat, priming, and the IAT are even scarier - those results have become well-established and even enshrined in law. Throughout the replication crisis, my greatest frustration has been the inconsistent narrative coming from researchers. They're simultaneously demanding formal academic critiques instead of blogs, ignoring those critiques when they do exist, and then replying with the same sort of informal aggression they decried in the first place. I think you're exactly right that Gelman's aggressive approach is motivated by the field's unwillingness to acknowledge anything milder. And seeing responses like Fiske, Cuddy, and Wansink have largely convinced me that there's a major double-standard at work between the level of seriousness applied when attacking critics and the level applied when presenting results as academic truth.
- deleted 9y ago[deleted]
- ghostcluster 9y ago> the results were widely touted and are still being raised to public acceptance even as they fall apart. The challenges to stereotype threat, priming, and the IAT are even scarier They're in the same boat. It is maddening how much stereotype threat gets brought up to defend certain ideas, or how it has worked its way into corporate diversity policy, and it appears to be built on a foundation of sand [1]. The worst part is, if you question it publicly, you stand a change of being treated like James Damore. [1] - https://www.psychologytoday.com/blog/rabble-rouser/201512/is-stereotype-threat-overcooked-overstated-and-oversold https://www.psychologytoday.com/blog/rabble-rouser/201512/is...
- moyix 9y agoSmall correction: Gelman is at Columbia, not NYU.
- nkurz 9y agoBut he called the piece "excellent and fair to all sides" and was pretty gracious in response to Cuddy's criticisms and how the NYT portrayed him. I honestly wasn't a huge fan of the article but re-reading it after a bit of sleep, and knowing that Gelman thought it was strong, I appreciated the issues it brought up. I love (genuinely, non-ironically) that you reconsidered the merits of the article after reading Gelman's positive assessment of it. A possibly underappreciated benefit of Gelman's unabashed approach to criticism is that when he writes "I thought Dominus’s article was excellent and fair to all sides", it's reasonable to assume that he actually believes this rather than just pretending to "make nice" with scripted double-talk. His willingness to criticize things that he sees as wrong makes it possible (for me, and apparently you) to trust his stated opinion of things he sees as right. And I don't think sexism can be dismissed outright, if people find it safer to talk shit to a woman compared to a man. You prefaced it with an "if", but do you think it is common that "people find it safer to talk shit to a woman compared to a man"? I'd guess that at this point most male academics would be significantly more reluctant to criticise in print the work of a female colleague than that of a fellow male. I see Gelman as trying to adhere (possibly recklessly) to a gender-blind approach to criticism, and find this refreshing.
- AlexCoventry 9y ago> do you think it is common that "people find it safer to talk shit to a woman compared to a man"? I don't know whether they find it safer, but they certainly do it more, and are conditioned to do so from childhood.
- intended 9y agoA lot of the issues aren't from academics in journals, as they are from facebook groups and other places where the internet hyper-cycle of polarization and mob formation occurs.