16 ms·
When the Revolution Came for Amy Cuddy
- q123321 9y agoThe most important sentence of the article, which appears at the end: "Although 11 new papers have recently been published that do not show the downstream effects of power posing on behaviors, Cuddy is still fighting for power posing." This is an astonishing number of replications to be published on one topic in psychology. 2-3x the next highest number I heave heard of. The placement of that sentence, plus the flattering photos, leads me to suspect a publicist was able to place a thumb on the scale.
- skybrian 9y agoAre studies of "downstream effects" considered to be replications? It sounds like they are something different.
- asdf2341 9y agoYes they are considered to be replications.
- dingbat 9y agowould note that near the end of the article its noted that Cuddy has conceded that the specific cortisol effects touted have not been borne out. she is "still fighting" for power posing but only as a useful technique which sometimes works for some people in some cases, as even the replicated studies seem to show improvements in subjective measures of "confidence", whatever that means.
- RcouF1uZ4gsC 9y ago>This moment of fizziness for the discipline was already underway when Cuddy arrived at Princeton’s graduate program in 2000, transferring there to follow her adviser, Susan Fiske, with whom she first worked at the University of Massachusetts at Amherst. This is the same Susan Fiske who called people who pointed out statistical errors in psychology papers "methodological terrorists". http://andrewgelman.com/2016/09/21/what-has-happened-down-here-is-the-winds-have-changed/ http://andrewgelman.com/2016/09/21/what-has-happened-down-he... The apple does not fall far from the tree.
- deleted 9y ago[deleted]
- throwawayjava 9y agoYou mischaracterize. Fiske is not criticizing "people who pointed out statistical errors" and begins the entire essay by explicitly stating as such: "Our field has always encourages -- required, really -- peer critiques"! She is criticizing "unfiltered trash-talk [on social media]... [and] online vigilantes [who] are attacking individuals, their research programs, and their careers". It's the same tension the article is referring to, between civility and public discourse. On one hand, you'd really like an open dialog about the quality of evidence presented. On the other hand, there's a real price to be paid when people are too worried about public humiliation to risk doing science on certain questions or working with certain people. As the quote from Jay Van Bavel put it, "It's become like politics — we’ve created two camps of people who shouldn’t be in two camps in the first place" This point resonated with me even though I'm on your side of those politics. What effect would it have on the OSS community if any even slightly bad code you write could end up in a hit piece published by The Atlantic, and if fact you witnessed people on Facebook saying your colleague, who maybe did write some kinda bad code that one time but is on the whole competent, is a charlatan who doesn't deserve the title of "programmer"?
- lovich 9y agoIs that not the case anyway? I admit I have no experience contributing but that's from hearing things like how the linux development team runs and people like Sarah Sharp quitting over the abuse. It's given me the impression that if I don't create something that is up to a teams standard from commit 0 then I will be ridiculed in a public forum
- throwawayjava 9y agoAbuse from your own insular community re: a particular project is one thing; public abuse published in some of the most popular print media on the planet resulting in lost jobs/lost job opporunities is quite another.
- geofft 9y agoIt occurs to me that the replication crisis largely came into being because scientists are rewarded for publishing papers that demonstrate some effect and not for publishing papers that fail to demonstrate anything, and so the right fix would have been to fix the broken incentives. But apparently (if this article is to be believed) no such change in incentives has happened - the only thing is that there are now also incentives for demonstrating that some other study wasn't replicable. Steadily doing good science, where good science is expected not to produce results much of the time, continues to have no support. Is it any wonder that the state of science seems a little better (at least we can admit the replication crisis now) but people like Cuddy feel like they have no future in the field, that having the misfortune of the one-in-twenty chance of getting a really interesting false positive at p = 0.05 means you either have to double down on the study being right after all or you have to quit the field? Are schools actually looking at past p-hacking while continuing to demand a huge publication record for hiring? This doesn't seem like we've solved any fundamental problems. (Also, as a side note, I'd fully expect that CS is at least as vulnerable to the same thing. McSherry et al.'s COST work is a good example of something that seems far worse than p-hacking: instead of having a clearly defined hypothesis and publishing the 1-in-20 papers that falsely state it's true, as in other fields, there's nothing clearly defined in scalable systems and 20 out of 20 people who build something excessively complicated can get a paper out of it.)
- fhood 9y agoIs this http://www.frankmcsherry.org/assets/COST.pdf http://www.frankmcsherry.org/assets/COST.pdf the paper you referenced?
- geofft 9y agoYup. See also this recent followup from him: https://github.com/frankmcsherry/blog/blob/master/posts/2017-09-23.md https://github.com/frankmcsherry/blog/blob/master/posts/2017... Literally the papers are "I built a thing, and it gets faster when you do this thing to it." There's no condition on whether the base case is realistic. To be clear, I don't think the authors of the papers he's taking down are bad people, or maliciously trying to get publications for uninteresting work, or too stupid to implement things efficiently on a single core, or anything. I think they're genuinely interested in the work they're doing, but responding to the pressures of the environment that allows them to keep doing that work.
- yawaramin 9y agoI'm sure there's already been a discussion on how much of the effect of power poses is due to confirmation bias or placebo effect, as opposed to some other physiological or psychosomatic reason. Especially since in the original study the subjects weren't actually told what they were doing, but in Dr Cuddy's talks and books they explicitly are. It's just curious to me that she and Dr Fiske wouldn't catch this crucial difference immediately. Effectively it turns power poses into little more than 'the power of positive thinking'. That said, personal attacks and especially those focusing on her brain function really need to stop. Psychologists treating each other like shit is not just ironic, it's a disgrace. If they can't even behave properly with each other, how can we expect them to treat patients properly?
- 09bjb 9y agoOne note; in general, I think we tend to diminish finding when we find out that the placebo effect is partly behind them. Wrongly, in my opinion. Better reactions might be, "Wow, we take this action and our bodies/brains take care of the rest of the effect through some placebo magic? That's amazing!!"
- yawaramin 9y agoThat's a great point. No one should be able to argue with Dr Cuddy if she decides to make her living pushing this 'placebo-trigger' technique.
- tunesmith 9y agoBut doesn't pointing out a placebo effect also have power? I know the placebo effect still exists even if you're aware of it being placebo, but does it lessen? And what if it's accompanied by shaming? Meaning, say the power pose effect doesn't exist. But early on, it's exciting and new, and there's a placebo effect (which is fine!), but people also think it's real. But then studies get replicated, and there's well-known doubt on whether the effect is real in a non-placebo manner. There might still be a placebo effect, but does it shrink? And then there's a lot of controversy and the study is dragged through the mud, and power poses are mocked. Might the placebo effect be lessened even further (or obliterated) from this? Could this be true even if it was originally valid and not just placebo?
- interstitial 9y agoAndrew Gelman's take: http://andrewgelman.com/2017/10/18/beyond-power-pose-using-replication-failures-better-understanding-data-collection-analysis-better-science/ http://andrewgelman.com/2017/10/18/beyond-power-pose-using-r...
- danso 9y agoAndrew Gelman (of NYU), who is featured in the story and is criticized by Cuddy as being unfair and trollish, wrote a response: http://andrewgelman.com/2017/10/18/beyond-power-pose-using-replication-failures-better-understanding-data-collection-analysis-better-science/ http://andrewgelman.com/2017/10/18/beyond-power-pose-using-r... I thought that Gelman wasn't presented in the best of light, especially his awkward quotes about not wanting to engage in "interpersonal conflict". But he called the piece "excellent and fair to all sides" and was pretty gracious in response to Cuddy's criticisms and how the NYT portrayed him. I honestly wasn't a huge fan of the article but re-reading it after a bit of sleep, and knowing that Gelman thought it was strong, I appreciated the issues it brought up. Certainly has to be one of the longest stories about p-hacking/p-curves in mainstream media. I sympathize with Cuddy and have no doubt that among the legit attention that Gelman and other critics have directed toward her, she's faced hatemail sparked by dumb sexism, which makes it easier to tar the legitimate criticism as being sexist. I do think Cuddy needs to face legitimate criticism. She may not have been a malicious fraudster, but ignoring criticism after a long time is not much different than actively being deceptive. I also can believe that she faces hostile criticism that isn't equally distributed among all the other researchers who used weak, but previously acceptable standards. And I don't think sexism can be dismissed outright, if people find it safer to talk shit to a woman compared to a man. What I wish the story had more of was how scientific criticism is done outside of Gelman's blog. Gelman is the main scientist/research I follow on the topic. Apparently some see his snarky critique as toxic, and Gelman says he is intending to reevaluate the way he writes about things so that his point doesn't get lost in cheap jokes. How do other blogs do it? I'm somewhat sympathetic to Gelman because I perceive that there is significant institutional resistance (and few incentives) to re-examine and correct a peer review study, and being harsh/edgy/rude and beating the drum over time might be the only way to get scientists to care? But that doesn't mean his critique can't be refined for the benefit of his mission.
- skybrian 9y agoStakes matter. A million dollar poker game might be structurally identical to a game played for fun, but its effects on the people involved are very different. We wouldn't expect the players in these two games to think the same way. Talking about the stakes is in direct conflict with the scientific ideal of focusing on the science without talking about its effects on the scientists involved. For example, from Gelman's response: "I just wanted to emphasize that there’s no reason her career, or even her famous Ted talk, has to rely on a particular intriguing idea (on there being a large and predictable effect of a certain pose) that happened not to work out." Well, except that maybe she wouldn't have gotten a Ted talk without it? He's saying, hey, this paper shouldn't have been as important as it was. And according to the scientific ideal, we shouldn't get too attached to any particular results. But in practice, putting bad results aside sometimes requires a significant change of identity. Forgetting about that, even for a moment, is always going to come across as tin-eared. The scientific ideal is more compatible with low stakes than high stakes. But talking about the game as if it's low-stakes (or ignoring the stakes altogether) when actually it's high-stakes is always going to make discussions focused on the science alone come across as unsympathetic.
- breitling 9y agoWasn't this study thoroughly debunked already? The co-author of this study, Dana Carney, distanced herself from this study [1]. She doesn't believe all the claims the study made were true. [1] http://www.npr.org/2016/10/01/496093672/power-poses-co-author-i-do-not-believe-the-effects-are-real http://www.npr.org/2016/10/01/496093672/power-poses-co-autho...
- maxerickson 9y agoThe article includes much discussion of Carney.
- dingbat 9y agothe article discusses that in detail, but the article is not really about the merits, or lack thereof, of that study, its about something else.
- cratermoon 9y agoSounds like straight up sexism to me.
- recursive 9y agoPerhaps you're wearing sexism-colored glasses.
- anotherproj 9y ago> "Although 11 new papers have recently been published that do not show the downstream effects of power posing on behaviors, Cuddy is still fighting for power posing." Sounds like straight up bad science to me
- kansface 9y agoCuddy deserved neither the spoils of her research - fame and fortune, nor the indignity of having her work toppled in a public forum. She certainly did her self no favors by digging in, but at the same time, the despoilers would use her as the poster child for bad science itself. This is a story in which a few people look to be "right", but no one looks particularly flattering, which is also why its a good read. Your comment adds very little.
- sctb 9y agoIf you're going to make a claim like this, could you please also give us some information? Otherwise you're just starting a flamewar and this is not a site for those. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- hirundo 9y agoCheers for scientific revolutions being gentler than political ones. Cuddy lost her reputation rather than her head. But losing a reputation hurts more, or at least longer. More civility could have helped, but there's no painless way to lose status. The scandal is that Cuddy's work met the standard of the bulk of published social science until lately. She studied the same phlogiston with the same tools as her colleagues. But when the revolution that comes for them, gently, is an epistemic advance, their pain is our gain. Join the revolution, comrad Cuddy!
- throwawayjava 9y agoThe tragedy of the ordeal is that she tried and was lined up against the wall regardless, mostly due to lack of civility and poor communication: > But the email that Simmons and Simonsohn had sent was, in fact, ambiguous: They had explicitly told her to drop the P-curve and yet left the impression that the paper was otherwise sound. At my request, Simmons looked back at his original email. I watched as he read it over. “Oh, yeah,” he said quietly. He had a pained look on his face. “We did say to drop the graph, didn’t we?” He read it over again, then sat back. “I didn’t remember that. This may be a big misunderstanding about — that email is too polite.” Cuddy and Carney had taken their advice literally.
- conistonwater 9y ago"lined up against the wall", really? I think you would be more convincing if you toned it down a bit on the metaphors.
- throwawayjava 9y agoIt was parent's metaphor, not mine. The substance holds.
- conistonwater 9y agoI don't see it in that comment. Where? I only see "Cuddy lost her reputation rather than her head.", which is not at all similar. (I don't see it in NYT either.)
- phunge 9y agoI think it's pretty easy for this discussion to become about personalities and not methodologies. Gelman is pretty brash and speaks his mind, so he could be correct on methodology and have that get lost in the noise. So I'll give my opinion as a datascientist: null hypothesis significance testing is broken, the whole thing needs to get chucked. It's not fixable, and it's not only p-hacking that's the problem. Go read Frank Harrell, I like his writings on the topic more than Gelman's.
- anigbrowl 9y agotl;dr Promising young researcher skated on stats and talked up faulty finding, career seriously damaged by lack of replicability; reformers who are too earnest about a good cause (replication, statistical rigor) can be total assholes to well-meaning people who just make mistakes; conflict ensues. It's OK to be oppositional or confrontational to people based on what they say and do, but self-styled reformists sometimes become personally vindictive in pursuit of their own interests. Picking a fight with a public figure is an easy way to gain publicity, and if your own reputation rests on winning that fight then it can be tempting to carry on into harassment. Gamergate was an extreme example of this.
- ndh2 9y agoNot a real tl;dr.
- hprotagonist 9y agoA good working definition of a good researcher, to me, is something like "strong opinions, loosely held". I don't think Cuddy, or anyone else, deserves to be dragged through the mud because their idea turns out not to hold up. That happens all the time, and it's not a bad thing at all. You start to deserve that dragging more when you become too attached to your pet theory and cease to be able to drop it in the face of overwhelming evidence to the contrary. You really deserve it when you use what political capital you've acquired to stomp on your critics despite the fact that they're actually right and you're actually wrong. What's interesting here is that a lot of the Sayre's Law-type political machinations are taking place on a much broader stage than normal. If someone was to go after my work with this kind of exposure, you bet I'd be defensive too. It's entirely possible to have a fair and neutral conversation about research practices without shanking someone's reputation because it seems like the field needs a sacrificial lamb this week. Nobody wins, and certainly no good research is advanced, when the knives come out like this with no evidence of actual malfeasance. (And, frankly, even when there is evidence of malfeasance. like with the STAP controversy, which lead to at least one actual death.)
- dingbat 9y agohas anyone gone down the list of most popular TED Talks and run similar replications of the claims made in other presentations? itd be interesting if there were some ratio that scored most-popular TED Talk vs. "B.S." content. i wonder if there'd be a link between popularity and amount of B.S.?
- fgjjgutjvnu 9y agoNYT tries to frame it as if the rules of science were unfairly changed. Not so - people had just been overly lax for a while. And sorry, NYT, but scientific knowledge is not a matter of opinions and goodwill. We can not simply decide her theories are true after all, because we empathize with her so much.
- tomkat0789 9y agoI published a couple of papers in grad school, and what the article describes made me imagine the peer review process busted open for the world to see. Reading reviews sucks: good reviewers provide smart reasons that you're wrong, which you have to beat with more research saying that you're right. The editor makes the final decision, not always in your favor. In their "open peer review", the psychologists seemed to have all the class of a bunch of redditors. An uncomfortable process is made worse! Instead of motivating people to publish lame results for tenure, this new situation seems to motivate people to act like Internet trolls or tabloid journalists. A more critical research community should help advance the field, but these researchers set a bad example.
- ghostcluster 9y agoThe work of these social psychologists absolutely deserves the scrutiny they are receiving. The field is rife with "surprising scientific effects" that do not replicate and turn out to be completely wrong, yet they linger in the popular culture and are brought up over and over again to defend ideas and policies that are not based in reality. Look at the reach of people like Malcolm Gladwell and stuff like his books and podcast, and NPR's Hidden Brain, which subsist on recounting these "surprising studies" and speculating about their profound meaning for society, to gigantic audiences. And they're wrong! Made up ideological BS with the gravitas of 'serious science' behind them. Power posing, stereotype threat, the bystander effect, implicit bias. So many popular hypotheses have snuck into the popular culture and they are wrong. They're based on fudging numbers and statistics and throwing out results that don't fit the hypothesis. They deserve to be combated in the public sphere. They don't deserve to be shielded to protect the careers of the hucksters.
- nickbauman 9y agoHow easy it is to forget the scientific method begins only after a hypothesis is proposed.
- maxxxxx 9y agoI used to love TED talks but over the last few years I have developed an aversion. It seems a lot of popular TED people have traded scientific integrity for showmanship (and money). Makes me wonder how Feynman would be doing today. He was a charismatic talker. He could be a superstar.