5 ms·

Gelman has forged his own path to "fame, fortune, and acclaim". It's not clear to me that this path has actually improved (this particular branch of) science. I

Gelman has forged his own path to "fame, fortune, and acclaim". It's not clear to me that this path has actually improved (this particular branch of) science. In fact, by the sound of it, that path may've been at least as destructive as he claims Fiske et al's was. Which isn't an indict of Gelman, as I hardly intuit anything but positive intentions on his part. I'm less concerned with Fiske vs. Gelman and much more concerned with Jay Van Bavel's observation that either way, science loses. Confining scientific critique to scientific venues seems like a reasonable bulwark against this sort of thing. If it had been Reviewer #3 in a program committee or editorial board, or a well-received letter to the editor, instead of a blog post picked up by thousands of internet warriors with no stats training, perhaps the result could've been refined or thrown out without expelling someone from research. If the feud hadn't been picked up by pop science press, then maybe a bad piece of science could've been questioned without poisoning the well for further inquiry.

mattmcknight 9y ago
"Confining scientific critique to scientific venues" Fiske controls the venue and is (mis-)using the "scientific venue" to denounce her critics. She is the one refusing to publish the very critiques you describe. The "scientific venue" has been corrupted. The well has not been "poisoned" for further inquiry, rather the necessity of replication and sensitivity to statistical errors has increased. Science was being done sloppily, and Fiske was both a beneficiary of that and in a role where she could have prevented it.
- throwawayjava 9y ago
  > The "scientific venue" has been corrupted That's a pretty strong accusation. Do you have any evidence? Cuddy herself, in the linked article, even asks: "Why not come to a conference or hold a seminar?" I don't think you're wrong about the lack of quality reviewing, but I also think that blogs and Facebook and articles in pop sci news venues are pretty shitty places to address the quality of editorial review. And your post demonstrates why: honest mistakes -- or even complete non-mistakes stemming from gaps in established best practices! -- turn into out-right accusations of full on corruption. The comments on NYT demonstrate the same problem from the other side: lots of gender politics when Gelman wasn't motivated by gender, implicitly or otherwise, but rather by the mass appeal of Cuddy's work. Needless to say, I also think you're massively over-stating the maliciousness of people like Cuddy and Fiske. > Science was being done sloppily, and Fiske was both a beneficiary of that and in a role where she could have prevented it. Simmons seems to agree with me that you're really over-reaching here: "The original study wasn’t particularly egregious," he said. "It was published in 2010 before anyone was thinking about this." There's a huge difference between doing science the way you were trained (but sub-optimally), and outright "corruption". There's also a HUGE difference between noticing a problem and constructively pushing for more rigor within a sub-community, and going after particular people within that community so harshly you destroy their careers. The former is great for science. The latter might possibly by great for improving rigor (I'm pessimistic and think it all comes down to incentives, which Gelman et al don't really do much to change). But equivocating bad methodological/statistical training/practice with unethical and even criminal ("corrupt") behavior? That sort of dishonest fire-and-brimstone talk is just as damaging as a lack of rigor.
  - conistonwater 9y ago
    Just so you know, the basic statistical problem with this type of research was pointed out back in the 60's (http://andrewgelman.com/2016/05/06/needed-an-intellectual-history-of-research-criticism-in-psychology/ http://andrewgelman.com/2016/05/06/needed-an-intellectual-hi...), it's just that nobody listened.
    - throwawayjava 9y ago
      My point is that: 1. failing to have read Meehl's critique doesn't amount to "corruption", 2. that calling people "corrupt" for having failed to read Meehl is not likely to be the most effective or the most civil path toward changing things, and 3. To the extent that discussion of scientific methodology on public forums causes or contributes to (2), that's a bad thing and perhaps suggests a fundamental cost of doing science in this manner.
      - conistonwater 9y ago
        I think you're using a different sense of the word "corrupt" from the one in the post you're replying to ("The "scientific venue" has been corrupted."). I know it's easy to lose nuance in internet discussions, but please read the posts above more carefully/charitably. Nothing Cuddy did was corrupt in any sense, only poor science.
        throwawayjava 9y ago
        I re-read the comment, and disagree. The preceding sentence is: Fiske controls the venue and is (mis-)using the "scientific venue" to denounce her critics I'm having trouble finding a charitable way to read the claim that someone is misusing a scientific venue. And much more to the point, I doubt even more that the subject of those sentences (or the e.g. tenure/promotion committee of that subject) could be reasonably expected to apply such an abundance of charity.
        conistonwater 9y ago
lliamander 9y ago
> Confining scientific critique to scientific venues seems like a reasonable bulwark against this sort of thing. I don't think turning a call for greater scientific rigor into a political fight is desirable, but neither do I think that confining this struggle to scientific venues is desirable (or practical, for that matter). EDIT: wording fix
srtjstjsj 9y ago
If Cuddy's phony research got into TED talks and books and a lecture circuit, why should non-phony criticism be locked up in journals?
- throwawayjava 9y ago
  I vacuously agree with you. See other threads for my opinion on TED talks.
ghostcluster 9y ago
"Confining scientific critique to scientific venues" When you take your BS "studies" and sell it to the audience at TED and then the wider internet, go on corporate speaking tours, sign book deals to promote your "science", and try to affect public policy, you are open to criticism from anyone who sees through your shoddy methodology and wants to correct the record.

mattmcknight 9y ago

"Confining scientific critique to scientific venues" Fiske controls the venue and is (mis-)using the "scientific venue" to denounce her critics. She is the one refusing to publish the very critiques you describe. The "scientific venue" has been corrupted. The well has not been "poisoned" for further inquiry, rather the necessity of replication and sensitivity to statistical errors has increased. Science was being done sloppily, and Fiske was both a beneficiary of that and in a role where she could have prevented it.

throwawayjava 9y ago
> The "scientific venue" has been corrupted That's a pretty strong accusation. Do you have any evidence? Cuddy herself, in the linked article, even asks: "Why not come to a conference or hold a seminar?" I don't think you're wrong about the lack of quality reviewing, but I also think that blogs and Facebook and articles in pop sci news venues are pretty shitty places to address the quality of editorial review. And your post demonstrates why: honest mistakes -- or even complete non-mistakes stemming from gaps in established best practices! -- turn into out-right accusations of full on corruption. The comments on NYT demonstrate the same problem from the other side: lots of gender politics when Gelman wasn't motivated by gender, implicitly or otherwise, but rather by the mass appeal of Cuddy's work. Needless to say, I also think you're massively over-stating the maliciousness of people like Cuddy and Fiske. > Science was being done sloppily, and Fiske was both a beneficiary of that and in a role where she could have prevented it. Simmons seems to agree with me that you're really over-reaching here: "The original study wasn’t particularly egregious," he said. "It was published in 2010 before anyone was thinking about this." There's a huge difference between doing science the way you were trained (but sub-optimally), and outright "corruption". There's also a HUGE difference between noticing a problem and constructively pushing for more rigor within a sub-community, and going after particular people within that community so harshly you destroy their careers. The former is great for science. The latter might possibly by great for improving rigor (I'm pessimistic and think it all comes down to incentives, which Gelman et al don't really do much to change). But equivocating bad methodological/statistical training/practice with unethical and even criminal ("corrupt") behavior? That sort of dishonest fire-and-brimstone talk is just as damaging as a lack of rigor.
- conistonwater 9y ago
  Just so you know, the basic statistical problem with this type of research was pointed out back in the 60's (http://andrewgelman.com/2016/05/06/needed-an-intellectual-history-of-research-criticism-in-psychology/ http://andrewgelman.com/2016/05/06/needed-an-intellectual-hi...), it's just that nobody listened.
  - throwawayjava 9y ago
    My point is that: 1. failing to have read Meehl's critique doesn't amount to "corruption", 2. that calling people "corrupt" for having failed to read Meehl is not likely to be the most effective or the most civil path toward changing things, and 3. To the extent that discussion of scientific methodology on public forums causes or contributes to (2), that's a bad thing and perhaps suggests a fundamental cost of doing science in this manner.
    - conistonwater 9y ago
      I think you're using a different sense of the word "corrupt" from the one in the post you're replying to ("The "scientific venue" has been corrupted."). I know it's easy to lose nuance in internet discussions, but please read the posts above more carefully/charitably. Nothing Cuddy did was corrupt in any sense, only poor science.
      - throwawayjava 9y ago
        I re-read the comment, and disagree. The preceding sentence is: Fiske controls the venue and is (mis-)using the "scientific venue" to denounce her critics I'm having trouble finding a charitable way to read the claim that someone is misusing a scientific venue. And much more to the point, I doubt even more that the subject of those sentences (or the e.g. tenure/promotion committee of that subject) could be reasonably expected to apply such an abundance of charity.
        conistonwater 9y ago

lliamander 9y ago

> Confining scientific critique to scientific venues seems like a reasonable bulwark against this sort of thing. I don't think turning a call for greater scientific rigor into a political fight is desirable, but neither do I think that confining this struggle to scientific venues is desirable (or practical, for that matter). EDIT: wording fix

srtjstjsj 9y ago

If Cuddy's phony research got into TED talks and books and a lecture circuit, why should non-phony criticism be locked up in journals?

throwawayjava 9y ago
I vacuously agree with you. See other threads for my opinion on TED talks.

ghostcluster 9y ago

"Confining scientific critique to scientific venues" When you take your BS "studies" and sell it to the audience at TED and then the wider internet, go on corporate speaking tours, sign book deals to promote your "science", and try to affect public policy, you are open to criticism from anyone who sees through your shoddy methodology and wants to correct the record.