8 ms·

I think this is a fantastic example of how a) anyone can call bullshit on science if it doesn't make sense and b) there are more and less correct ways of callin

I think this is a fantastic example of how a) anyone can call bullshit on science if it doesn't make sense and b) there are more and less correct ways of calling bullshit on established science. You don't need to hop on a podcast or radio show in order to question the orthodoxy of a scientific claim, and in fact you shouldn't if you want to be taken seriously. Nick Brown would have hurt his case had he done that, and instead found a path that actually knocked down a faulty concept in a way that has lasting effects. This is the process working. This is all okay, and welcome, IMO, though possibly not by everyone involved.

jvanderbot 5y ago
I agree. The real shining light here is that a determined amateur was able to get free, volunteer help from inside the community to weather the publication process. Those who read the review / revision process in this article and were dismayed / convinced the process was flawed should realize this is precisely how review works in many fields. It takes a year to get a paper reviewed and revised, and a "25 page" (TFA) statement of revision is par. Not "all" will have the stomach for that, but once you are part of that process, it's normal.
- horsawlarway 5y ago
  You consider having to threaten the reputation of an academic journal by emailing the CEO directly before you even make it to the review for a well written, concise, critical paper "part of that process"? Not even mentioning that it required the publicity and reputation of a well known figure (Sokal) to even open that door for them. If that's the "shining light" of this industry - I sure as fuck don't want to see the dirty alleyway.
  - jvanderbot 5y ago
    I didn't say that. I said the shining light was insiders helping. I also said the peer review process is challenging and that was an accurate representation. Two different things.
    - native_samples 5y ago
      Is Sokal really an "insider" to psychology?
      - horsawlarway 5y ago
        I'd argue no. He's a mathematician and physicist. You could claim that Harris Friedman is an insider - although I don't find a nearly retired (at the time - now actually retired) college professor to be particularly "inside" the journal space, but he is certainly a member of the field. The people who were insiders (Barbara Fredrickson and the reviewers at American Psychologist) are decidedly unhelpful and uninterested outside of throwing folks under the bus and equivocating around how so much fucking fraud/bullshit ended up in their papers.
nostromo 5y ago
The people that publish and promote bullshit science spend a lot of time on podcasts, so I don't see why the people falsifying their research shouldn't as well.
bonoboTP 5y ago
> This is the process working. This is all okay, and welcome No! This sort of thing casts enormous doubt on the field of psychology and many soft sciences as a whole. A lot of it is academic politics and people being afraid of calling things out in fear of hurting their career prospects. It's much easier to just stay silent, ignore it, and play along. Who knows who will sit on your next grant proposal's committee. Academia is full of this and it's a disgrace.
- FredPret 5y ago
  I, for one, will never trust another “finding” coming out of the field of psychology. Between this and the marshmallow study, I think psychology should be considered an art. We’re many centuries away from this progressing to being a science on par with physics. There’s nothing wrong with that, a wise therapist has much to contribute - but not to science.
  - lotsofpulp 5y ago
    This xkcd is relevant: https://xkcd.com/435/ https://xkcd.com/435/ Unless the variables in the experiment are decently quantifiable. It is a garbage in, garbage out situation.
    - FredPret 5y ago
      The problem with psychology is: the subject matter is the most complex system we know of. In order to make definitive statements about said system, you need to start with the most basic behaviours and characteristics. As you work your way up, you’ll mountains of data to capture every permutation of human behaviour and emotion. We’re so, so far from any of that.
      - naasking 5y ago
        The messier the system, the more rigorous the methodology should be. Pre-registering trials to fight p-hacking, and open data should be the norm. That's decidedly not what we've gotten from this field, and some prominent figures have even been fighting such measures and consider the reproducibility crisis to not be a big deal. That's the problem.
hex4def6 5y ago
I'm not sure I agree. In his own words, he had to ship around for a respected authority so that it would be taken seriously. Once he had that, the journal have a bullshit response that it was passed two months since the original article was published, and so not eligible for publication. Sokal through his connections, went to the CEO to get that decision over ruled (edit: by threatening that the journal would be humiliated by refusing to publish it). The original debunker would never have got that far on his own. To me, this sounds like a process that is far from some sort of egalitarian "judge ideas on their merit rather than the person presenting them." If he hasn't got Sokal involved, I wonder if this would have gotten published at all. Edit: also, this casts a pretty negative light on the field, given that it's been cited 350+ times in this field. Not one of those other papers dug into the content (or understood it) enough to question it, and/or raise the alarm bells. It's one thing Its one thing if this was some obscure paper that never got eyes on it, but that is obviously not the case.
- jokethrowaway 5y ago
  Science can only be as good as the people doing it. The problem is that most papers, especially on very complicated topics, are not science and just serve the purpose of proving a point or helping someone's career. Even without taking easy jabs at psychology (you would need full brain understanding of its inner workings and evolution over a lifetime, to have a 100% clear picture of what's happening - we probably have 0.0001% - it's all based on observations and interpretations), the reproducibility crisis is affecting a lot of areas. Hopefully with more time and more people getting involved in the field, we'll get more and more science. If I think about nutrition / fitness, between 40-20 years ago and now we made great progress: we went from "Fats are bad and make you fat" (pure propaganda to sell processed sugars), "BULK on a 4000 diet" to any video of Jeff Nippard quoting incredibly detailed studies about minutiae of nutrition and building muscle.
  - deleted 5y ago
    [deleted]
- kurthr 5y ago
  The journal publishers (similar to FB et al) are commercial entities designed to make money from desperate academics trying to publish. They thrive on novelty and readership, but also "respect" (rather than ads and "engagement") from not having their occasionally outrageous papers questioned. Sometimes to stoke or avoid scandal and increase readership they might include a debunking paper, but it's important to remember these are not bought by individuals but by departments who won't stock "less respected" journals. Now, if perhaps there were a respected public psychology paper service like psyarXive at the time that might have made things slightly better. https://psyarxiv.com/ https://psyarxiv.com/
tdrgabi 5y ago
I understand what you mean, but I see a series of lucky moments, that all had to happen for the paper to be published. Sokal picked it up, even if usually he doesn't. The journal rejected it and they had to extra push it etc. You can see how many rebutals, done without podcasts are never heard of when going through the oficial path.
- gjvc 5y ago
  I understand what you mean, but I see a series of lucky moments, that all had to happen for the paper to be published. The role of luck is often under-estimated.
  - CRConrad 5y ago
    > The role of luck is often under-estimated. Then so is the role of bad luck. Betcha there are more papers that don't have the luck this one did than there are that do, donchathink?
pasquinelli 5y ago
that's an odd interpretation. this seems less like a system doing what it's supposed to do and correcting itself and more like an instance of that system failing to do what it's supposed to do and correcting itself.
endgame 5y ago
The article describes events set into motion over a decade ago. People calling bullshit on things that happened over the past couple of years have had a much worse time of it.
emilfihlman 5y ago
This is absolutely not the process working. It took 3 people months of work to debunk a claim. These resources just don't exist in reasonable numbers and we absolutely need short, piercing, direct call outs that quickly demonstrate that fuckery is going on. Feelings are the worst thing that has happened to science.

jvanderbot 5y ago

I agree. The real shining light here is that a determined amateur was able to get free, volunteer help from inside the community to weather the publication process. Those who read the review / revision process in this article and were dismayed / convinced the process was flawed should realize this is precisely how review works in many fields. It takes a year to get a paper reviewed and revised, and a "25 page" (TFA) statement of revision is par. Not "all" will have the stomach for that, but once you are part of that process, it's normal.

horsawlarway 5y ago
You consider having to threaten the reputation of an academic journal by emailing the CEO directly before you even make it to the review for a well written, concise, critical paper "part of that process"? Not even mentioning that it required the publicity and reputation of a well known figure (Sokal) to even open that door for them. If that's the "shining light" of this industry - I sure as fuck don't want to see the dirty alleyway.
- jvanderbot 5y ago
  I didn't say that. I said the shining light was insiders helping. I also said the peer review process is challenging and that was an accurate representation. Two different things.
  - native_samples 5y ago
    Is Sokal really an "insider" to psychology?
    - horsawlarway 5y ago
      I'd argue no. He's a mathematician and physicist. You could claim that Harris Friedman is an insider - although I don't find a nearly retired (at the time - now actually retired) college professor to be particularly "inside" the journal space, but he is certainly a member of the field. The people who were insiders (Barbara Fredrickson and the reviewers at American Psychologist) are decidedly unhelpful and uninterested outside of throwing folks under the bus and equivocating around how so much fucking fraud/bullshit ended up in their papers.

nostromo 5y ago

The people that publish and promote bullshit science spend a lot of time on podcasts, so I don't see why the people falsifying their research shouldn't as well.

bonoboTP 5y ago

> This is the process working. This is all okay, and welcome No! This sort of thing casts enormous doubt on the field of psychology and many soft sciences as a whole. A lot of it is academic politics and people being afraid of calling things out in fear of hurting their career prospects. It's much easier to just stay silent, ignore it, and play along. Who knows who will sit on your next grant proposal's committee. Academia is full of this and it's a disgrace.

FredPret 5y ago
I, for one, will never trust another “finding” coming out of the field of psychology. Between this and the marshmallow study, I think psychology should be considered an art. We’re many centuries away from this progressing to being a science on par with physics. There’s nothing wrong with that, a wise therapist has much to contribute - but not to science.
- lotsofpulp 5y ago
  This xkcd is relevant: https://xkcd.com/435/ https://xkcd.com/435/ Unless the variables in the experiment are decently quantifiable. It is a garbage in, garbage out situation.
  - FredPret 5y ago
    The problem with psychology is: the subject matter is the most complex system we know of. In order to make definitive statements about said system, you need to start with the most basic behaviours and characteristics. As you work your way up, you’ll mountains of data to capture every permutation of human behaviour and emotion. We’re so, so far from any of that.
    - naasking 5y ago
      The messier the system, the more rigorous the methodology should be. Pre-registering trials to fight p-hacking, and open data should be the norm. That's decidedly not what we've gotten from this field, and some prominent figures have even been fighting such measures and consider the reproducibility crisis to not be a big deal. That's the problem.

hex4def6 5y ago

I'm not sure I agree. In his own words, he had to ship around for a respected authority so that it would be taken seriously. Once he had that, the journal have a bullshit response that it was passed two months since the original article was published, and so not eligible for publication. Sokal through his connections, went to the CEO to get that decision over ruled (edit: by threatening that the journal would be humiliated by refusing to publish it). The original debunker would never have got that far on his own. To me, this sounds like a process that is far from some sort of egalitarian "judge ideas on their merit rather than the person presenting them." If he hasn't got Sokal involved, I wonder if this would have gotten published at all. Edit: also, this casts a pretty negative light on the field, given that it's been cited 350+ times in this field. Not one of those other papers dug into the content (or understood it) enough to question it, and/or raise the alarm bells. It's one thing Its one thing if this was some obscure paper that never got eyes on it, but that is obviously not the case.

jokethrowaway 5y ago
Science can only be as good as the people doing it. The problem is that most papers, especially on very complicated topics, are not science and just serve the purpose of proving a point or helping someone's career. Even without taking easy jabs at psychology (you would need full brain understanding of its inner workings and evolution over a lifetime, to have a 100% clear picture of what's happening - we probably have 0.0001% - it's all based on observations and interpretations), the reproducibility crisis is affecting a lot of areas. Hopefully with more time and more people getting involved in the field, we'll get more and more science. If I think about nutrition / fitness, between 40-20 years ago and now we made great progress: we went from "Fats are bad and make you fat" (pure propaganda to sell processed sugars), "BULK on a 4000 diet" to any video of Jeff Nippard quoting incredibly detailed studies about minutiae of nutrition and building muscle.
- deleted 5y ago
  [deleted]
kurthr 5y ago
The journal publishers (similar to FB et al) are commercial entities designed to make money from desperate academics trying to publish. They thrive on novelty and readership, but also "respect" (rather than ads and "engagement") from not having their occasionally outrageous papers questioned. Sometimes to stoke or avoid scandal and increase readership they might include a debunking paper, but it's important to remember these are not bought by individuals but by departments who won't stock "less respected" journals. Now, if perhaps there were a respected public psychology paper service like psyarXive at the time that might have made things slightly better. https://psyarxiv.com/ https://psyarxiv.com/

tdrgabi 5y ago

I understand what you mean, but I see a series of lucky moments, that all had to happen for the paper to be published. Sokal picked it up, even if usually he doesn't. The journal rejected it and they had to extra push it etc. You can see how many rebutals, done without podcasts are never heard of when going through the oficial path.

gjvc 5y ago
I understand what you mean, but I see a series of lucky moments, that all had to happen for the paper to be published. The role of luck is often under-estimated.
- CRConrad 5y ago
  > The role of luck is often under-estimated. Then so is the role of bad luck. Betcha there are more papers that don't have the luck this one did than there are that do, donchathink?

pasquinelli 5y ago

that's an odd interpretation. this seems less like a system doing what it's supposed to do and correcting itself and more like an instance of that system failing to do what it's supposed to do and correcting itself.

endgame 5y ago

The article describes events set into motion over a decade ago. People calling bullshit on things that happened over the past couple of years have had a much worse time of it.

emilfihlman 5y ago

This is absolutely not the process working. It took 3 people months of work to debunk a claim. These resources just don't exist in reasonable numbers and we absolutely need short, piercing, direct call outs that quickly demonstrate that fuckery is going on. Feelings are the worst thing that has happened to science.