4 ms·

Keeping aside the clearly political nature of the content, this study itself seems quite poorly designed & amateurish. A well designed study would mix "bullshi

by sfifs 11y ago

Keeping aside the clearly political nature of the content, this study itself seems quite poorly designed & amateurish. A well designed study would mix "bullshit" and "non-bullshit" statements of similar complexity and look at signals on both for comparison. Better still, the "bullshit" statements would be derived from mixing words from the "non-bullshit" statement pool.

wbillingsley 11y ago
Funny you should mention that, as I believe it's covered in the author's follow-on research paper "Misperceiving Bullshit as Profoud is Associated with Peer-Reviewers of Politically-Oriented plosOne Articles"
- notahacker 11y ago
  My favourite Onion-worthy line from the article was "research on bullshit is still in its infancy".
tommorris 11y ago
A much better paper came out last year that used tweets from Deepak Chopra and quotes from a random "new age wisdom" generator. PDF: http://journal.sjdm.org/15/15923a/jdm15923a.pdf http://journal.sjdm.org/15/15923a/jdm15923a.pdf And there's some followups... Dalton 2016 - http://journal.sjdm.org/15/15923ac/jdm15923ac.pdf http://journal.sjdm.org/15/15923ac/jdm15923ac.pdf Reply from Pennycook et al. - http://journal.sjdm.org/15/15923ac/jdm15923acr.pdf http://journal.sjdm.org/15/15923ac/jdm15923acr.pdf
- algorias 11y ago
  I had to chuckle at this passage from the first link: > participants were shown a list of activities (e.g., biking, reading) directly below the following instructions: “Below is a list of leisure activities. If you are reading this, please choose the “other” box below and type in ‘I read the instructions’”. This attention check proved rather difficult with 35.4% of the sample failing The perils of using poorly compensated students for your studies!
  - avs733 11y ago
    This issue isn't just with students or compensation. A lot of the quality issues have more to do with serious questions of reading comprehension and the over use of (poorly designed) survey methodologies. It ends up being a lot like the 'democracy' complaints...its not a great method but it is better than all the others for what it does. Not only are 'attention check questions' common, there is some interesting research on their effectiveness. Eventually, I suspect/hope that NLP and neural networks will enable more authentic data collection across many spectrums.
raverbashing 11y ago
"To exclude the possibility that conservatives are more likely to see profoundness in statements in general, we additionally include simple, mundane statements in our study" From the article itself

by sfifs 11y ago

wbillingsley 11y ago

Funny you should mention that, as I believe it's covered in the author's follow-on research paper "Misperceiving Bullshit as Profoud is Associated with Peer-Reviewers of Politically-Oriented plosOne Articles"

notahacker 11y ago
My favourite Onion-worthy line from the article was "research on bullshit is still in its infancy".

tommorris 11y ago

A much better paper came out last year that used tweets from Deepak Chopra and quotes from a random "new age wisdom" generator. PDF: http://journal.sjdm.org/15/15923a/jdm15923a.pdf http://journal.sjdm.org/15/15923a/jdm15923a.pdf And there's some followups... Dalton 2016 - http://journal.sjdm.org/15/15923ac/jdm15923ac.pdf http://journal.sjdm.org/15/15923ac/jdm15923ac.pdf Reply from Pennycook et al. - http://journal.sjdm.org/15/15923ac/jdm15923acr.pdf http://journal.sjdm.org/15/15923ac/jdm15923acr.pdf

algorias 11y ago
I had to chuckle at this passage from the first link: > participants were shown a list of activities (e.g., biking, reading) directly below the following instructions: “Below is a list of leisure activities. If you are reading this, please choose the “other” box below and type in ‘I read the instructions’”. This attention check proved rather difficult with 35.4% of the sample failing The perils of using poorly compensated students for your studies!
- avs733 11y ago
  This issue isn't just with students or compensation. A lot of the quality issues have more to do with serious questions of reading comprehension and the over use of (poorly designed) survey methodologies. It ends up being a lot like the 'democracy' complaints...its not a great method but it is better than all the others for what it does. Not only are 'attention check questions' common, there is some interesting research on their effectiveness. Eventually, I suspect/hope that NLP and neural networks will enable more authentic data collection across many spectrums.

raverbashing 11y ago

"To exclude the possibility that conservatives are more likely to see profoundness in statements in general, we additionally include simple, mundane statements in our study" From the article itself