3 ms·
A much better paper came out last year that used tweets from Deepak Chopra and quotes from a random "new age wisdom" generator. PDF: http://journal.sjdm.org/15
by tommorris 10y ago
A much better paper came out last year that used tweets from Deepak Chopra and quotes from a random "new age wisdom" generator.
PDF: http://journal.sjdm.org/15/15923a/jdm15923a.pdf http://journal.sjdm.org/15/15923a/jdm15923a.pdf
And there's some followups...
Dalton 2016 - http://journal.sjdm.org/15/15923ac/jdm15923ac.pdf http://journal.sjdm.org/15/15923ac/jdm15923ac.pdf
Reply from Pennycook et al. - http://journal.sjdm.org/15/15923ac/jdm15923acr.pdf http://journal.sjdm.org/15/15923ac/jdm15923acr.pdf
- algorias 10y agoI had to chuckle at this passage from the first link: > participants were shown a list of activities (e.g., biking, reading) directly below the following instructions: “Below is a list of leisure activities. If you are reading this, please choose the “other” box below and type in ‘I read the instructions’”. This attention check proved rather difficult with 35.4% of the sample failing The perils of using poorly compensated students for your studies!
- avs733 10y agoThis issue isn't just with students or compensation. A lot of the quality issues have more to do with serious questions of reading comprehension and the over use of (poorly designed) survey methodologies. It ends up being a lot like the 'democracy' complaints...its not a great method but it is better than all the others for what it does. Not only are 'attention check questions' common, there is some interesting research on their effectiveness. Eventually, I suspect/hope that NLP and neural networks will enable more authentic data collection across many spectrums.