3 ms·
I enjoyed the target paper in question that Gelman was posting about (as is common with Gelman, you have to wade through his comments to find the work of others
by derbOac 5y ago
I enjoyed the target paper in question that Gelman was posting about (as is common with Gelman, you have to wade through his comments to find the work of others buried in it). It was interesting and makes a strong case for use of more interesting controls in behavioral research.
After I thought about it for a bit though, I had some concerns. I read another paper on a totally different topic where they generated a sort of negative control, and although that paper was interesting too, a problem with it was that it was really easy to argue that the negative control wasn't actually a control, that it was set up in such a way as to inadvertently induce the thing they were trying to eliminate.
I kinda wondered about this paper too. They seem to assume you'll go along with the idea that how you imagine a video game character to act will induce the most extreme effect size you could imagine. But is how you report you'll act even remotely comparable to how you imagine a video game character will react? I'm not really sure. I could imagine some people being like "WTH? I have no idea" and blowing the task off, or seeing the video game character as incoherent and having any kind of behavior consistent with it. People's own behavior can be difficult to predict from what they say, so it seems like a step even further removed to put any weight into what they say a videogame character would do.
I guess it's an interesting idea that deserves more attention, but in my experience, psychological controls are notoriously difficult to implement, which is why you don't see it used more often.