3 ms·

You should indeed be extremely skeptical of studies whose only dependent variable is a voluntary participant response and where the participant has an incentive

You should indeed be extremely skeptical of studies whose only dependent variable is a voluntary participant response and where the participant has an incentive to tell the experimenter what they think they want to hear (for example, to make the experiment end faster).

kenjackson 3y ago
Except in this case for the follow up interviews there was no incentive to lie.
- canjobear 3y ago
  They’re incentivized to tell the experimenters what they want to hear so they can get paid and go home faster.
  - somenameforme 3y ago
    Having participated in plenty of these (as it was required for class credit, and they also paid reasonably well for an in-between classes hour type thing to go do), the 'researchers' can also be completely obnoxious. One study I was involved in had me listen to some rant about how women do better on a math exam I was about to take, being left in a room for about 10 minutes alone, and then being given the exam - on pink paper. It was generic algebra 2 / SAT level test. After the exam the researchers just endlessly hounded me from a hundred different angles trying to get me to express doubt or uncertainty in how I performed on it. If I just wanted my $20 and credit, it would have been far easier to just give them what they wanted. But I've been ever the contrarian. They also said they'd email me my results in 8 weeks. They did not. I've always wondered if I ended up getting culled from their 'study' as an 'outlier.'
    - AlbertCory 3y ago
      Funny, but my one and only contact with PLATO when I was at the U of I was for a Psych 100 experiment like this.
    - kenjackson 3y ago
      That just sounds like bad research. In general if you know their angle that clearly then they’ve failed at how to do a research study of this type — obnoxious or not. It sounds like they don’t know how to do this type of research.
      - somenameforme 3y ago
        I'm not sure that was the problem. Journals don't tend to publish negative results, and social science journals seem to have 0 issues with publishing studies that are overtly designed to confirm a hypothesis, rather than challenge it. Add in a bit of publish or perish, and this behavior is ubiquitous. What would you say is an example of good research in social psychology? As an example of bad science, you look to what's likely the single most well known experiment in social psychology - the Stanford Prisoner Experiment. It's not only failed to replicate, but was mostly entirely fabricated. [1] The famous clip of the guy screaming out 'I can’t stand another night! I just can’t take it anymore!' came from a guy who was intentionally faking a meltdown, because he needed to go study and they weren't allowing him access to his books, the guards were actively trained to be cruel, and more. Incidentally Zimbardo (the experimenter), far from being shunned, has received countless accolades for this experiment, some as recent as 2012. [1] - https://gen.medium.com/the-lifespan-of-a-lie-d869212b1f62 https://gen.medium.com/the-lifespan-of-a-lie-d869212b1f62
  - anigbrowl 3y ago
    In most psychological experiments, volunteers are told it will take an hour/an afternoon/whatever at the time they sign up to participate. The notion that all of them actually want to cut corners and just obtain the cash in return for the bare minimum of time and effort is an assumption about people in general, not a fact.
  - kenjackson 3y ago
    In the follow ups they are simply asked to recount and give their confidence of recounting. It’s only the original interview they are told to try to remember details even if they can’t. The premise of the study is that the first interview is used to plant the memories. The follow up interviews are to see if they stick. They specifically aren’t trying to intervene in those interviews.
Timon3 3y ago
How would you propose, in this specific study, they verify what you want? Please be specific regarding what questions you want asked, when they should be asked, and how these questions can't be discarded by saying "well, they could be faking it".
- canjobear 3y ago
  I think this whole experimental paradigm is inherently unfixable.
  - Timon3 3y ago
    So we just stop studying psychology completely?
    - swayvil 3y ago
      You can't trust psychology because people sometimes lie. You can't trust physics because you might be hallucinating. What's left? Authoritarianism? Consensus? Tradition?
    - canjobear 3y ago
      No, but the focus for studies like this should be on nonvoluntary responses like reaction times and EEG in my opinion.
      - Timon3 3y ago
        How do you know that the EEG monitoring doesn't influence people? Couldn't they subconsciously fake it due to being actively monitored?
    - cdogl 3y ago
      Or the people undertaking these studies (and more importantly, the people peer reviewing and publishing the work) push back on broad conclusions that aren't substantiated by the research. Having seen the nonsense my partner has to put up with in order to get a post-graduate qualification to practice as a professional psychologist I'm not holding my breath. From where I'm sitting the replication crisis in this field looks like a crisis of incompetence and apathy.
darkerside 3y ago
Is it that hard to imagine that teens and the mentally impaired would do the same, focusing on resolving the present conflict (locked in room) over a future hypothetical problem (locked in jail)?

kenjackson 3y ago

Except in this case for the follow up interviews there was no incentive to lie.

canjobear 3y ago
They’re incentivized to tell the experimenters what they want to hear so they can get paid and go home faster.
- somenameforme 3y ago
  Having participated in plenty of these (as it was required for class credit, and they also paid reasonably well for an in-between classes hour type thing to go do), the 'researchers' can also be completely obnoxious. One study I was involved in had me listen to some rant about how women do better on a math exam I was about to take, being left in a room for about 10 minutes alone, and then being given the exam - on pink paper. It was generic algebra 2 / SAT level test. After the exam the researchers just endlessly hounded me from a hundred different angles trying to get me to express doubt or uncertainty in how I performed on it. If I just wanted my $20 and credit, it would have been far easier to just give them what they wanted. But I've been ever the contrarian. They also said they'd email me my results in 8 weeks. They did not. I've always wondered if I ended up getting culled from their 'study' as an 'outlier.'
  - AlbertCory 3y ago
    Funny, but my one and only contact with PLATO when I was at the U of I was for a Psych 100 experiment like this.
  - kenjackson 3y ago
    That just sounds like bad research. In general if you know their angle that clearly then they’ve failed at how to do a research study of this type — obnoxious or not. It sounds like they don’t know how to do this type of research.
    - somenameforme 3y ago
      I'm not sure that was the problem. Journals don't tend to publish negative results, and social science journals seem to have 0 issues with publishing studies that are overtly designed to confirm a hypothesis, rather than challenge it. Add in a bit of publish or perish, and this behavior is ubiquitous. What would you say is an example of good research in social psychology? As an example of bad science, you look to what's likely the single most well known experiment in social psychology - the Stanford Prisoner Experiment. It's not only failed to replicate, but was mostly entirely fabricated. [1] The famous clip of the guy screaming out 'I can’t stand another night! I just can’t take it anymore!' came from a guy who was intentionally faking a meltdown, because he needed to go study and they weren't allowing him access to his books, the guards were actively trained to be cruel, and more. Incidentally Zimbardo (the experimenter), far from being shunned, has received countless accolades for this experiment, some as recent as 2012. [1] - https://gen.medium.com/the-lifespan-of-a-lie-d869212b1f62 https://gen.medium.com/the-lifespan-of-a-lie-d869212b1f62
- anigbrowl 3y ago
  In most psychological experiments, volunteers are told it will take an hour/an afternoon/whatever at the time they sign up to participate. The notion that all of them actually want to cut corners and just obtain the cash in return for the bare minimum of time and effort is an assumption about people in general, not a fact.
- kenjackson 3y ago
  In the follow ups they are simply asked to recount and give their confidence of recounting. It’s only the original interview they are told to try to remember details even if they can’t. The premise of the study is that the first interview is used to plant the memories. The follow up interviews are to see if they stick. They specifically aren’t trying to intervene in those interviews.

Timon3 3y ago

How would you propose, in this specific study, they verify what you want? Please be specific regarding what questions you want asked, when they should be asked, and how these questions can't be discarded by saying "well, they could be faking it".

canjobear 3y ago
I think this whole experimental paradigm is inherently unfixable.
- Timon3 3y ago
  So we just stop studying psychology completely?
  - swayvil 3y ago
    You can't trust psychology because people sometimes lie. You can't trust physics because you might be hallucinating. What's left? Authoritarianism? Consensus? Tradition?
  - canjobear 3y ago
    No, but the focus for studies like this should be on nonvoluntary responses like reaction times and EEG in my opinion.
    - Timon3 3y ago
      How do you know that the EEG monitoring doesn't influence people? Couldn't they subconsciously fake it due to being actively monitored?
  - cdogl 3y ago
    Or the people undertaking these studies (and more importantly, the people peer reviewing and publishing the work) push back on broad conclusions that aren't substantiated by the research. Having seen the nonsense my partner has to put up with in order to get a post-graduate qualification to practice as a professional psychologist I'm not holding my breath. From where I'm sitting the replication crisis in this field looks like a crisis of incompetence and apathy.

darkerside 3y ago

Is it that hard to imagine that teens and the mentally impaired would do the same, focusing on resolving the present conflict (locked in room) over a future hypothetical problem (locked in jail)?