4 ms·

An important caveat for Psychology studies is to ask, "If the participants are told exactly what is being measured, do the results stay the same?" If the answer

An important caveat for Psychology studies is to ask, "If the participants are told exactly what is being measured, do the results stay the same?" If the answer is "No" you should question the validity of the study. Here for example, if you tell people that "intellectual humility; taking the perspective of others; recognising uncertainty; and having the capacity to search for a compromise." all lead to "emotional wellbeing, and relationship satisfaction" just prior to taking the test, would that effect size be similar to intentionally omitting that information and having people journal for four weeks? As far as I can tell there was no informed control cohort in this study.

nine_k 7y ago
A lot of psychological studies measure effects invisible to the participants, often masking the real experiment as a technicality of a fake experiment that the participants know about. It is well-known that "When a measure becomes a target, it ceases to be a good measure."
- dgellow 7y ago
  > often masking the real experiment as a technicality of a fake experiment that the participants know about That’s a common trope, but as far as I understand, based on discussions with friends who are in the domain, studies based on deception will in general be blocked by ethic committees (at least in Germany). Don’t quote me though, I’m not in psychology myself, and may not know what I’m talking about here :)
  - Shoop 7y ago
    Based on what my friend (who is running a small psychology study) told me, non harmful deception is allowed so long as you explain the deception in the debriefing of your experiment. This is in the US.
    - acjohnson55 7y ago
      Can confirm, as someone who has been a voluntary subject a couple times. But it's kind of weird, because deceptive design is so typical that I find myself speculating what the real study is while being studied.
      - dgellow 7y ago
        > I find myself speculating what the real study is while being studied. Faire point. I’m wondering how much studies are impacted by that in reality. Is there a known way to kind of estimate/mesure?
  - jpfed 7y ago
    Deception in various forms was incredibly common in U.S. studies when I was majoring in psych. Even if you didn't say anything like "We're studying weight loss" when actually you were studying anxiety, you might just not talk much about what the study is about while including tasks that lead people to naturally think the study is about X when really it's about Y.
wnmurphy 7y ago
Except that merely being aware of one of your own processes can definitely alter it. If you think about your blink rate, for example.
Valmar 7y ago
I wonder how often this happens in Psychology studies. Psychology, currently, isn't in a good position regarding repeatability for most of its studies.
shkkmo 7y ago
> "If the participants are told exactly what is being measured, do the results stay the same?" If the answer is "No" you should question the validity of the study. What is your basis for your assertion? This does not match with what I understand about experimental design.

nine_k 7y ago

A lot of psychological studies measure effects invisible to the participants, often masking the real experiment as a technicality of a fake experiment that the participants know about. It is well-known that "When a measure becomes a target, it ceases to be a good measure."

dgellow 7y ago
> often masking the real experiment as a technicality of a fake experiment that the participants know about That’s a common trope, but as far as I understand, based on discussions with friends who are in the domain, studies based on deception will in general be blocked by ethic committees (at least in Germany). Don’t quote me though, I’m not in psychology myself, and may not know what I’m talking about here :)
- Shoop 7y ago
  Based on what my friend (who is running a small psychology study) told me, non harmful deception is allowed so long as you explain the deception in the debriefing of your experiment. This is in the US.
  - acjohnson55 7y ago
    Can confirm, as someone who has been a voluntary subject a couple times. But it's kind of weird, because deceptive design is so typical that I find myself speculating what the real study is while being studied.
    - dgellow 7y ago
      > I find myself speculating what the real study is while being studied. Faire point. I’m wondering how much studies are impacted by that in reality. Is there a known way to kind of estimate/mesure?
- jpfed 7y ago
  Deception in various forms was incredibly common in U.S. studies when I was majoring in psych. Even if you didn't say anything like "We're studying weight loss" when actually you were studying anxiety, you might just not talk much about what the study is about while including tasks that lead people to naturally think the study is about X when really it's about Y.

wnmurphy 7y ago

Except that merely being aware of one of your own processes can definitely alter it. If you think about your blink rate, for example.

Valmar 7y ago

I wonder how often this happens in Psychology studies. Psychology, currently, isn't in a good position regarding repeatability for most of its studies.

shkkmo 7y ago

> "If the participants are told exactly what is being measured, do the results stay the same?" If the answer is "No" you should question the validity of the study. What is your basis for your assertion? This does not match with what I understand about experimental design.