3 ms·

Also in psychological evaluations. A questionnaire may include items that people pretending to have some condition think they should answer a certain way, but w

Also in psychological evaluations. A questionnaire may include items that people pretending to have some condition think they should answer a certain way, but would be untrue in (most) actual incidences of that condition.

gruez 6y ago
This would be easily foiled by reading the wikipedia page of the condition beforehand, right?
- HelloNurse 6y ago
  Such advanced cheating would require enough intelligence, mindfulness and self control, if not psychology competence, that it can be considered a proxy indicator that the subject is more or less OK.
  - DrewRWx 6y ago
    The real Catch-22.
  - metiscus 6y ago
    The problem would then be catching those cheaters.
jimmaswell 6y ago
What are some examples?
- derbOac 6y ago
  Depends on the domain but in general it relies on item responses that people in general tend to think are typical of some "condition" (in a broad sense of the term) but actually are not. This requires a lot of data collection usually. So, for example, items that people tend to think would be incorrectly responded to by people with dementia, but are actually easy for people with dementia to respond to. Or items that sound like they'd be endorsed by people with severe mental illness but are not. In more sophisticated implementations, it's not just responses to these types of items, but multivariate patterns of responses that are unusual that are flagged. So, for example, not getting items increasingly incorrect with increasing difficulty (i.e., getting easy items incorrect and hard items correct) might be flagged, just for one example. Specific examples are hard to list because (a) it would violate test copyright and/or other ethical issues, and (b) are sort of idiosyncratic and require actual data on the items. With cognitive testing for example, lots of times certain types of memory are intact in severe brain injury but people who are faking don't realize it.
derbOac 6y ago
Actually this is kind of telling with regard to the "brown M&M principle." The the items in an effort/response validity measure reflect some kind of systematic schema on the part of respondents. It's very non-arbitrary. The brown M&Ms in contrast are sort of trivial and arbitrary. That is, there's nothing about M&Ms that has anything to do with actual stage safety or management, and the assumption that attention to M&Ms and stage safety are the same is kind of odd. In fact, one might argue that the brown M&M clause pulls for a sort of cargo cult phenomenon, where people are following something to the letter without regard to deeper principles or understanding of what's important or not. Sometimes psychological tests do have items that test for random responding, which is maybe more comparable. But even then I don't think it's the same, because it's still item responses that are critical. To me the brown M&M test is like looking at the response time to an initial consent page, and arguing that if that is unusually long the person probably isn't responding to the items per se. I think there's maybe something to the brown M&M test but to me it pulls for the wrong things. Why not just actually, you know, make sure they abide by the long list of supposedly safety-critical items that you specified? If you as the performer are so sure that they're safety critical, but can't be bothered to check those, and instead rely on brown M&Ms, because it's easy to check in your dressing room, who's the one being lax about the stage?

gruez 6y ago

This would be easily foiled by reading the wikipedia page of the condition beforehand, right?

HelloNurse 6y ago
Such advanced cheating would require enough intelligence, mindfulness and self control, if not psychology competence, that it can be considered a proxy indicator that the subject is more or less OK.
- DrewRWx 6y ago
  The real Catch-22.
- metiscus 6y ago
  The problem would then be catching those cheaters.

jimmaswell 6y ago

What are some examples?

derbOac 6y ago
Depends on the domain but in general it relies on item responses that people in general tend to think are typical of some "condition" (in a broad sense of the term) but actually are not. This requires a lot of data collection usually. So, for example, items that people tend to think would be incorrectly responded to by people with dementia, but are actually easy for people with dementia to respond to. Or items that sound like they'd be endorsed by people with severe mental illness but are not. In more sophisticated implementations, it's not just responses to these types of items, but multivariate patterns of responses that are unusual that are flagged. So, for example, not getting items increasingly incorrect with increasing difficulty (i.e., getting easy items incorrect and hard items correct) might be flagged, just for one example. Specific examples are hard to list because (a) it would violate test copyright and/or other ethical issues, and (b) are sort of idiosyncratic and require actual data on the items. With cognitive testing for example, lots of times certain types of memory are intact in severe brain injury but people who are faking don't realize it.

derbOac 6y ago

Actually this is kind of telling with regard to the "brown M&M principle." The the items in an effort/response validity measure reflect some kind of systematic schema on the part of respondents. It's very non-arbitrary. The brown M&Ms in contrast are sort of trivial and arbitrary. That is, there's nothing about M&Ms that has anything to do with actual stage safety or management, and the assumption that attention to M&Ms and stage safety are the same is kind of odd. In fact, one might argue that the brown M&M clause pulls for a sort of cargo cult phenomenon, where people are following something to the letter without regard to deeper principles or understanding of what's important or not. Sometimes psychological tests do have items that test for random responding, which is maybe more comparable. But even then I don't think it's the same, because it's still item responses that are critical. To me the brown M&M test is like looking at the response time to an initial consent page, and arguing that if that is unusually long the person probably isn't responding to the items per se. I think there's maybe something to the brown M&M test but to me it pulls for the wrong things. Why not just actually, you know, make sure they abide by the long list of supposedly safety-critical items that you specified? If you as the performer are so sure that they're safety critical, but can't be bothered to check those, and instead rely on brown M&Ms, because it's easy to check in your dressing room, who's the one being lax about the stage?