8 ms·

The problem with psychometric tests is that they're easy to game by answering them with the values the company is almost certainly looking for. Any company iss

The problem with psychometric tests is that they're easy to game by answering them with the values the company is almost certainly looking for. Any company issuing a psychometric test is looking for obedience, hard work, discipline, profit orientation, civility, non-threatening creativity, etc. etc. etc. So what you get by discouraging a broad diversity of personalities is a mix of people who actually have those specific characteristics and sociopaths. The latter group ends up running the company, and of course they issue utterly sociopathic "psychometric evaluations", which are basically just encoded discrimination against entire classes of people, regardless of job fitness, because the people up top are such incompetent leaders that they need subservient workers to obey their crazy orders without question.

dylangs1030 13y ago
>The problem with psychometric tests is that they're easy to game by answering them with the values the company is almost certainly looking for. You'd think that, but a quality psychometric test can't be gamed if it was developed in-house and is only taken once by the candidate. This particular company might not have had one of those, however. Professional personality tests can't be gamed - you also can't find these on the internet, because they're very strictly copyrighted.
- asdfdsa1234 13y ago
  Can you go into more detail? It's hard to imagine how a test could be devised that wouldn't be gameable by giving answers that would be expected of a person with a normal personality.
  - dylangs1030 13y ago
    It's not particularly difficult to devise a personality exam that way - in the best case scenario, the candidate cannot game the test, and in the worst case scenario, they game it but the proctor is fully aware of the gaming, so you simply reject the test (or the candidate). Instead of trying to get rid of questions that are gameable, the psychometrists design the test with statistical consistency - things as obscure as, say, what color the candidate answers in a multiple choice question out of orange, yellow, blue and red can be used to correlate with other questions (not literally, but to a candidate it would be equally obscure and seemingly innocuous as a question). If a candidate answers one question in what they perceive to be the societal ideal, this will be exposed in other questions where it's not possible to cross-consistently game the question unless you've read and thoroughly understand a manual of psychology or psychometry. Tests designed this way allow for a certain amount of "gaming" by candidates before it crosses a statistical threshold, at which point it essentially tells the proctor, "The answers are so inconsistent that the candidate wasn't being honest." and you have to throw out the whole test (which in the context of hiring, means rejecting the candidate). The reason why this works is because while people will answer "Would you consider yourself hard working?" with "Always" or some other unrealistically gamed superlative, they won't realize that other questions that are seemingly unrelated are highly correlated with that quality - if you answer yes to one and no to another, you probably lied on the transparent one, with high confidence. Tests like this[1] have been around for so long that they consistently get better, though there are some valid criticisms of them being easier for non-minorities (for a variety of reasons). They are used in clinical and professional contexts, and while the quality personality tests will theoretically be consistent across multiple-test takings (in other words, results are relatively immutable), in practice you should probably avoid giving one individual a lot of exposure to the same exact test twice. Hope that helps. [1]: http://en.wikipedia.org/wiki/Minnesota_Multiphasic_Personality_Inventory http://en.wikipedia.org/wiki/Minnesota_Multiphasic_Personali...
    - bentcorner 13y ago
      > The reason why this works is because while people will answer "Would you consider yourself hard working?" with "Always" or some other unrealistically gamed superlative, they won't realize that other questions that are seemingly unrelated are highly correlated with that quality Can you provide a deeper example? I'm honestly curious - what sort of question is innocuous enough to be answered honestly, yet useful enough to provide information? (i.e., do slackers like the color yellow or something?)
      - kronusaturn 13y ago
        This is actually part of the so-called "bogus pipeline." Test-takers are told that the test can detect any attempt to lie, causing them to answer the questions more honestly than they would otherwise. The MMPI does detect inconsistent and unrealistically extreme answers, but it's not quite as foolproof as claimed above. http://en.wikipedia.org/wiki/Bogus_pipeline http://en.wikipedia.org/wiki/Bogus_pipeline
      - crucini 13y ago
        Here are some: http://www.physicschick.com/pole/2012xxxx/psych.txt http://www.physicschick.com/pole/2012xxxx/psych.txt By googling those, you can go deeper into the rabbit hole.
    - RogerL 13y ago
      Okay, that is the MMPI. That is not evidence that any test that is not the MMPI cannot be gamed. The MMPI (and I disgree with it in so many ways) was researched and tested for decades, and perhaps they can detect lying. Perhaps. I don't buy that any lesser test cannot be easily gamed absent a heck of a lot of experimental evidence for that specific test.
    - disgruntledphd2 13y ago
      I'm a psychologist, and I game personality tests for fun. The MMPI is harder than the Myers Briggs, but not that difficult. Additionally its really only optimal for clinical samples, at which it is very good. However, everyone in the field knows that these tests can be gamed, the only open question is how many people game them consistently. You can probably estimate the proportion of social desirability exhibited in job interviews by comparing to non-job situations (such as a sample matched on all relevant covariates (whatever they are) from the general population. Nonetheless, believing in personality tests as an accurate indicator of personality is as misguided as believing that Facebook represents the social graph of all its daily active users accurately, i.e., somewhat misguided. And I am aware of lie scale, and they are trivial if you actually read the question. Protip: If a question says always or never, its probably designed to trip you up. I do agree that personality tests are more accurate than this thread makes them out to be, but they are certainly not as useful as your comment implies.
- HelloMcFly 13y ago
  Often, at least in the work contexts, it doesn't matter if they can be gamed. Validity testing is done with the effects of gaming built-in, and there is some evidence that gaming has relations with performance benefits. A acquaintance of mine is finishing his dissertation on faking on personality and integrity tests, so I'll know more on that soon.
  - dylangs1030 13y ago
    Yes, but in that context gaming is known and accounted for. My definition of gaming was manipulation without the proctor or evaluation showing any statistically significant deviation, which is virtually impossible on modern personality tests. The "gaming" becomes transparent and used to score the candidate, if it's present at all.
    - crucini 13y ago
      I appreciate your input, especially as counterweight to those who underestimate the sophistication of these tests. However I question your absolute statement. What man can create, man can circumvent. (How's that for an absolute statement?) Would you say that the people who created the test cannot game it?
      - dylangs1030 13y ago
        No, it can certainly be gamed. It's just so hard without prior knowledge and preparation as to be statistically negligent. This is also why I said you should avoid giving the same test twice. But a single, cold test administration should be very difficult to game. You should also read upthread, what the actual psychologist said. It clarified my comment really well.
        foobarbazqux 13y ago
        > But a single, cold test administration should be very difficult to game. Oh come on. This was posted in a different comment: http://www.physicschick.com/pole/2012xxxx/psych.txt http://www.physicschick.com/pole/2012xxxx/psych.txt The following are very obviously the "correct" true or false answers to these questions from the MMPI-2: T * My mother is a good woman. F * Evil spirits possess me at times. F * There seems to be a lump in my throat much of the time. T * At times I feel like swearing. T * My hands and feet are usually warm enough. F * Ghosts or spirits can influence people for good or bad. F * Someone has been trying to poison me. F * Everything tastes the same. F * Someone has been trying to rob me. F * Bad words, often terrible words, come into my mind and I cannot get rid of them. F * Often I feel as if there is a tight band around my head. F * Peculiar odors come to me at times. F * My soul sometimes leaves my body. F * When a man is with a woman he is usually thinking about things related to her sex. F * I often feel as if things are not real. F * Someone has it in for me. F * My neck spots with red often. T * Once in awhile I laugh at a dirty joke. F * I hear strange things when I am alone. F * In walking I am very careful to step over sidewalk cracks. F * At one or more times in my life, I felt that someone was making me do things by hypnotizing me. I cut out a few because I couldn't see how true or false mattered.
LaGrange 13y ago
Actually, spotting and weeding out sociopaths is one of the points. I know that you may believe that you're a good judge of character, but actually a smart sociopath (not the norm, btw) is likely to be able to game you just as easily as the test.
- foobarbazqux 13y ago
  Well, my main objection is to hiring people on the basis of their character, whether it's with a test or not. Assess for whether or not the person can do the job and make you a profit.
  - HelloMcFly 13y ago
    If one's character meaningfully predicts job performance on top of whatever other methods you are using, then assessing on their character is the same as assessing on whether or not they can do they job (or at least, how well they can do it). Integrity tests have demonstrated this in many contexts, even taking into account the possibility of faking.
    - foobarbazqux 13y ago
      Except that obviously wasn't the case in the linked article.
  - karlkatzke 13y ago
    Can you spot the person who interviews well, but is a skilled sociopath who will destroy the morale of your team, causing other skilled talent to leave? That's the value of these tests (when properly applied): They allow an evaluator to make statistically accurate predictions of specific types of behavior. The key is the two words: statistically and specific. A single composite score will not tell you anything, but the idea that there's a 68% chance that the person you're about to hire is a schizophrenic kleptomaniac should give you pause.
    - foobarbazqux 13y ago

dylangs1030 13y ago

>The problem with psychometric tests is that they're easy to game by answering them with the values the company is almost certainly looking for. You'd think that, but a quality psychometric test can't be gamed if it was developed in-house and is only taken once by the candidate. This particular company might not have had one of those, however. Professional personality tests can't be gamed - you also can't find these on the internet, because they're very strictly copyrighted.

asdfdsa1234 13y ago
Can you go into more detail? It's hard to imagine how a test could be devised that wouldn't be gameable by giving answers that would be expected of a person with a normal personality.
- dylangs1030 13y ago
  It's not particularly difficult to devise a personality exam that way - in the best case scenario, the candidate cannot game the test, and in the worst case scenario, they game it but the proctor is fully aware of the gaming, so you simply reject the test (or the candidate). Instead of trying to get rid of questions that are gameable, the psychometrists design the test with statistical consistency - things as obscure as, say, what color the candidate answers in a multiple choice question out of orange, yellow, blue and red can be used to correlate with other questions (not literally, but to a candidate it would be equally obscure and seemingly innocuous as a question). If a candidate answers one question in what they perceive to be the societal ideal, this will be exposed in other questions where it's not possible to cross-consistently game the question unless you've read and thoroughly understand a manual of psychology or psychometry. Tests designed this way allow for a certain amount of "gaming" by candidates before it crosses a statistical threshold, at which point it essentially tells the proctor, "The answers are so inconsistent that the candidate wasn't being honest." and you have to throw out the whole test (which in the context of hiring, means rejecting the candidate). The reason why this works is because while people will answer "Would you consider yourself hard working?" with "Always" or some other unrealistically gamed superlative, they won't realize that other questions that are seemingly unrelated are highly correlated with that quality - if you answer yes to one and no to another, you probably lied on the transparent one, with high confidence. Tests like this[1] have been around for so long that they consistently get better, though there are some valid criticisms of them being easier for non-minorities (for a variety of reasons). They are used in clinical and professional contexts, and while the quality personality tests will theoretically be consistent across multiple-test takings (in other words, results are relatively immutable), in practice you should probably avoid giving one individual a lot of exposure to the same exact test twice. Hope that helps. [1]: http://en.wikipedia.org/wiki/Minnesota_Multiphasic_Personality_Inventory http://en.wikipedia.org/wiki/Minnesota_Multiphasic_Personali...
  - bentcorner 13y ago
    > The reason why this works is because while people will answer "Would you consider yourself hard working?" with "Always" or some other unrealistically gamed superlative, they won't realize that other questions that are seemingly unrelated are highly correlated with that quality Can you provide a deeper example? I'm honestly curious - what sort of question is innocuous enough to be answered honestly, yet useful enough to provide information? (i.e., do slackers like the color yellow or something?)
    - kronusaturn 13y ago
      This is actually part of the so-called "bogus pipeline." Test-takers are told that the test can detect any attempt to lie, causing them to answer the questions more honestly than they would otherwise. The MMPI does detect inconsistent and unrealistically extreme answers, but it's not quite as foolproof as claimed above. http://en.wikipedia.org/wiki/Bogus_pipeline http://en.wikipedia.org/wiki/Bogus_pipeline
    - crucini 13y ago
      Here are some: http://www.physicschick.com/pole/2012xxxx/psych.txt http://www.physicschick.com/pole/2012xxxx/psych.txt By googling those, you can go deeper into the rabbit hole.
  - RogerL 13y ago
    Okay, that is the MMPI. That is not evidence that any test that is not the MMPI cannot be gamed. The MMPI (and I disgree with it in so many ways) was researched and tested for decades, and perhaps they can detect lying. Perhaps. I don't buy that any lesser test cannot be easily gamed absent a heck of a lot of experimental evidence for that specific test.
  - disgruntledphd2 13y ago
    I'm a psychologist, and I game personality tests for fun. The MMPI is harder than the Myers Briggs, but not that difficult. Additionally its really only optimal for clinical samples, at which it is very good. However, everyone in the field knows that these tests can be gamed, the only open question is how many people game them consistently. You can probably estimate the proportion of social desirability exhibited in job interviews by comparing to non-job situations (such as a sample matched on all relevant covariates (whatever they are) from the general population. Nonetheless, believing in personality tests as an accurate indicator of personality is as misguided as believing that Facebook represents the social graph of all its daily active users accurately, i.e., somewhat misguided. And I am aware of lie scale, and they are trivial if you actually read the question. Protip: If a question says always or never, its probably designed to trip you up. I do agree that personality tests are more accurate than this thread makes them out to be, but they are certainly not as useful as your comment implies.
HelloMcFly 13y ago
Often, at least in the work contexts, it doesn't matter if they can be gamed. Validity testing is done with the effects of gaming built-in, and there is some evidence that gaming has relations with performance benefits. A acquaintance of mine is finishing his dissertation on faking on personality and integrity tests, so I'll know more on that soon.
- dylangs1030 13y ago
  Yes, but in that context gaming is known and accounted for. My definition of gaming was manipulation without the proctor or evaluation showing any statistically significant deviation, which is virtually impossible on modern personality tests. The "gaming" becomes transparent and used to score the candidate, if it's present at all.
  - crucini 13y ago
    I appreciate your input, especially as counterweight to those who underestimate the sophistication of these tests. However I question your absolute statement. What man can create, man can circumvent. (How's that for an absolute statement?) Would you say that the people who created the test cannot game it?
    - dylangs1030 13y ago
      No, it can certainly be gamed. It's just so hard without prior knowledge and preparation as to be statistically negligent. This is also why I said you should avoid giving the same test twice. But a single, cold test administration should be very difficult to game. You should also read upthread, what the actual psychologist said. It clarified my comment really well.
      - foobarbazqux 13y ago
        > But a single, cold test administration should be very difficult to game. Oh come on. This was posted in a different comment: http://www.physicschick.com/pole/2012xxxx/psych.txt http://www.physicschick.com/pole/2012xxxx/psych.txt The following are very obviously the "correct" true or false answers to these questions from the MMPI-2: T * My mother is a good woman. F * Evil spirits possess me at times. F * There seems to be a lump in my throat much of the time. T * At times I feel like swearing. T * My hands and feet are usually warm enough. F * Ghosts or spirits can influence people for good or bad. F * Someone has been trying to poison me. F * Everything tastes the same. F * Someone has been trying to rob me. F * Bad words, often terrible words, come into my mind and I cannot get rid of them. F * Often I feel as if there is a tight band around my head. F * Peculiar odors come to me at times. F * My soul sometimes leaves my body. F * When a man is with a woman he is usually thinking about things related to her sex. F * I often feel as if things are not real. F * Someone has it in for me. F * My neck spots with red often. T * Once in awhile I laugh at a dirty joke. F * I hear strange things when I am alone. F * In walking I am very careful to step over sidewalk cracks. F * At one or more times in my life, I felt that someone was making me do things by hypnotizing me. I cut out a few because I couldn't see how true or false mattered.

LaGrange 13y ago

Actually, spotting and weeding out sociopaths is one of the points. I know that you may believe that you're a good judge of character, but actually a smart sociopath (not the norm, btw) is likely to be able to game you just as easily as the test.

foobarbazqux 13y ago
Well, my main objection is to hiring people on the basis of their character, whether it's with a test or not. Assess for whether or not the person can do the job and make you a profit.
- HelloMcFly 13y ago
  If one's character meaningfully predicts job performance on top of whatever other methods you are using, then assessing on their character is the same as assessing on whether or not they can do they job (or at least, how well they can do it). Integrity tests have demonstrated this in many contexts, even taking into account the possibility of faking.
  - foobarbazqux 13y ago
    Except that obviously wasn't the case in the linked article.
- karlkatzke 13y ago
  Can you spot the person who interviews well, but is a skilled sociopath who will destroy the morale of your team, causing other skilled talent to leave? That's the value of these tests (when properly applied): They allow an evaluator to make statistically accurate predictions of specific types of behavior. The key is the two words: statistically and specific. A single composite score will not tell you anything, but the idea that there's a 68% chance that the person you're about to hire is a schizophrenic kleptomaniac should give you pause.
  - foobarbazqux 13y ago