3 ms·

Fwiw, the Big 5 (OCEAN) personality inventory is one of the more robust findings in psychology (much more so than all the examples from the blog post) https://

Fwiw, the Big 5 (OCEAN) personality inventory is one of the more robust findings in psychology (much more so than all the examples from the blog post) https://en.wikipedia.org/wiki/Big_Five_personality_traits https://en.wikipedia.org/wiki/Big_Five_personality_traits and the four Meyers Briggs "types" are quite correlated with four of the Big 5. https://en.wikipedia.org/wiki/Myers%E2%80%93Briggs_Type_Indicator#Big_Five https://en.wikipedia.org/wiki/Myers%E2%80%93Briggs_Type_Indi... Biggest issue with Meyers Briggs is that it gives people the impression that types are discrete (i.e., traits have a bimodal distribution) when in fact the traits are normally distributed. And of course it's just out of date since MB was developed decades ago and is entrenched in the business world.

YeGoblynQueenne 5y ago
>> Fwiw, the Big 5 (OCEAN) personality inventory is one of the more robust findings in psychology (much more so than all the examples from the blog post) That depends on who you ask: Many investigations into the structure of individual differences theorize in terms of latent variables, but rely on Principal Components Analyses (PCA) when it comes to the analysis of empirical data. However, the extraction of a principal components structure, by itself, will not ordinarily shed much light on the correspondence with a putative latent variable structure. The reason is that PCA is not a latent variablemodel but a data reduction technique (e.g., Bartholomew, 2004). This is no problem as long as one does not go beyond the obvious interpretation of a principal component, which is that it is a conveniently weighted sumscore. Unfortunately, however, this is not the preferred interpretation among the enthusiastic users of principal components analysis. Consider, for instance, the personality literature,where people have discovered that executing a PCA of large numbers of personality subtest scores, and selecting components by the usual selection criteria, often returns five principal components. What is the interpretation of these components? They are “biologically based psychological tendencies,” and as such are endowed with causal forces (McCrae et al., 2000, p. 173). This interpretation cannot be justified solely on the basis of a PCA, if only because PCA is a formative model and not a reflective one (Bollen & Lennox, 1991; Borsboom, Mellenbergh, & Van Heerden, 2003). As such, it conceptualizes constructs as causally determined by the observations, rather than the other way around (Edwards & Bagozzi, 2000). In the case of PCA, the causal relation is moreover rather uninteresting; principal component scores are “caused” by their indicators inmuch the same way that sumscores are “caused” by item scores. Clearly, there is no conceivable way in which the Big Five could cause subtest scores on personality tests (or anything else, for that matter), unless they were in fact not principal components, but belonged to a more interesting species of theoretical entities; for instance, latent variables. Testing the hypothesis that the personality traits in question are causal determinants of personality test scores thus, at a minimum, requires the specification of a reflective latent variable model (Edwards & Bagozzi, 2000). A good example would be a Confirmatory Factor Analysis (CFA) model. Now it turns out that, with respect to the Big Five, CFA gives Big Problems. For instance, McCrae, Zonderman, Costa, Bond, & Paunonen (1996) found that a five factor model is not supported by the data, even though the tests involved in the analysis were specifically designed on the basis of the PCA solution. What does one conclude from this? Well, obviously, because the Big Five exist, but CFA cannot find them, CFA is wrong. “In actual analyses of personality data [. . .] structures that are known to be reliable [from principal components analyses] showed poor fits when evaluated by CFA techniques. We believe this points to serious problems with CFA itself when used to examine personality structure” (McCrae et al., 1996, p. 563). Denny Borsboom, The attack of the psychometricians, Psychometrika. 2006 Sep;71(3):425-440 https://pubmed.ncbi.nlm.nih.gov/19946599/ https://pubmed.ncbi.nlm.nih.gov/19946599/ In other words: bad statistics, badly applied.
- jessriedel 5y ago
  You describe a reasonable minority critique of the use of Big 5 for latent variable theorizing. That some "enthusistic users" of Big 5 mis-apply it or claim unjustified confidence in downstream conclusions does not at conflict with my comment that you are responding to. Indeed, a similar critique can be leveled even more strongly at the other purported psychological findings mentioned in the blog post.
  - YeGoblynQueenne 5y ago
    The critique is that the existence of the Big 5 itself is the result of misapplication of PCA for a task where a latent variable model would be required, the identification of personality traits. To clarify, personality traits are "latent" in the sense that they are not directly observable in personality test scores, so any analysis of test scores that simply aggregates test scores is not capable of detecting personality traits. PCA simply aggregates data to reduce the dimensionality of the data but it can't detect latent variables. Using PCA to detect personality traits is like using a thermometer to measure atmospheric pressure. What's worse, psychometrists seem to have taken the reading on the thermometer and proclaimed "Atmostperhic pressure is 25 [degrees Celsius]!".
    - jessriedel 5y ago
      > The critique is that the existence of the Big 5 itself is the result of misapplication of PCA for a task where a latent variable model would be required, the identification of personality traits. As stated, that critique is wrong in this sense: for a given population of, say, new employees at a business, PCA is sufficient to establish correlations between things that are easy to measure (Big 5 assessment test scores) and things that are much more expensive to measure but which the business cares about (how the employee will perform if assigned to a certain team). In this way, mere correlational data, without the construction of a latent variable model, is operationally useful. Objections that the tests don't really "detect personality" "directly" are semantical in this context. Yes, if the business starts using (implicitly or explicitly) a causal model of latent personality variables to make claims or predictions that take them outside the population distribution for which the correlations have been measured, then they could be making a mistake. But this does not invalidate the use of Big 5 assessments in general. > Using PCA to detect personality traits is like using a thermometer to measure atmospheric pressure. What's worse, psychometrists seem to have taken the reading on the thermometer and proclaimed "Atmostperhic pressure is 25 [degrees Celsius]!". Given a fixed population (in this case, a population of weather events) where temperature and pressure are well correlated, it is completely reasonable to use a thermometer to estimate pressure, and to take concrete action based on this. By all means, criticize people for being insufficiently precise (e.g. for not stating what their error bars or what the relevant population is), but don't pretend that this completely invalidates everything and Big 5 should be ignored.

YeGoblynQueenne 5y ago

>> Fwiw, the Big 5 (OCEAN) personality inventory is one of the more robust findings in psychology (much more so than all the examples from the blog post) That depends on who you ask: Many investigations into the structure of individual differences theorize in terms of latent variables, but rely on Principal Components Analyses (PCA) when it comes to the analysis of empirical data. However, the extraction of a principal components structure, by itself, will not ordinarily shed much light on the correspondence with a putative latent variable structure. The reason is that PCA is not a latent variablemodel but a data reduction technique (e.g., Bartholomew, 2004). This is no problem as long as one does not go beyond the obvious interpretation of a principal component, which is that it is a conveniently weighted sumscore. Unfortunately, however, this is not the preferred interpretation among the enthusiastic users of principal components analysis. Consider, for instance, the personality literature,where people have discovered that executing a PCA of large numbers of personality subtest scores, and selecting components by the usual selection criteria, often returns five principal components. What is the interpretation of these components? They are “biologically based psychological tendencies,” and as such are endowed with causal forces (McCrae et al., 2000, p. 173). This interpretation cannot be justified solely on the basis of a PCA, if only because PCA is a formative model and not a reflective one (Bollen & Lennox, 1991; Borsboom, Mellenbergh, & Van Heerden, 2003). As such, it conceptualizes constructs as causally determined by the observations, rather than the other way around (Edwards & Bagozzi, 2000). In the case of PCA, the causal relation is moreover rather uninteresting; principal component scores are “caused” by their indicators inmuch the same way that sumscores are “caused” by item scores. Clearly, there is no conceivable way in which the Big Five could cause subtest scores on personality tests (or anything else, for that matter), unless they were in fact not principal components, but belonged to a more interesting species of theoretical entities; for instance, latent variables. Testing the hypothesis that the personality traits in question are causal determinants of personality test scores thus, at a minimum, requires the specification of a reflective latent variable model (Edwards & Bagozzi, 2000). A good example would be a Confirmatory Factor Analysis (CFA) model. Now it turns out that, with respect to the Big Five, CFA gives Big Problems. For instance, McCrae, Zonderman, Costa, Bond, & Paunonen (1996) found that a five factor model is not supported by the data, even though the tests involved in the analysis were specifically designed on the basis of the PCA solution. What does one conclude from this? Well, obviously, because the Big Five exist, but CFA cannot find them, CFA is wrong. “In actual analyses of personality data [. . .] structures that are known to be reliable [from principal components analyses] showed poor fits when evaluated by CFA techniques. We believe this points to serious problems with CFA itself when used to examine personality structure” (McCrae et al., 1996, p. 563). Denny Borsboom, The attack of the psychometricians, Psychometrika. 2006 Sep;71(3):425-440 https://pubmed.ncbi.nlm.nih.gov/19946599/ https://pubmed.ncbi.nlm.nih.gov/19946599/ In other words: bad statistics, badly applied.

jessriedel 5y ago
You describe a reasonable minority critique of the use of Big 5 for latent variable theorizing. That some "enthusistic users" of Big 5 mis-apply it or claim unjustified confidence in downstream conclusions does not at conflict with my comment that you are responding to. Indeed, a similar critique can be leveled even more strongly at the other purported psychological findings mentioned in the blog post.
- YeGoblynQueenne 5y ago
  The critique is that the existence of the Big 5 itself is the result of misapplication of PCA for a task where a latent variable model would be required, the identification of personality traits. To clarify, personality traits are "latent" in the sense that they are not directly observable in personality test scores, so any analysis of test scores that simply aggregates test scores is not capable of detecting personality traits. PCA simply aggregates data to reduce the dimensionality of the data but it can't detect latent variables. Using PCA to detect personality traits is like using a thermometer to measure atmospheric pressure. What's worse, psychometrists seem to have taken the reading on the thermometer and proclaimed "Atmostperhic pressure is 25 [degrees Celsius]!".
  - jessriedel 5y ago
    > The critique is that the existence of the Big 5 itself is the result of misapplication of PCA for a task where a latent variable model would be required, the identification of personality traits. As stated, that critique is wrong in this sense: for a given population of, say, new employees at a business, PCA is sufficient to establish correlations between things that are easy to measure (Big 5 assessment test scores) and things that are much more expensive to measure but which the business cares about (how the employee will perform if assigned to a certain team). In this way, mere correlational data, without the construction of a latent variable model, is operationally useful. Objections that the tests don't really "detect personality" "directly" are semantical in this context. Yes, if the business starts using (implicitly or explicitly) a causal model of latent personality variables to make claims or predictions that take them outside the population distribution for which the correlations have been measured, then they could be making a mistake. But this does not invalidate the use of Big 5 assessments in general. > Using PCA to detect personality traits is like using a thermometer to measure atmospheric pressure. What's worse, psychometrists seem to have taken the reading on the thermometer and proclaimed "Atmostperhic pressure is 25 [degrees Celsius]!". Given a fixed population (in this case, a population of weather events) where temperature and pressure are well correlated, it is completely reasonable to use a thermometer to estimate pressure, and to take concrete action based on this. By all means, criticize people for being insufficiently precise (e.g. for not stating what their error bars or what the relevant population is), but don't pretend that this completely invalidates everything and Big 5 should be ignored.