6 ms·

While this is true for a lot of common psychological constructs, the Big Five were not defined according to agreement or consensus. The definitions were driven

While this is true for a lot of common psychological constructs, the Big Five were not defined according to agreement or consensus. The definitions were driven by the lexical hypothesis whereby the wide variety of personality descriptive adjectives in human language were factor analyzed and found to cluster into 5 distinct factors. So this isn't tautological in the way that you've described, though it is certainly related to the way human beings describe each other and the way that instantiates in human language. Source 1: https://en.wikipedia.org/wiki/Lexical_hypothesis https://en.wikipedia.org/wiki/Lexical_hypothesis Source 2: I'm a psychologist and did research in this area

jawns 5y ago
But isn't that just tautology with an intermediate step? You might have Personality Descriptive Adjective X that is found to cluster into what we call a Big Five trait. But Adjective X has its own definition, and it's likely that the tautology (at least insofar as these life satisfaction correlations are concerned) happens at that level.
- dr_dshiv 5y ago
  I suppose they asked 1000s of different questions, each representing a different personality characteristic. After factor analysis, some arbitrary number of factors were found to account for most of the variance. Why did they settle for 5 vs 8 vs 12?
  - anbende 5y ago
    There are a number of metrics that are commonly used to settle on a number of factors to extract. There's some judgment in it to be sure, but you are looking at data-driven phenomena as well like percentage of variance explained, a precipitous drop in new added value, and etc. Essentially you come to a point where extracting another value isn't explaining much more variance or producing a factor that's meaningful (e.g., two items of conscientiousness with negative wording are the entire new "factor" and only take variance explained from 80% to 82%). https://www.theanalysisfactor.com/factor-analysis-how-many-factors/ https://www.theanalysisfactor.com/factor-analysis-how-many-f...
    - disgruntledphd2 5y ago
      Like pretty much every unsupervised learning problem, there is no data which can tell you what N should be. If there was, you could trivially use that measure to convert it to a supervised learning problem. More generally, all of those metrics are pretty fluffy (the scree plot is probably the best, as it doesn't give you a p-value). In order to actually do this right, you need multiple studies and CFA, but the standard in the field is to refit your exploratory model as a CFA on the original dataset, which is almost certainly causing overfitting in most cases.
  - planet-and-halo 5y ago
    Other models actually did settle for higher numbers of factors, but higher numbers were usually found to have categories that could be reasonably collapsed into one of the Big 5. Fwiw, my understanding is that most psychologists believe that with further study, these categories will map to actual functioning of the brain and nerve systems. For example, "Neuroticism" will somehow map to the excitability of certain nerve systems and "Agreeableness" will somehow map to functioning of our mirror and mentalizing systems. "Conscientiousness" will map to inhibiting functions, "Openness" will map to approach/flee type emotional functions, and "Extraversion" will map to systems like dopamine release on securing rewards for behavior.
    - disgruntledphd2 5y ago
      That's a belief that's a long way from being testable though.
- anbende 5y ago
  Tautology implies that the definition we give is redundant, but none of these things were defined in this way. This is because none of the research relies on this kind of dictionary definition. Instead, they literally ask people hundreds of questions of the type: "To what extent would you consider yourself to be [adjective]?" (0 - 'not at all' to 5 - 'complete') Then they factor analyze the results to see which adjectives tend to be highly related to each other. Factor analysis builds on correlation to find shared variation among large groups of indicator variables. So it is related to how people think of the adjectives and what they mean, but it is not the case that we are just lumping definitions together or "saying the same thing in different words" in the way of a tautology. Instead, what we're saying is that "Most of these personality adjectives mostly mean the same thing. Here's the thing they all mostly mean. If humans in all languages have tons of words that mostly mean this common thing, maybe this common thing is significant and worthy of study." Regarding life satisfaction, the sense that one finds their life satisfactory and "would change nothing" (one of the items in the Satisfaction with Life Scale that was most likely used to measure it) doesn't show up in those other traits in any major way. In fact, to the extent that a description of the self is related to a world-view, belief or other external factor, it will never be a tautology, because a self-view and other-view or world-view will never share a common definition or tautology by virtue of the self-other distinction.
  - disgruntledphd2 5y ago
    > Then they factor analyze the results to see which adjectives tend to be highly related to each other. Factor analysis builds on correlation to find shared variation among large groups of indicator variables. The big problem with these results (and god knows I've analysed enough of them) is that the extent that they map to other characteristics measured without surveys is pretty much unknown. If you want to create a new measure of extraversion, then you use the older methods (i.e. other surveys), and this only really tells you that the new measure correlates with the old. I vaguely recall Eysenck doing physiological tests for extraversion/introversion which would avoid this problem, but I'm not aware of any more recent research in this area (mind you, I've been out of the field for a decade now).
vlovich123 5y ago
Humorism was a field of academic study for two millennia before germ theory demonstrated how laughable the entire field was. It’s always important to remember that doing research in a field doesn’t necessarily mean that the field itself is on a solid foundation that maps to reality. Psychology and psychiatry really do struggle to demonstrate statistically meaningful results behind most therapies (CBT is perhaps a notable exception, although even there results can be mixed). Analyses like the big 5 are squarely in the “let me read your tea leaves” camp of “science” IMO. They basically play a trick of “sure, the other personality theories were bogus but this time we did stumble upon a good way of characterizing it” while taking at face value that the entire endeavor as valid. Neuroscience has a bit more behind it because it’s actually rooted in quantifiable measurements somewhat and even there it’s difficult. It’s like trying to reverse-engineer a modern CPU from X-ray lithography. If you know the theory underpinning everything, you can start to make educated guesses. Without that you’re not going to get anywhere close to what’s going on let alone root cause if something is broken.
- anbende 5y ago
  A major problem with humorism was that it did not have any empirical evidence to support it. Modern personality theory DOES. Personality theory uses a variety of techniques to show that a construct is internally consistent and related to real-world indicators. If independent people can view a video or a person and they reliably rate that entity similarly (inter-rater reliability), we can say with some certainty that there is something there. Relating a construct like this to a variety of real-world outcomes allows us to ascertain its relationship to the world. This evidence is messy and not nearly as solid as what's found in the harder sciences, but your dismissal of the modern statistics of questionnaire-based measurement does a very well-researched field of inquiry a huge disservice.
  - vlovich123 5y ago
    My analogy to humorism was not to imply that there’s no empirical evidence at all here. Just that the field of psychology has continually struggled to actually develop testable hypotheses that lead to scientific theories. There’s a giant chasm between “I have a signal” and “here’s what that signal means”. You can’t start out with a data set, pick out some signals and then say what they mean. That’s scientific theory 101. You build a hypothetical model and then design experiments to test those theories. It’s not me doing the field a disservice. The practitioners have consistently and repeatedly done bad science and statistics. I’ll be more supportive once the field actually starts producing falsifiable theories that are supported by evidence and the practitioners are better at calling out their peers.
analog31 5y ago
Given that the definitions of words and phrases are possibly subjective, I've often wondered if the personality test largely measures how people interpret the questions.
des1nderlase 5y ago
So how do we objectively measure all 5 dimensions? I.e. would I roughly assess myself similar to how others (on average) would assess me?
- bildung 5y ago
  There is a bunch of item pools that has been developed for this: https://en.wikipedia.org/wiki/Big_Five_personality_traits#Measurements https://en.wikipedia.org/wiki/Big_Five_personality_traits#Me...
disgruntledphd2 5y ago
To be fair, Costa and McCrae did a lot of great work on the Big 5, but they really didn't like it when SEM/CFA didn't really support their theory. In fact, they liked it so little that they invented Procrustean rotation so that they could make the factors come out the "correct" way. Once I discovered that, I paid a lot less attention to the Big 5. Also, I haven't checked the source papers, but that all blue stuff could come from a general tendency to agree with the questions/put a positive spin on that. I do know that not all personality measures are balanced on positive vs negative questions, to remove this bias (life satisfaction methods are almost certainly not balanced in this way).

jawns 5y ago

But isn't that just tautology with an intermediate step? You might have Personality Descriptive Adjective X that is found to cluster into what we call a Big Five trait. But Adjective X has its own definition, and it's likely that the tautology (at least insofar as these life satisfaction correlations are concerned) happens at that level.

dr_dshiv 5y ago
I suppose they asked 1000s of different questions, each representing a different personality characteristic. After factor analysis, some arbitrary number of factors were found to account for most of the variance. Why did they settle for 5 vs 8 vs 12?
- anbende 5y ago
  There are a number of metrics that are commonly used to settle on a number of factors to extract. There's some judgment in it to be sure, but you are looking at data-driven phenomena as well like percentage of variance explained, a precipitous drop in new added value, and etc. Essentially you come to a point where extracting another value isn't explaining much more variance or producing a factor that's meaningful (e.g., two items of conscientiousness with negative wording are the entire new "factor" and only take variance explained from 80% to 82%). https://www.theanalysisfactor.com/factor-analysis-how-many-factors/ https://www.theanalysisfactor.com/factor-analysis-how-many-f...
  - disgruntledphd2 5y ago
    Like pretty much every unsupervised learning problem, there is no data which can tell you what N should be. If there was, you could trivially use that measure to convert it to a supervised learning problem. More generally, all of those metrics are pretty fluffy (the scree plot is probably the best, as it doesn't give you a p-value). In order to actually do this right, you need multiple studies and CFA, but the standard in the field is to refit your exploratory model as a CFA on the original dataset, which is almost certainly causing overfitting in most cases.
- planet-and-halo 5y ago
  Other models actually did settle for higher numbers of factors, but higher numbers were usually found to have categories that could be reasonably collapsed into one of the Big 5. Fwiw, my understanding is that most psychologists believe that with further study, these categories will map to actual functioning of the brain and nerve systems. For example, "Neuroticism" will somehow map to the excitability of certain nerve systems and "Agreeableness" will somehow map to functioning of our mirror and mentalizing systems. "Conscientiousness" will map to inhibiting functions, "Openness" will map to approach/flee type emotional functions, and "Extraversion" will map to systems like dopamine release on securing rewards for behavior.
  - disgruntledphd2 5y ago
    That's a belief that's a long way from being testable though.
anbende 5y ago
Tautology implies that the definition we give is redundant, but none of these things were defined in this way. This is because none of the research relies on this kind of dictionary definition. Instead, they literally ask people hundreds of questions of the type: "To what extent would you consider yourself to be [adjective]?" (0 - 'not at all' to 5 - 'complete') Then they factor analyze the results to see which adjectives tend to be highly related to each other. Factor analysis builds on correlation to find shared variation among large groups of indicator variables. So it is related to how people think of the adjectives and what they mean, but it is not the case that we are just lumping definitions together or "saying the same thing in different words" in the way of a tautology. Instead, what we're saying is that "Most of these personality adjectives mostly mean the same thing. Here's the thing they all mostly mean. If humans in all languages have tons of words that mostly mean this common thing, maybe this common thing is significant and worthy of study." Regarding life satisfaction, the sense that one finds their life satisfactory and "would change nothing" (one of the items in the Satisfaction with Life Scale that was most likely used to measure it) doesn't show up in those other traits in any major way. In fact, to the extent that a description of the self is related to a world-view, belief or other external factor, it will never be a tautology, because a self-view and other-view or world-view will never share a common definition or tautology by virtue of the self-other distinction.
- disgruntledphd2 5y ago
  > Then they factor analyze the results to see which adjectives tend to be highly related to each other. Factor analysis builds on correlation to find shared variation among large groups of indicator variables. The big problem with these results (and god knows I've analysed enough of them) is that the extent that they map to other characteristics measured without surveys is pretty much unknown. If you want to create a new measure of extraversion, then you use the older methods (i.e. other surveys), and this only really tells you that the new measure correlates with the old. I vaguely recall Eysenck doing physiological tests for extraversion/introversion which would avoid this problem, but I'm not aware of any more recent research in this area (mind you, I've been out of the field for a decade now).

vlovich123 5y ago

Humorism was a field of academic study for two millennia before germ theory demonstrated how laughable the entire field was. It’s always important to remember that doing research in a field doesn’t necessarily mean that the field itself is on a solid foundation that maps to reality. Psychology and psychiatry really do struggle to demonstrate statistically meaningful results behind most therapies (CBT is perhaps a notable exception, although even there results can be mixed). Analyses like the big 5 are squarely in the “let me read your tea leaves” camp of “science” IMO. They basically play a trick of “sure, the other personality theories were bogus but this time we did stumble upon a good way of characterizing it” while taking at face value that the entire endeavor as valid. Neuroscience has a bit more behind it because it’s actually rooted in quantifiable measurements somewhat and even there it’s difficult. It’s like trying to reverse-engineer a modern CPU from X-ray lithography. If you know the theory underpinning everything, you can start to make educated guesses. Without that you’re not going to get anywhere close to what’s going on let alone root cause if something is broken.

anbende 5y ago
A major problem with humorism was that it did not have any empirical evidence to support it. Modern personality theory DOES. Personality theory uses a variety of techniques to show that a construct is internally consistent and related to real-world indicators. If independent people can view a video or a person and they reliably rate that entity similarly (inter-rater reliability), we can say with some certainty that there is something there. Relating a construct like this to a variety of real-world outcomes allows us to ascertain its relationship to the world. This evidence is messy and not nearly as solid as what's found in the harder sciences, but your dismissal of the modern statistics of questionnaire-based measurement does a very well-researched field of inquiry a huge disservice.
- vlovich123 5y ago
  My analogy to humorism was not to imply that there’s no empirical evidence at all here. Just that the field of psychology has continually struggled to actually develop testable hypotheses that lead to scientific theories. There’s a giant chasm between “I have a signal” and “here’s what that signal means”. You can’t start out with a data set, pick out some signals and then say what they mean. That’s scientific theory 101. You build a hypothetical model and then design experiments to test those theories. It’s not me doing the field a disservice. The practitioners have consistently and repeatedly done bad science and statistics. I’ll be more supportive once the field actually starts producing falsifiable theories that are supported by evidence and the practitioners are better at calling out their peers.

analog31 5y ago

Given that the definitions of words and phrases are possibly subjective, I've often wondered if the personality test largely measures how people interpret the questions.

des1nderlase 5y ago

So how do we objectively measure all 5 dimensions? I.e. would I roughly assess myself similar to how others (on average) would assess me?

bildung 5y ago
There is a bunch of item pools that has been developed for this: https://en.wikipedia.org/wiki/Big_Five_personality_traits#Measurements https://en.wikipedia.org/wiki/Big_Five_personality_traits#Me...

disgruntledphd2 5y ago

To be fair, Costa and McCrae did a lot of great work on the Big 5, but they really didn't like it when SEM/CFA didn't really support their theory. In fact, they liked it so little that they invented Procrustean rotation so that they could make the factors come out the "correct" way. Once I discovered that, I paid a lot less attention to the Big 5. Also, I haven't checked the source papers, but that all blue stuff could come from a general tendency to agree with the questions/put a positive spin on that. I do know that not all personality measures are balanced on positive vs negative questions, to remove this bias (life satisfaction methods are almost certainly not balanced in this way).