4 ms·

The Ongoing Accomplishment of the Big Five

61 points by barry-cotter 6y ago

roenxi 6y ago
> What is interesting is the degree to which the Five Factor Model is insulated from falsification. It makes no causal predictions; others may make predictions about correlations, and these may replicate, or not. I remember struggling with feelings I would describe similarly when being taught how to fit an ARMA model to the stock market. Professor was very proud of himself and I was baffled about why he'd thought he'd done something. Eventually I decided maybe such models serve a function in academics very similar to a compression algorithm. The end goal is simply to be able to describe the data without having to present the data line by line - nothing more. No predictive or support in decision making is implied. The Big 5 model looks like it is a lot better than nothing. "Do you normally feel good?" and "Do you normally feel joyful?" are different questions, but determining when they are practically different requires a deep mastery of the language. Mr. Banana makes some good criticisms and the model will probably be abused to justify some stupid stuff out in the wild as many things in psychology are. But for academic sense, being replicable is all the model needs to do unless someone comes up with something clearly better.
- deleted 6y ago
  [deleted]
- vlovich123 6y ago
  What I took away from that analysis is that the entire field is self correlated (good) but lacking any underlying falsifiable theory (bad) or any ability to predict anything (bad). Repeatability, falsifiability, and predictability are the three requirements for any science. Remove one or more and it’s no longer science regardless of how much you want to believe it’s providing useful results. The history of science is littered with many such approaches. It could be the big five could eventually find better theoretical footing. My observation of the social sciences is less keen on this outcome. They perennially fail to accomplish it and thus move to “greener” pastures in terms of their financial incentives. For an example, consider the case where the study authors had a hypothesis that three of the five traits would predict something, only 2 did and they decided this was a success. Meanwhile astrophysics is arguing about potential sources of errors when results are off by 0.001%. Obviously different fields but the social sciences lack of establishing a solid quantifiable footing is non-ironically dismissed as “humans are difficult” rather than acknowledging that the basic science that’s attempted to be done is flawed and will never yield scientific results with the course it’s on.
  - AnthonyMouse 6y ago
    The fundamental problem with social "science" is that it's trying to predict human behavior, but human behavior is uncomputable. Literally. If you publish a study that says humans will do something, the humans will read your study, take it into account, and then do something different. That's the halting problem, game over.
    - mjburgess 6y ago
      This seems equivalent to the insight that a machine cannot be a programmer, ie., a system which cannot frame open problems cannot design systems which solve closed problems. NB. open here meaning aprox. "a problem class which is framed as a class by the solver"; closed meaning, "one of a class of pre-solved problems". Since humans are extreme open-problem framers, their framing strategies are themselves updated adaptively (by the environment) -- which cannot be reduced to any class of closed problems. Since (social) science does not have a theory of open-framing (ie., intelligence & problem solving proper) each description can only be a closed-solution description of a particular behaviour -- which then becomes invalidated when we reframe (eg., via reading the journal article). Your claim, re the haulting problem, suggests that there -- in some sense -- can never be a computable theory of open framing (, this seems highly plausible to me). In which case psychology is broadly impossible. However things like anger, family, etc. do exist. Psychology as a science of behaviour may be impossible; but there is a science of deterministic forces of human behaviour.
    - deleted 6y ago
      [deleted]
    - quotemstr 6y ago
      Of course human behavior is computable, at least in aggregate. How do you think marketing works? The mind is a computer and there's nothing special about it.
      - coldtea 6y ago
        There's a lot of thing special about it, whether it's a computer or not. For one, it's a whole different architecture. Second, it's several orders of magnitude more complex. Third, it changes based on all kind of strange inputs, not just conditions in the program and some IO (e.g. changes in chemicals in the body). Even the substructure where the "program" runs changes properties (like a CPU changing material properties mid-execution). Fourth, the "program" is self-modifying constantly, as opposed to a static program or different reprogrammings of an FPGI. Fifth, there's no manual or clear schematics.
exolymph 6y ago
The same author's discussion of surveys is even better: https://carcinisation.com/2020/12/11/survey-chicken/ https://carcinisation.com/2020/12/11/survey-chicken/
hyperpallium2 6y ago
Acts of clicking links are better data than surveys.
- spanktheuser 6y ago
  This is probably true, especially given the state of the surveys (for example, it required less than 10 minutes perusal to uncover obvious bias against distracted-type ADHD diagnostic surveys and by extension, females with ADHD). Although online history may provide superior illumination of a patient’s condition, crucially, it can’t be standardized. As such, it will never support the following cases: 1. Allow health insurers to predict and thus price the cost of psychiatric care. 2. Serve as an “impartial” basis for insurers to approve / deny claims in accordance with their pricing model and profit goals. 3. Create a defensible, scientific-seeming rationale for allowing some population monitored access to schedule III controlled substances while denying it to the wider public. 4. Allow wealthy, educated and self-directed individuals to learn the responses needed to obtain access to medically beneficial controlled substances without resorting to the black market. Cynics, marxists and progressives would likely enumerate several additional cases, but the pattern remains the same. Survey-based psychological tools correlate weakly with patient outcomes and diagnostic accuracy and strongly with the politics and economics of private health care.
- staticautomatic 6y ago
  Sometimes.
- phkahler 6y ago
  And advertisers still cant predict what I may be interested in.
  - hyperpallium2 6y ago
    They mix in that deliberatwly, so as not to creep you out...?
    - phkahler 6y ago
Reimersholme 6y ago
The question I have; given a huge question set with varying questions - would it be possible to draw random subsets of questions and present to a randomly drawn set of participants (i.e. not WEIRD only) and get a consistent set of orthogonal factors coming out, even if they are not OCEAN?
draw_down 6y ago
I don’t think I can understand the point of any of this. Why would you want to empirically measure things like “expression” and emotion?
appleflaxen 6y ago
This was a fantastic essay on psychological measurement of personality types. The take-home for me was: The Big Five are, in a sense, protected from falsification. They make no predictions; there is no underlying causal model. As I understand it, no study could be devised to prove that the Big Five aren’t real, because they make no formal pretense to reality. They are innocent mathematical constructs that fall out of particular survey instruments administered to particular populations. It is a data point in the idea that 1. only things that could conceivably be measured are real (unmeasurable/unobservable things are magic / not real) 2. but not all measurable (=real) things are important / valuable
throw0101a 6y ago
The accomplishment is 'limited' in its applicability: > As a novice to the study of the Big Five, I found that I had many misconceptions. For example, I thought that the Big Five were considered to be universal, in some sense, and not just descriptive of WEIRD college students. But in fact, it is not simply that the Big Five factors fail to fall out of the analysis of large surveys in other languages and cultures. They don’t even fall out of Big Five-specific questions administered in non-WEIRD populations (Gurven et al., 2013). Many of the concepts mentioned in the Big Five survey instruments do not even exist in non-WEIRD languages as such, particularly abstract nouns (Gurven et al., 2013). Of course a concept that does not exist across cultures could not form the basis for a universally important personality trait, as measured by language. Be mindful of other cultures / worldviews: * https://en.wikipedia.org/wiki/WEIRD https://en.wikipedia.org/wiki/WEIRD * https://en.wikipedia.org/wiki/Cross-cultural_psychology https://en.wikipedia.org/wiki/Cross-cultural_psychology Joseph Henrich recently published a book on the topic, discussed about a month ago: * https://www.theatlantic.com/magazine/archive/2020/10/joseph-henrich-weird-people/615496/ https://www.theatlantic.com/magazine/archive/2020/10/joseph-... * https://news.ycombinator.com/item?id=25227902 https://news.ycombinator.com/item?id=25227902

61 points by barry-cotter 6y ago

> What is interesting is the degree to which the Five Factor Model is insulated from falsification. It makes no causal predictions; others may make predictions about correlations, and these may replicate, or not. I remember struggling with feelings I would describe similarly when being taught how to fit an ARMA model to the stock market. Professor was very proud of himself and I was baffled about why he'd thought he'd done something. Eventually I decided maybe such models serve a function in academics very similar to a compression algorithm. The end goal is simply to be able to describe the data without having to present the data line by line - nothing more. No predictive or support in decision making is implied. The Big 5 model looks like it is a lot better than nothing. "Do you normally feel good?" and "Do you normally feel joyful?" are different questions, but determining when they are practically different requires a deep mastery of the language. Mr. Banana makes some good criticisms and the model will probably be abused to justify some stupid stuff out in the wild as many things in psychology are. But for academic sense, being replicable is all the model needs to do unless someone comes up with something clearly better.

deleted 6y ago
[deleted]
vlovich123 6y ago
What I took away from that analysis is that the entire field is self correlated (good) but lacking any underlying falsifiable theory (bad) or any ability to predict anything (bad). Repeatability, falsifiability, and predictability are the three requirements for any science. Remove one or more and it’s no longer science regardless of how much you want to believe it’s providing useful results. The history of science is littered with many such approaches. It could be the big five could eventually find better theoretical footing. My observation of the social sciences is less keen on this outcome. They perennially fail to accomplish it and thus move to “greener” pastures in terms of their financial incentives. For an example, consider the case where the study authors had a hypothesis that three of the five traits would predict something, only 2 did and they decided this was a success. Meanwhile astrophysics is arguing about potential sources of errors when results are off by 0.001%. Obviously different fields but the social sciences lack of establishing a solid quantifiable footing is non-ironically dismissed as “humans are difficult” rather than acknowledging that the basic science that’s attempted to be done is flawed and will never yield scientific results with the course it’s on.
- AnthonyMouse 6y ago
  The fundamental problem with social "science" is that it's trying to predict human behavior, but human behavior is uncomputable. Literally. If you publish a study that says humans will do something, the humans will read your study, take it into account, and then do something different. That's the halting problem, game over.
  - mjburgess 6y ago
    This seems equivalent to the insight that a machine cannot be a programmer, ie., a system which cannot frame open problems cannot design systems which solve closed problems. NB. open here meaning aprox. "a problem class which is framed as a class by the solver"; closed meaning, "one of a class of pre-solved problems". Since humans are extreme open-problem framers, their framing strategies are themselves updated adaptively (by the environment) -- which cannot be reduced to any class of closed problems. Since (social) science does not have a theory of open-framing (ie., intelligence & problem solving proper) each description can only be a closed-solution description of a particular behaviour -- which then becomes invalidated when we reframe (eg., via reading the journal article). Your claim, re the haulting problem, suggests that there -- in some sense -- can never be a computable theory of open framing (, this seems highly plausible to me). In which case psychology is broadly impossible. However things like anger, family, etc. do exist. Psychology as a science of behaviour may be impossible; but there is a science of deterministic forces of human behaviour.
  - deleted 6y ago
    [deleted]
  - quotemstr 6y ago
    Of course human behavior is computable, at least in aggregate. How do you think marketing works? The mind is a computer and there's nothing special about it.
    - coldtea 6y ago
      There's a lot of thing special about it, whether it's a computer or not. For one, it's a whole different architecture. Second, it's several orders of magnitude more complex. Third, it changes based on all kind of strange inputs, not just conditions in the program and some IO (e.g. changes in chemicals in the body). Even the substructure where the "program" runs changes properties (like a CPU changing material properties mid-execution). Fourth, the "program" is self-modifying constantly, as opposed to a static program or different reprogrammings of an FPGI. Fifth, there's no manual or clear schematics.

exolymph 6y ago

The same author's discussion of surveys is even better: https://carcinisation.com/2020/12/11/survey-chicken/ https://carcinisation.com/2020/12/11/survey-chicken/

hyperpallium2 6y ago

Acts of clicking links are better data than surveys.

spanktheuser 6y ago
This is probably true, especially given the state of the surveys (for example, it required less than 10 minutes perusal to uncover obvious bias against distracted-type ADHD diagnostic surveys and by extension, females with ADHD). Although online history may provide superior illumination of a patient’s condition, crucially, it can’t be standardized. As such, it will never support the following cases: 1. Allow health insurers to predict and thus price the cost of psychiatric care. 2. Serve as an “impartial” basis for insurers to approve / deny claims in accordance with their pricing model and profit goals. 3. Create a defensible, scientific-seeming rationale for allowing some population monitored access to schedule III controlled substances while denying it to the wider public. 4. Allow wealthy, educated and self-directed individuals to learn the responses needed to obtain access to medically beneficial controlled substances without resorting to the black market. Cynics, marxists and progressives would likely enumerate several additional cases, but the pattern remains the same. Survey-based psychological tools correlate weakly with patient outcomes and diagnostic accuracy and strongly with the politics and economics of private health care.
staticautomatic 6y ago
Sometimes.
phkahler 6y ago
And advertisers still cant predict what I may be interested in.
- hyperpallium2 6y ago
  They mix in that deliberatwly, so as not to creep you out...?
  - phkahler 6y ago

Reimersholme 6y ago

The question I have; given a huge question set with varying questions - would it be possible to draw random subsets of questions and present to a randomly drawn set of participants (i.e. not WEIRD only) and get a consistent set of orthogonal factors coming out, even if they are not OCEAN?

draw_down 6y ago

I don’t think I can understand the point of any of this. Why would you want to empirically measure things like “expression” and emotion?

appleflaxen 6y ago

This was a fantastic essay on psychological measurement of personality types. The take-home for me was: The Big Five are, in a sense, protected from falsification. They make no predictions; there is no underlying causal model. As I understand it, no study could be devised to prove that the Big Five aren’t real, because they make no formal pretense to reality. They are innocent mathematical constructs that fall out of particular survey instruments administered to particular populations. It is a data point in the idea that 1. only things that could conceivably be measured are real (unmeasurable/unobservable things are magic / not real) 2. but not all measurable (=real) things are important / valuable

throw0101a 6y ago

The accomplishment is 'limited' in its applicability: > As a novice to the study of the Big Five, I found that I had many misconceptions. For example, I thought that the Big Five were considered to be universal, in some sense, and not just descriptive of WEIRD college students. But in fact, it is not simply that the Big Five factors fail to fall out of the analysis of large surveys in other languages and cultures. They don’t even fall out of Big Five-specific questions administered in non-WEIRD populations (Gurven et al., 2013). Many of the concepts mentioned in the Big Five survey instruments do not even exist in non-WEIRD languages as such, particularly abstract nouns (Gurven et al., 2013). Of course a concept that does not exist across cultures could not form the basis for a universally important personality trait, as measured by language. Be mindful of other cultures / worldviews: * https://en.wikipedia.org/wiki/WEIRD https://en.wikipedia.org/wiki/WEIRD * https://en.wikipedia.org/wiki/Cross-cultural_psychology https://en.wikipedia.org/wiki/Cross-cultural_psychology Joseph Henrich recently published a book on the topic, discussed about a month ago: * https://www.theatlantic.com/magazine/archive/2020/10/joseph-henrich-weird-people/615496/ https://www.theatlantic.com/magazine/archive/2020/10/joseph-... * https://news.ycombinator.com/item?id=25227902 https://news.ycombinator.com/item?id=25227902