4 ms·

Am I overreacting to consider these kinds of psychometric studies to be not much better than phrenology? "Behavioral phrenology" maybe?

by api 5y ago

civilized 5y ago
It's an interesting idea. But to me the marshmallow test at least had some plausible connection to personality. But maybe in the days of phrenology, people thought a hooked nose* had a plausible connection to personality as well? Weird to think about. *Sorry, this is physiognomy not phrenology. The same basic point stands though.
- well_i_guess 5y ago
  I think that the issue is that there is no true metric for “highly marketable talents/traits.” One generations genius could be another generations average worker, solely because market forces eliminate the competitive advantage of certain things. Many, many authors seem to lament the distractability of the current generations yet I would bet you many of the most famous people to Gen Z are incredibly attention-fickle. Whereas, 20 years ago, focus would probably be an essential skill for key performers.
  - fancifalmanima 5y ago
    Focus is almost surely an essential skill for key performers. Even among the most famous Gen Z -- you don't think they focus on their social media presence and what they do? What is an 8 hour photo shoot if not focusing? A lot of work goes into what social media influencers post, its not all done on a whim. There's also plenty of Gen Z doing other more traditional work (almost everyone of that generation, really). If anything, they've probably had to develop coping mechanisms from an extremely early age to deal with distraction, compared to prior generations.
    - brimble 5y ago
      I'm reminded of the SlateStarCodex post that mulls over the difference between "real" ADD and just having totally ordinary (but pretty great) difficulty focusing on the exact same boring crap on a computer screen day, after day, after day—especially if, in the latter case, a lot of the people these folks are comparing themselves to, when deciding that they might have ADD, are already on ADD meds (or coke...) for exactly that reason. If our society needs 1% of the population to be accountants (to pick an example) but only 0.1% of the population either have incredible focus abilities or don't find accounting brain-meltingly dull, then at least 90% of accountants are going to feel like they have a lot of trouble focusing at work. Once enough start medicating (legally or otherwise) it's gonna feel to others like they really do have a condition that most don't, but they both kinda do (in a practical sense, they do need to focus better to keep up with their peers) and kinda don't (in that it's sort of our society that's sick, not them—they're just acting like most people would, in that situation).
  - jrumbut 5y ago
    My poorly informed impression is that the key challenge of any data driven investigation is striking the balance between how hard something is to measure and how close it is to what you really want to know. The marshmallow test was so appealing because it was incredibly easy to perform and seemed like it was pretty close to a measure of the kind of self-control and discipline that's needed to succeed in a variety of life's most important challenges.
- frgtpsswrdlame 5y ago
  >But maybe in the days of phrenology, people thought a hooked nose had a plausible connection to personality as well? Phrenology is bumps on the head right? I think hook nose would be physiognomy. But yes, the idea was that your behavior was due to your brain and your brain was composed of many different parts that each controlled different propensities or abilities. Then it was just a matter of identifying where those propensities lived, in relation to the head and then you could feel for the differences from person to person across the surface of the head. From the naive viewpoint it is plausible, oh, you say the back right section of my head, above the ear is a bit larger so the self-control portion of my brain is well developed? I've always thought so! Setting aside the marshmallow test, you can easily see how scientific theories about this sort of thing, both right and wrong, easily integrate.
DeusExMachina 5y ago
Given the current replication crisis, I would say, not much. https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
gumby 5y ago
Yes, I think it’s better, for the reason this article explains: people are following up and revisiting the conclusions. Nutrition studies are more like phrenology in that they are so hard to do with so many confounding factors that you can’t really trust any macro conclusion.
- yboris 5y ago
  I think your distrust in nutrition studies might stem from the fact there are nefarious entities publishing things. Various industries have a financial interest in making it look like their product, because it contains substance X, is beneficial to people. So they can design the most flimsy experiment with no pre-registration, and re-run it numerous times until they get the result they want. Lots of conclusions from nutrition studies (especially meta analyses) are robust and useful to follow. Consider the NutritionFacts website as a good starting point: a non-profit which has no ads, no industry "partners", etc - focused on distilling well-designed studies to see what everyday people can use from them. https://nutritionfacts.org/ https://nutritionfacts.org/
  - gumby 5y ago
    I was not even considering the issue of bad actors. Simply that longitudinal, multi-variate studies of sufficient scale are essentially impossible to conduct. Even though nutrition is one of the very oldest, and perhaps the very oldest, fields of human study, it still remains in the “butterfly collecting” phase of development as a science. It’s very very hard. I’m glad some people try.
lr4444lr 5y ago
Delayed gratification AFAIK has solid research as a trait predictive of many things. That a child's ability at 4 or 5 to do it being predictive of their adult self is something else, though.
- TrinaryWorksToo 5y ago
  There could easily be confounders to that though. Like people who are wealthy might be able to delay satisfaction better than poor, because their needs are more satisfied.
  - dahart 5y ago
    Indeed, and the article mentions this. “The Watts study findings support a common criticism of the marshmallow test: that waiting out temptation for a later reward is largely a middle or upper class behavior. If you come from a place of shortages and broken promises, eating the treat in front of you now might be the better bet than trusting there will be more later.”
    - fancifalmanima 5y ago
      To say this more explicitly, even the idea of waiting for the second marshmallow being the "preferred" behavior is somewhat classist. Sounds more like the test is just testing for an adaptation that happens to be well suited to living in a upper-middle class to wealthy environment. If resources are scarce, the kid that takes what they can get now rather than trusting other people will do better in the long run.
  - nostrademons 5y ago
    A lot of observable phenomena function as positive feedback loops, simply because positive feedback loops are usually needed to generate effect sizes that become "observable" beyond individual variation. It's very likely the being able to delay gratification makes you wealthy, which makes you better able to delay gratification, which makes you wealthier, and so on. And that's why we have discernible social classes, where mobility from one to another becomes very difficult. Breaking the feedback loop usually involves doing something farsighted, risky, and irrational - for example, risking getting fired from your retail job by studying programming and applying to software engineering jobs in your downtime, or quitting your stable corporate job to found a startup.

Am I overreacting to consider these kinds of psychometric studies to be not much better than phrenology? "Behavioral phrenology" maybe?

by api 5y ago

Am I overreacting to consider these kinds of psychometric studies to be not much better than phrenology? "Behavioral phrenology" maybe?

civilized 5y ago

It's an interesting idea. But to me the marshmallow test at least had some plausible connection to personality. But maybe in the days of phrenology, people thought a hooked nose* had a plausible connection to personality as well? Weird to think about. *Sorry, this is physiognomy not phrenology. The same basic point stands though.

well_i_guess 5y ago
I think that the issue is that there is no true metric for “highly marketable talents/traits.” One generations genius could be another generations average worker, solely because market forces eliminate the competitive advantage of certain things. Many, many authors seem to lament the distractability of the current generations yet I would bet you many of the most famous people to Gen Z are incredibly attention-fickle. Whereas, 20 years ago, focus would probably be an essential skill for key performers.
- fancifalmanima 5y ago
  Focus is almost surely an essential skill for key performers. Even among the most famous Gen Z -- you don't think they focus on their social media presence and what they do? What is an 8 hour photo shoot if not focusing? A lot of work goes into what social media influencers post, its not all done on a whim. There's also plenty of Gen Z doing other more traditional work (almost everyone of that generation, really). If anything, they've probably had to develop coping mechanisms from an extremely early age to deal with distraction, compared to prior generations.
  - brimble 5y ago
    I'm reminded of the SlateStarCodex post that mulls over the difference between "real" ADD and just having totally ordinary (but pretty great) difficulty focusing on the exact same boring crap on a computer screen day, after day, after day—especially if, in the latter case, a lot of the people these folks are comparing themselves to, when deciding that they might have ADD, are already on ADD meds (or coke...) for exactly that reason. If our society needs 1% of the population to be accountants (to pick an example) but only 0.1% of the population either have incredible focus abilities or don't find accounting brain-meltingly dull, then at least 90% of accountants are going to feel like they have a lot of trouble focusing at work. Once enough start medicating (legally or otherwise) it's gonna feel to others like they really do have a condition that most don't, but they both kinda do (in a practical sense, they do need to focus better to keep up with their peers) and kinda don't (in that it's sort of our society that's sick, not them—they're just acting like most people would, in that situation).
- jrumbut 5y ago
  My poorly informed impression is that the key challenge of any data driven investigation is striking the balance between how hard something is to measure and how close it is to what you really want to know. The marshmallow test was so appealing because it was incredibly easy to perform and seemed like it was pretty close to a measure of the kind of self-control and discipline that's needed to succeed in a variety of life's most important challenges.
frgtpsswrdlame 5y ago
>But maybe in the days of phrenology, people thought a hooked nose had a plausible connection to personality as well? Phrenology is bumps on the head right? I think hook nose would be physiognomy. But yes, the idea was that your behavior was due to your brain and your brain was composed of many different parts that each controlled different propensities or abilities. Then it was just a matter of identifying where those propensities lived, in relation to the head and then you could feel for the differences from person to person across the surface of the head. From the naive viewpoint it is plausible, oh, you say the back right section of my head, above the ear is a bit larger so the self-control portion of my brain is well developed? I've always thought so! Setting aside the marshmallow test, you can easily see how scientific theories about this sort of thing, both right and wrong, easily integrate.

DeusExMachina 5y ago

Given the current replication crisis, I would say, not much. https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis

gumby 5y ago

Yes, I think it’s better, for the reason this article explains: people are following up and revisiting the conclusions. Nutrition studies are more like phrenology in that they are so hard to do with so many confounding factors that you can’t really trust any macro conclusion.

yboris 5y ago
I think your distrust in nutrition studies might stem from the fact there are nefarious entities publishing things. Various industries have a financial interest in making it look like their product, because it contains substance X, is beneficial to people. So they can design the most flimsy experiment with no pre-registration, and re-run it numerous times until they get the result they want. Lots of conclusions from nutrition studies (especially meta analyses) are robust and useful to follow. Consider the NutritionFacts website as a good starting point: a non-profit which has no ads, no industry "partners", etc - focused on distilling well-designed studies to see what everyday people can use from them. https://nutritionfacts.org/ https://nutritionfacts.org/
- gumby 5y ago
  I was not even considering the issue of bad actors. Simply that longitudinal, multi-variate studies of sufficient scale are essentially impossible to conduct. Even though nutrition is one of the very oldest, and perhaps the very oldest, fields of human study, it still remains in the “butterfly collecting” phase of development as a science. It’s very very hard. I’m glad some people try.

lr4444lr 5y ago

Delayed gratification AFAIK has solid research as a trait predictive of many things. That a child's ability at 4 or 5 to do it being predictive of their adult self is something else, though.

TrinaryWorksToo 5y ago
There could easily be confounders to that though. Like people who are wealthy might be able to delay satisfaction better than poor, because their needs are more satisfied.
- dahart 5y ago
  Indeed, and the article mentions this. “The Watts study findings support a common criticism of the marshmallow test: that waiting out temptation for a later reward is largely a middle or upper class behavior. If you come from a place of shortages and broken promises, eating the treat in front of you now might be the better bet than trusting there will be more later.”
  - fancifalmanima 5y ago
    To say this more explicitly, even the idea of waiting for the second marshmallow being the "preferred" behavior is somewhat classist. Sounds more like the test is just testing for an adaptation that happens to be well suited to living in a upper-middle class to wealthy environment. If resources are scarce, the kid that takes what they can get now rather than trusting other people will do better in the long run.
- nostrademons 5y ago
  A lot of observable phenomena function as positive feedback loops, simply because positive feedback loops are usually needed to generate effect sizes that become "observable" beyond individual variation. It's very likely the being able to delay gratification makes you wealthy, which makes you better able to delay gratification, which makes you wealthier, and so on. And that's why we have discernible social classes, where mobility from one to another becomes very difficult. Breaking the feedback loop usually involves doing something farsighted, risky, and irrational - for example, risking getting fired from your retail job by studying programming and applying to software engineering jobs in your downtime, or quitting your stable corporate job to found a startup.