7 ms·

New study disavows marshmallow test’s predictive powers

109 points by npalli 5y ago

api 5y ago
Am I overreacting to consider these kinds of psychometric studies to be not much better than phrenology? "Behavioral phrenology" maybe?
- civilized 5y ago
  It's an interesting idea. But to me the marshmallow test at least had some plausible connection to personality. But maybe in the days of phrenology, people thought a hooked nose* had a plausible connection to personality as well? Weird to think about. *Sorry, this is physiognomy not phrenology. The same basic point stands though.
  - well_i_guess 5y ago
    I think that the issue is that there is no true metric for “highly marketable talents/traits.” One generations genius could be another generations average worker, solely because market forces eliminate the competitive advantage of certain things. Many, many authors seem to lament the distractability of the current generations yet I would bet you many of the most famous people to Gen Z are incredibly attention-fickle. Whereas, 20 years ago, focus would probably be an essential skill for key performers.
    - fancifalmanima 5y ago
      Focus is almost surely an essential skill for key performers. Even among the most famous Gen Z -- you don't think they focus on their social media presence and what they do? What is an 8 hour photo shoot if not focusing? A lot of work goes into what social media influencers post, its not all done on a whim. There's also plenty of Gen Z doing other more traditional work (almost everyone of that generation, really). If anything, they've probably had to develop coping mechanisms from an extremely early age to deal with distraction, compared to prior generations.
      - brimble 5y ago
        I'm reminded of the SlateStarCodex post that mulls over the difference between "real" ADD and just having totally ordinary (but pretty great) difficulty focusing on the exact same boring crap on a computer screen day, after day, after day—especially if, in the latter case, a lot of the people these folks are comparing themselves to, when deciding that they might have ADD, are already on ADD meds (or coke...) for exactly that reason. If our society needs 1% of the population to be accountants (to pick an example) but only 0.1% of the population either have incredible focus abilities or don't find accounting brain-meltingly dull, then at least 90% of accountants are going to feel like they have a lot of trouble focusing at work. Once enough start medicating (legally or otherwise) it's gonna feel to others like they really do have a condition that most don't, but they both kinda do (in a practical sense, they do need to focus better to keep up with their peers) and kinda don't (in that it's sort of our society that's sick, not them—they're just acting like most people would, in that situation).
    - jrumbut 5y ago
      My poorly informed impression is that the key challenge of any data driven investigation is striking the balance between how hard something is to measure and how close it is to what you really want to know. The marshmallow test was so appealing because it was incredibly easy to perform and seemed like it was pretty close to a measure of the kind of self-control and discipline that's needed to succeed in a variety of life's most important challenges.
  - frgtpsswrdlame 5y ago
    >But maybe in the days of phrenology, people thought a hooked nose had a plausible connection to personality as well? Phrenology is bumps on the head right? I think hook nose would be physiognomy. But yes, the idea was that your behavior was due to your brain and your brain was composed of many different parts that each controlled different propensities or abilities. Then it was just a matter of identifying where those propensities lived, in relation to the head and then you could feel for the differences from person to person across the surface of the head. From the naive viewpoint it is plausible, oh, you say the back right section of my head, above the ear is a bit larger so the self-control portion of my brain is well developed? I've always thought so! Setting aside the marshmallow test, you can easily see how scientific theories about this sort of thing, both right and wrong, easily integrate.
- DeusExMachina 5y ago
  Given the current replication crisis, I would say, not much. https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
- gumby 5y ago
  Yes, I think it’s better, for the reason this article explains: people are following up and revisiting the conclusions. Nutrition studies are more like phrenology in that they are so hard to do with so many confounding factors that you can’t really trust any macro conclusion.
  - yboris 5y ago
    I think your distrust in nutrition studies might stem from the fact there are nefarious entities publishing things. Various industries have a financial interest in making it look like their product, because it contains substance X, is beneficial to people. So they can design the most flimsy experiment with no pre-registration, and re-run it numerous times until they get the result they want. Lots of conclusions from nutrition studies (especially meta analyses) are robust and useful to follow. Consider the NutritionFacts website as a good starting point: a non-profit which has no ads, no industry "partners", etc - focused on distilling well-designed studies to see what everyday people can use from them. https://nutritionfacts.org/ https://nutritionfacts.org/
    - gumby 5y ago
      I was not even considering the issue of bad actors. Simply that longitudinal, multi-variate studies of sufficient scale are essentially impossible to conduct. Even though nutrition is one of the very oldest, and perhaps the very oldest, fields of human study, it still remains in the “butterfly collecting” phase of development as a science. It’s very very hard. I’m glad some people try.
- lr4444lr 5y ago
  Delayed gratification AFAIK has solid research as a trait predictive of many things. That a child's ability at 4 or 5 to do it being predictive of their adult self is something else, though.
  - TrinaryWorksToo 5y ago
    There could easily be confounders to that though. Like people who are wealthy might be able to delay satisfaction better than poor, because their needs are more satisfied.
    - dahart 5y ago
      Indeed, and the article mentions this. “The Watts study findings support a common criticism of the marshmallow test: that waiting out temptation for a later reward is largely a middle or upper class behavior. If you come from a place of shortages and broken promises, eating the treat in front of you now might be the better bet than trusting there will be more later.”
      - fancifalmanima 5y ago
        To say this more explicitly, even the idea of waiting for the second marshmallow being the "preferred" behavior is somewhat classist. Sounds more like the test is just testing for an adaptation that happens to be well suited to living in a upper-middle class to wealthy environment. If resources are scarce, the kid that takes what they can get now rather than trusting other people will do better in the long run.
    - nostrademons 5y ago
      A lot of observable phenomena function as positive feedback loops, simply because positive feedback loops are usually needed to generate effect sizes that become "observable" beyond individual variation. It's very likely the being able to delay gratification makes you wealthy, which makes you better able to delay gratification, which makes you wealthier, and so on. And that's why we have discernible social classes, where mobility from one to another becomes very difficult. Breaking the feedback loop usually involves doing something farsighted, risky, and irrational - for example, risking getting fired from your retail job by studying programming and applying to software engineering jobs in your downtime, or quitting your stable corporate job to found a startup.
erichocean 5y ago
The article, and especially the headline, are extremely misleading. The actual result: measures of self-control either weakly or strongly predict positive life outcomes, depending on the measure and how much adjusting was done, e.g. > [The study] created a new measure of the time each original preschooler waited before taking a bite (or getting the reward) to adjust for variables such as age, gender and experiment conditions. This study found that the "marshmallow test"—as a single measure—is no more or less predictive than a basket of other measures of self-control the study tested, or any of those other measures of self-control taken alone. Despite the misleading article and headline, the study itself seems well-designed (e.g. pre-registered), but the conclusion in the headline is utterly wrong as that is not what the study found: self-control matters, can be measured, and those measures weakly or strongly predict positive life outcomes. Here's an accurate headline: Self-control still predicts positive life outcomes, Marshmallow Test creator finds.
- feanaro 5y ago
  Yes, but how would you then inanely riff on psychology, which all the cool kids are doing nowadays?
  - deleted 5y ago
    [deleted]
- KerrAvon 5y ago
  The original headline isn't inaccurate, though it is clumsily worded and neither it nor your proposed headline fully describe the results of the study. The study itself says the following (quoting verbatim). Note that the second point is more or less what the headline says. - Self-regulation composite (preschool & ages 17-37) predicts capital formation at 46. - Preschool delay of gratification alone does not predict capital formation at 46. - The composite is more predictive partly because it consists of many items. - No evidence of more predictive power for self-regulation reported later in life.
- nefitty 5y ago
  Thank you for the clarification. You've thought about this a lot. If a friend asked you for advice on how to help their struggling teenage son improve his self-control, what do you think you would say?
karaterobot 5y ago
A good test for whether a psychological or sociological study may turn out to be hard to replicate is: does it make a sweeping claim about something as complex as human beings? If it's not tentative, incremental, wrapped in caveats and conditionals, I don't put much weight in it anymore.
- goatlover 5y ago
  Pretty much this.
- chinchilla2020 5y ago
  Also, they only tested each child once. The same child might behave different based on how hungry they are every time it is offered.
acchow 5y ago
From the Journal Article: "They included a total of 550 students from Stanford's Bing Nursery School, aged about 4 years old (ranging from 2 to 6). Many of the participants are children of Stanford faculty and staff." https://www.sciencedirect.com/science/article/pii/S0167268119302641?via%3Dihub#bib0024 https://www.sciencedirect.com/science/article/pii/S016726811... How can we conclude anything at all about the general population using a sample of Stanford kids?
- gkop 5y ago
  This isn’t my field, but I think this is par for the course unfortunately and a manifestation of a larger issue. Eg https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0168354&type=printable https://journals.plos.org/plosone/article/file?id=10.1371/jo...
- dahart 5y ago
  I like to ask this whenever Dunning Kruger comes up; DK was a sample of Cornell undergrads, and the study was one tenth the size of the Stanford study. DK participants were volunteers of a psych class who got extra credit. Presumably they needed and could use extra credit, which may have excluded the A and the F students. It’s hard to imagine ways to start with more bias, or how we can possibly accept this sample as representative of humanity.
- ren_engineer 5y ago
  Social sciences are all a sham, now think about the trillions of dollars of government spending that are based on that same sham science as justification. People wonder why so many government programs fail, it's because they are built on a rotten foundation https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis >How can we conclude anything at all about the general population using a sample of Stanford kids? it's a well known problem that is rarely brought up, WEIRD bias. Most social science research participants are college students being bribed with extra credit or gift cards https://en.wikipedia.org/wiki/Psychology#WEIRD_bias https://en.wikipedia.org/wiki/Psychology#WEIRD_bias
  - 5y ago
brimble 5y ago
I've got some pretty good predictive powers, myself. I predict that in twenty years, no matter how thoroughly this is debunked, I'll still see this treated as true constantly, and, even when what's under discussion is taking action based on its being true, I'll only get eye-rolls and head-pats and plain disapproval/loss-of-face for bringing up that it's questionable at best, then everyone will go on treating it as true.
mansoon 5y ago
This deserves better study.
ramesh31 5y ago
Sure. And fifty years worth of other studies have shown its' effectiveness. This is meaningless noise in the absence of meta analysis.
- awb 5y ago
  Related meta analysis: https://addictions.psych.ucla.edu/wp-content/uploads/sites/160/2018/01/PsychoPharm-Delayed-Reward-Discounting-and-Addictive-Behavior-a-Meta-Analysis.pdf https://addictions.psych.ucla.edu/wp-content/uploads/sites/1... > Conclusions These results provide strong evidence of greater DRD in individuals exhibiting addictive behavior in general and particularly in individuals who meet criteria for an addictive disorder. They don’t draw conclusions about causative success, just a correlation with addiction.
dang 5y ago
All: if you're going to post here, can please make sure you're not posting a shallow dismissal? Those are the quickest and easiest reactions to post, but they're repetitive and boring. This site is supposed to be for interesting conversation, and that requires new information—not things we've all heard before. Hint: if you're making a strong, large statement—e.g. an emphatic claim about an entire category of things—then it's most likely a shallow comment. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
awb 5y ago
Here’s a 2011 meta analysis that reports that DRD (delayed reward discounting — basically putting lower importance on delayed gratification and instead putting greater importance on immediate rewards) is highly associated with addictive personalities: https://addictions.psych.ucla.edu/wp-content/uploads/sites/160/2018/01/PsychoPharm-Delayed-Reward-Discounting-and-Addictive-Behavior-a-Meta-Analysis.pdf https://addictions.psych.ucla.edu/wp-content/uploads/sites/1... > Conclusions These results provide strong evidence of greater DRD in individuals exhibiting addictive behavior in general and particularly in individuals who meet criteria for an addictive disorder.
scotuswroteus 5y ago
David Brooks in shambles
learn_more 5y ago
Sounds like it is predictive. Just not when: > Controlling for differences such as household income and cognitive abilities ... So it's a (predictive) IQ test. Perhaps it disavows the prior assumed basis of "deferred gratification", but not the predictive power of the test.
suzzer99 5y ago
I've never understood the marshmallow test. I'm supposed to sit there and stare at a delicious marshmallow for some indeterminate amount of time in order to get one extra marshmallow? Offer me a whole bag and we'll talk. I've always wondered if this test measures more of the child's willingness to please the researcher, and not so much their capacity for delayed gratification.
- lifeplusplus 5y ago
  that's the point, 2 is greater than 1, not 100x but still greater, would you choose greater benefit at the cost self gratification or short term gain?
  - swid 5y ago
    I think the other poster has a point - The marginal utility of the second marshmallow is lower. Then there is added cost in the time spent waiting. Even if you have to wait in the room regardless, I think it’s fair to say there is a cost to delaying the gratification. Why one extra? What if I just give you 5% of a marshmallow? 1%? A coherent argument can be made that the extra costs are not worth it. I don’t particularly like marshmallows, but maybe I’ll eat one. Maybe two marshmallows in the future would be higher utility or whatnot than one marshmallow now, but that isn’t always the case.
    - suzzer99 5y ago
      Exactly. I think 4-5 year old me would have weighed the utility of sitting there having a battle of wills with myself for one extra marshmallow, and quickly decided it wasn't worth it. What if I hold out for 10 minutes only to finally give in, and then the researcher returns seconds later? Better to fail fast than risk that. At the moment there seems to be a lot of weight behind the idea that we only have so much willpower to burn in a day. So you might hold out against that tempting box of donuts all day at work, only to break down stop at a bar on the way home for a few pints. I think on some level I've always felt that avoidance is better than fighting when it comes to willpower. First option - avoid the temptation. Second option - give in. Only if #1 isn't feasible and #2 is a very bad outcome, then go to battle.
jl2718 5y ago
“Adding the marshmallow test results to the index does virtually nothing to the prognosis, the study finds.” This does not mean that the test is not predictive. It means that the index (a bunch of measurements) contains statistical dependencies. From a practical view, the marshmallow test result depends on many cognitive factors unrelated to self-control. The child must understand the instructions, remember them for the duration of the test, trust the provider, value the second marshmallow, and then make a decision. To be of any value, it should have been tested against a standard cognitive battery, which it almost certainly would have failed to improve upon. Cognitive tests have worked extremely well to predict life outcomes for decades now if not centuries.
rilezg 5y ago
I think the 'golden goose award' page from 2015 (linked in the article) gives a better overview of the original research than the article: https://www.goldengooseaward.org/01awardees/marshmallowtest https://www.goldengooseaward.org/01awardees/marshmallowtest A small quote: "But this is not a story of fate – of children’s long-term success being determined by their self-control as four-year-olds. It is a story about how children can change: those who are “low delayers” can in fact learn to be “high delayers,” and gain the life benefits that self-control imparts." So this is more olds than news, but perhaps it is good to be reminded that we all have room to grow (or shrink) from who we were at age 4. I personally would bet high-delayers can also learn to become low-delayers, and I also would bet there are times in life when you would be better off eating the marshmallow now instead of investing it for another 30 years at 5% because the man in the suit told you to.
renewiltord 5y ago
Oh interesting. Self-regulation is correlated with good outcomes but the marshmallow test is a poor test of self regulation. Okay, interesting. What I would enjoy, I think, is taking a monthly test battery and uploading that to a central database with other self-researchers and then looking at that in a historic sense to derive ideas to study. Obviously, since one is post-hoc slicing one will find many spurious correlations but perhaps these correlations will yield interesting areas to search around. Does anyone know of anything like this?
treis 5y ago
It's just wild how bad people can be at their job and sometimes no one stops them. And this guy even manage to fail his way to success.
rahimnathwani 5y ago
IIRC one theory is that parental wealth may be a confounding variable. Roughly speaking, if you grow up in a poor household, you're more likely to face unpredictable situations, including situations in which promises cannot be kept (e.g. due to unexpected financial shocks). So, if you're offered a marshmallow now or two later, you'll rationally (based on past experience) go for the sure thing. I don't remember whether I read this in relation to the original study, or a follow-up. But it sounds more plausible than the original explanation.
- s1artibartfast 5y ago
  That can all be true and be compatible with the fact that the ability to delay gratification provides certain advantages.
  - deleted 5y ago
    [deleted]
  - rahimnathwani 5y ago
    Right, but the original experiment did not measure (A) 'the ability to delay gratification'. It measured (B) the decision children took in the face of uncertainty. Only if all children assessed the chance of getting the promised second marshmallow as ~100%, would be (B) be a reasonable proxy for (A).
    - s1artibartfast 5y ago
      I would argue that B is a subset of A. If you can't delay gratification because your mental model is an inconsistant world with broken promises, you still failed to delay gratification. B may explain the response to A, and be rational based on personal experience, but the action was still taken. The question of why some children can delay gratification is different that what how predictive delayed gratification is of life outcomes. It can also be true that broken promises are a better prediction of future outcomes.
      - 5y ago
jollybean 5y ago
"The new study may be a final blow to destiny implications formed in marshmallow test research," The broader implications with respect to the 'integrity' of the scientific work in various disciplines here is almost the whole story. That we can't count on 'the marshmellow' test is marginal relative to how these issues add up and how we don't seem to have a strategy or set of tactics to address the reality of the 'fuzziness' of science, especially in an era where we need to convince people that 'vaccines are safe'. I don't agree with dang's thesis that the generalisation is 'shallow'. I think the generalisation is the story here. It's the new, perennial issue. We are witnessing a new phase in the evolution of Enlightenment values particularly because we are coming to terms with the reality of a lot of what is being published these days, it's the heart of the matter. We need much more dialogue about this, especially institutionally.

109 points by npalli 5y ago

api 5y ago

Am I overreacting to consider these kinds of psychometric studies to be not much better than phrenology? "Behavioral phrenology" maybe?

civilized 5y ago
It's an interesting idea. But to me the marshmallow test at least had some plausible connection to personality. But maybe in the days of phrenology, people thought a hooked nose* had a plausible connection to personality as well? Weird to think about. *Sorry, this is physiognomy not phrenology. The same basic point stands though.
- well_i_guess 5y ago
  I think that the issue is that there is no true metric for “highly marketable talents/traits.” One generations genius could be another generations average worker, solely because market forces eliminate the competitive advantage of certain things. Many, many authors seem to lament the distractability of the current generations yet I would bet you many of the most famous people to Gen Z are incredibly attention-fickle. Whereas, 20 years ago, focus would probably be an essential skill for key performers.
  - fancifalmanima 5y ago
    Focus is almost surely an essential skill for key performers. Even among the most famous Gen Z -- you don't think they focus on their social media presence and what they do? What is an 8 hour photo shoot if not focusing? A lot of work goes into what social media influencers post, its not all done on a whim. There's also plenty of Gen Z doing other more traditional work (almost everyone of that generation, really). If anything, they've probably had to develop coping mechanisms from an extremely early age to deal with distraction, compared to prior generations.
    - brimble 5y ago
      I'm reminded of the SlateStarCodex post that mulls over the difference between "real" ADD and just having totally ordinary (but pretty great) difficulty focusing on the exact same boring crap on a computer screen day, after day, after day—especially if, in the latter case, a lot of the people these folks are comparing themselves to, when deciding that they might have ADD, are already on ADD meds (or coke...) for exactly that reason. If our society needs 1% of the population to be accountants (to pick an example) but only 0.1% of the population either have incredible focus abilities or don't find accounting brain-meltingly dull, then at least 90% of accountants are going to feel like they have a lot of trouble focusing at work. Once enough start medicating (legally or otherwise) it's gonna feel to others like they really do have a condition that most don't, but they both kinda do (in a practical sense, they do need to focus better to keep up with their peers) and kinda don't (in that it's sort of our society that's sick, not them—they're just acting like most people would, in that situation).
  - jrumbut 5y ago
    My poorly informed impression is that the key challenge of any data driven investigation is striking the balance between how hard something is to measure and how close it is to what you really want to know. The marshmallow test was so appealing because it was incredibly easy to perform and seemed like it was pretty close to a measure of the kind of self-control and discipline that's needed to succeed in a variety of life's most important challenges.
- frgtpsswrdlame 5y ago
  >But maybe in the days of phrenology, people thought a hooked nose had a plausible connection to personality as well? Phrenology is bumps on the head right? I think hook nose would be physiognomy. But yes, the idea was that your behavior was due to your brain and your brain was composed of many different parts that each controlled different propensities or abilities. Then it was just a matter of identifying where those propensities lived, in relation to the head and then you could feel for the differences from person to person across the surface of the head. From the naive viewpoint it is plausible, oh, you say the back right section of my head, above the ear is a bit larger so the self-control portion of my brain is well developed? I've always thought so! Setting aside the marshmallow test, you can easily see how scientific theories about this sort of thing, both right and wrong, easily integrate.
DeusExMachina 5y ago
Given the current replication crisis, I would say, not much. https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
gumby 5y ago
Yes, I think it’s better, for the reason this article explains: people are following up and revisiting the conclusions. Nutrition studies are more like phrenology in that they are so hard to do with so many confounding factors that you can’t really trust any macro conclusion.
- yboris 5y ago
  I think your distrust in nutrition studies might stem from the fact there are nefarious entities publishing things. Various industries have a financial interest in making it look like their product, because it contains substance X, is beneficial to people. So they can design the most flimsy experiment with no pre-registration, and re-run it numerous times until they get the result they want. Lots of conclusions from nutrition studies (especially meta analyses) are robust and useful to follow. Consider the NutritionFacts website as a good starting point: a non-profit which has no ads, no industry "partners", etc - focused on distilling well-designed studies to see what everyday people can use from them. https://nutritionfacts.org/ https://nutritionfacts.org/
  - gumby 5y ago
    I was not even considering the issue of bad actors. Simply that longitudinal, multi-variate studies of sufficient scale are essentially impossible to conduct. Even though nutrition is one of the very oldest, and perhaps the very oldest, fields of human study, it still remains in the “butterfly collecting” phase of development as a science. It’s very very hard. I’m glad some people try.
lr4444lr 5y ago
Delayed gratification AFAIK has solid research as a trait predictive of many things. That a child's ability at 4 or 5 to do it being predictive of their adult self is something else, though.
- TrinaryWorksToo 5y ago
  There could easily be confounders to that though. Like people who are wealthy might be able to delay satisfaction better than poor, because their needs are more satisfied.
  - dahart 5y ago
    Indeed, and the article mentions this. “The Watts study findings support a common criticism of the marshmallow test: that waiting out temptation for a later reward is largely a middle or upper class behavior. If you come from a place of shortages and broken promises, eating the treat in front of you now might be the better bet than trusting there will be more later.”
    - fancifalmanima 5y ago
      To say this more explicitly, even the idea of waiting for the second marshmallow being the "preferred" behavior is somewhat classist. Sounds more like the test is just testing for an adaptation that happens to be well suited to living in a upper-middle class to wealthy environment. If resources are scarce, the kid that takes what they can get now rather than trusting other people will do better in the long run.
  - nostrademons 5y ago
    A lot of observable phenomena function as positive feedback loops, simply because positive feedback loops are usually needed to generate effect sizes that become "observable" beyond individual variation. It's very likely the being able to delay gratification makes you wealthy, which makes you better able to delay gratification, which makes you wealthier, and so on. And that's why we have discernible social classes, where mobility from one to another becomes very difficult. Breaking the feedback loop usually involves doing something farsighted, risky, and irrational - for example, risking getting fired from your retail job by studying programming and applying to software engineering jobs in your downtime, or quitting your stable corporate job to found a startup.

erichocean 5y ago

The article, and especially the headline, are extremely misleading. The actual result: measures of self-control either weakly or strongly predict positive life outcomes, depending on the measure and how much adjusting was done, e.g. > [The study] created a new measure of the time each original preschooler waited before taking a bite (or getting the reward) to adjust for variables such as age, gender and experiment conditions. This study found that the "marshmallow test"—as a single measure—is no more or less predictive than a basket of other measures of self-control the study tested, or any of those other measures of self-control taken alone. Despite the misleading article and headline, the study itself seems well-designed (e.g. pre-registered), but the conclusion in the headline is utterly wrong as that is not what the study found: self-control matters, can be measured, and those measures weakly or strongly predict positive life outcomes. Here's an accurate headline: Self-control still predicts positive life outcomes, Marshmallow Test creator finds.

feanaro 5y ago
Yes, but how would you then inanely riff on psychology, which all the cool kids are doing nowadays?
- deleted 5y ago
  [deleted]
KerrAvon 5y ago
The original headline isn't inaccurate, though it is clumsily worded and neither it nor your proposed headline fully describe the results of the study. The study itself says the following (quoting verbatim). Note that the second point is more or less what the headline says. - Self-regulation composite (preschool & ages 17-37) predicts capital formation at 46. - Preschool delay of gratification alone does not predict capital formation at 46. - The composite is more predictive partly because it consists of many items. - No evidence of more predictive power for self-regulation reported later in life.
nefitty 5y ago
Thank you for the clarification. You've thought about this a lot. If a friend asked you for advice on how to help their struggling teenage son improve his self-control, what do you think you would say?

karaterobot 5y ago

A good test for whether a psychological or sociological study may turn out to be hard to replicate is: does it make a sweeping claim about something as complex as human beings? If it's not tentative, incremental, wrapped in caveats and conditionals, I don't put much weight in it anymore.

goatlover 5y ago
Pretty much this.
chinchilla2020 5y ago
Also, they only tested each child once. The same child might behave different based on how hungry they are every time it is offered.

acchow 5y ago

From the Journal Article: "They included a total of 550 students from Stanford's Bing Nursery School, aged about 4 years old (ranging from 2 to 6). Many of the participants are children of Stanford faculty and staff." https://www.sciencedirect.com/science/article/pii/S0167268119302641?via%3Dihub#bib0024 https://www.sciencedirect.com/science/article/pii/S016726811... How can we conclude anything at all about the general population using a sample of Stanford kids?

gkop 5y ago
This isn’t my field, but I think this is par for the course unfortunately and a manifestation of a larger issue. Eg https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0168354&type=printable https://journals.plos.org/plosone/article/file?id=10.1371/jo...
dahart 5y ago
I like to ask this whenever Dunning Kruger comes up; DK was a sample of Cornell undergrads, and the study was one tenth the size of the Stanford study. DK participants were volunteers of a psych class who got extra credit. Presumably they needed and could use extra credit, which may have excluded the A and the F students. It’s hard to imagine ways to start with more bias, or how we can possibly accept this sample as representative of humanity.
ren_engineer 5y ago
Social sciences are all a sham, now think about the trillions of dollars of government spending that are based on that same sham science as justification. People wonder why so many government programs fail, it's because they are built on a rotten foundation https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis >How can we conclude anything at all about the general population using a sample of Stanford kids? it's a well known problem that is rarely brought up, WEIRD bias. Most social science research participants are college students being bribed with extra credit or gift cards https://en.wikipedia.org/wiki/Psychology#WEIRD_bias https://en.wikipedia.org/wiki/Psychology#WEIRD_bias
- 5y ago

brimble 5y ago

I've got some pretty good predictive powers, myself. I predict that in twenty years, no matter how thoroughly this is debunked, I'll still see this treated as true constantly, and, even when what's under discussion is taking action based on its being true, I'll only get eye-rolls and head-pats and plain disapproval/loss-of-face for bringing up that it's questionable at best, then everyone will go on treating it as true.

mansoon 5y ago

This deserves better study.

ramesh31 5y ago

Sure. And fifty years worth of other studies have shown its' effectiveness. This is meaningless noise in the absence of meta analysis.

awb 5y ago
Related meta analysis: https://addictions.psych.ucla.edu/wp-content/uploads/sites/160/2018/01/PsychoPharm-Delayed-Reward-Discounting-and-Addictive-Behavior-a-Meta-Analysis.pdf https://addictions.psych.ucla.edu/wp-content/uploads/sites/1... > Conclusions These results provide strong evidence of greater DRD in individuals exhibiting addictive behavior in general and particularly in individuals who meet criteria for an addictive disorder. They don’t draw conclusions about causative success, just a correlation with addiction.

dang 5y ago

All: if you're going to post here, can please make sure you're not posting a shallow dismissal? Those are the quickest and easiest reactions to post, but they're repetitive and boring. This site is supposed to be for interesting conversation, and that requires new information—not things we've all heard before. Hint: if you're making a strong, large statement—e.g. an emphatic claim about an entire category of things—then it's most likely a shallow comment. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

awb 5y ago

Here’s a 2011 meta analysis that reports that DRD (delayed reward discounting — basically putting lower importance on delayed gratification and instead putting greater importance on immediate rewards) is highly associated with addictive personalities: https://addictions.psych.ucla.edu/wp-content/uploads/sites/160/2018/01/PsychoPharm-Delayed-Reward-Discounting-and-Addictive-Behavior-a-Meta-Analysis.pdf https://addictions.psych.ucla.edu/wp-content/uploads/sites/1... > Conclusions These results provide strong evidence of greater DRD in individuals exhibiting addictive behavior in general and particularly in individuals who meet criteria for an addictive disorder.

scotuswroteus 5y ago

David Brooks in shambles

learn_more 5y ago

Sounds like it is predictive. Just not when: > Controlling for differences such as household income and cognitive abilities ... So it's a (predictive) IQ test. Perhaps it disavows the prior assumed basis of "deferred gratification", but not the predictive power of the test.

suzzer99 5y ago

I've never understood the marshmallow test. I'm supposed to sit there and stare at a delicious marshmallow for some indeterminate amount of time in order to get one extra marshmallow? Offer me a whole bag and we'll talk. I've always wondered if this test measures more of the child's willingness to please the researcher, and not so much their capacity for delayed gratification.

lifeplusplus 5y ago
that's the point, 2 is greater than 1, not 100x but still greater, would you choose greater benefit at the cost self gratification or short term gain?
- swid 5y ago
  I think the other poster has a point - The marginal utility of the second marshmallow is lower. Then there is added cost in the time spent waiting. Even if you have to wait in the room regardless, I think it’s fair to say there is a cost to delaying the gratification. Why one extra? What if I just give you 5% of a marshmallow? 1%? A coherent argument can be made that the extra costs are not worth it. I don’t particularly like marshmallows, but maybe I’ll eat one. Maybe two marshmallows in the future would be higher utility or whatnot than one marshmallow now, but that isn’t always the case.
  - suzzer99 5y ago
    Exactly. I think 4-5 year old me would have weighed the utility of sitting there having a battle of wills with myself for one extra marshmallow, and quickly decided it wasn't worth it. What if I hold out for 10 minutes only to finally give in, and then the researcher returns seconds later? Better to fail fast than risk that. At the moment there seems to be a lot of weight behind the idea that we only have so much willpower to burn in a day. So you might hold out against that tempting box of donuts all day at work, only to break down stop at a bar on the way home for a few pints. I think on some level I've always felt that avoidance is better than fighting when it comes to willpower. First option - avoid the temptation. Second option - give in. Only if #1 isn't feasible and #2 is a very bad outcome, then go to battle.

jl2718 5y ago

“Adding the marshmallow test results to the index does virtually nothing to the prognosis, the study finds.” This does not mean that the test is not predictive. It means that the index (a bunch of measurements) contains statistical dependencies. From a practical view, the marshmallow test result depends on many cognitive factors unrelated to self-control. The child must understand the instructions, remember them for the duration of the test, trust the provider, value the second marshmallow, and then make a decision. To be of any value, it should have been tested against a standard cognitive battery, which it almost certainly would have failed to improve upon. Cognitive tests have worked extremely well to predict life outcomes for decades now if not centuries.

rilezg 5y ago

I think the 'golden goose award' page from 2015 (linked in the article) gives a better overview of the original research than the article: https://www.goldengooseaward.org/01awardees/marshmallowtest https://www.goldengooseaward.org/01awardees/marshmallowtest A small quote: "But this is not a story of fate – of children’s long-term success being determined by their self-control as four-year-olds. It is a story about how children can change: those who are “low delayers” can in fact learn to be “high delayers,” and gain the life benefits that self-control imparts." So this is more olds than news, but perhaps it is good to be reminded that we all have room to grow (or shrink) from who we were at age 4. I personally would bet high-delayers can also learn to become low-delayers, and I also would bet there are times in life when you would be better off eating the marshmallow now instead of investing it for another 30 years at 5% because the man in the suit told you to.

renewiltord 5y ago

Oh interesting. Self-regulation is correlated with good outcomes but the marshmallow test is a poor test of self regulation. Okay, interesting. What I would enjoy, I think, is taking a monthly test battery and uploading that to a central database with other self-researchers and then looking at that in a historic sense to derive ideas to study. Obviously, since one is post-hoc slicing one will find many spurious correlations but perhaps these correlations will yield interesting areas to search around. Does anyone know of anything like this?

treis 5y ago

It's just wild how bad people can be at their job and sometimes no one stops them. And this guy even manage to fail his way to success.

rahimnathwani 5y ago

IIRC one theory is that parental wealth may be a confounding variable. Roughly speaking, if you grow up in a poor household, you're more likely to face unpredictable situations, including situations in which promises cannot be kept (e.g. due to unexpected financial shocks). So, if you're offered a marshmallow now or two later, you'll rationally (based on past experience) go for the sure thing. I don't remember whether I read this in relation to the original study, or a follow-up. But it sounds more plausible than the original explanation.

s1artibartfast 5y ago
That can all be true and be compatible with the fact that the ability to delay gratification provides certain advantages.
- deleted 5y ago
  [deleted]
- rahimnathwani 5y ago
  Right, but the original experiment did not measure (A) 'the ability to delay gratification'. It measured (B) the decision children took in the face of uncertainty. Only if all children assessed the chance of getting the promised second marshmallow as ~100%, would be (B) be a reasonable proxy for (A).
  - s1artibartfast 5y ago
    I would argue that B is a subset of A. If you can't delay gratification because your mental model is an inconsistant world with broken promises, you still failed to delay gratification. B may explain the response to A, and be rational based on personal experience, but the action was still taken. The question of why some children can delay gratification is different that what how predictive delayed gratification is of life outcomes. It can also be true that broken promises are a better prediction of future outcomes.
    - 5y ago

jollybean 5y ago

"The new study may be a final blow to destiny implications formed in marshmallow test research," The broader implications with respect to the 'integrity' of the scientific work in various disciplines here is almost the whole story. That we can't count on 'the marshmellow' test is marginal relative to how these issues add up and how we don't seem to have a strategy or set of tactics to address the reality of the 'fuzziness' of science, especially in an era where we need to convince people that 'vaccines are safe'. I don't agree with dang's thesis that the generalisation is 'shallow'. I think the generalisation is the story here. It's the new, perennial issue. We are witnessing a new phase in the evolution of Enlightenment values particularly because we are coming to terms with the reality of a lot of what is being published these days, it's the heart of the matter. We need much more dialogue about this, especially institutionally.