8 ms·

> The researchers also, when analyzing their test’s results, controlled for certain factors—such as the income of a child’s household Is that a valid thing to

> The researchers also, when analyzing their test’s results, controlled for certain factors—such as the income of a child’s household Is that a valid thing to do? If "will power" was a strongly inherited trait, and the theory is that will power leads to increase household income in adult life, then this controlling would remove most of the signal. The original paper relied partly on a questionnaire that asked the parents how good their kids were at self-control. That seems unlikely to be reliable. They also used SAT scores, which seems like a better measure. The abstract for the new paper doesn't mention what they measured. In conclusion, I'm not sure I learned anything here.

hydrox24 8y ago
They used a lot of different measures. I'll try and summarise the key ones here: Academic achievement was measured using the Woodcock-Johnson Psycho-Educational Battery Revised, a "commonly used measure of cognitive ability and achievement". They used various sub-tests from this battery for measuring maths/english seperately. Behavioural problems were measured by reports from mothers. (They used the 1991 "Child Behavior Checklist") This was apparently done to be consistent with the original report. > [child cognitive indicators were] measured at age 24 months by the Bayley Mental Development Index (MDI; Bayley, 1991) and at age 36 months by the Bracken Basic Concept Scale (BBCS; Bracken, 1984). And the child's home environment was measured by the HOME inventory (Caldwell & Bradley, 1984).
misja111 8y ago
>> Is that a valid thing to do? If "will power" was a strongly inherited trait, and the theory is that will power leads to increase household income in adult life, then this controlling would remove most of the signal. Exactly. The way to control for the income of a child's household while not filtering out a possible inherited trait, would be to do the test with only adopted children. Then it would be ok to control for parents' income.
- ethbro 8y ago
  > The way to control for the income of a child's household while not filtering out a possible inherited trait, would be to do the test with only adopted children. It's truisms like this that make me cut psychologists and economists slack when results fail to reproduce. If experiments are prohibitively expensive or impossible to recruit large enough populations, the right answer isn't to just toss hands up, but to do the best you can with what you have. So respect to all the people working in "messy" real life fields and still trying to push the frontier of knowledge!
  - JumpCrisscross 8y ago
    I don't think it's a truism. I would have never wondered about the marshmallow test amongst adopted children had this study not been done.
  - TangoTrotFox 8y ago
    Publishing results which are then used as the basis for other work, which are not replicable, is not 'pushing' the frontier of knowledge. It's not even stagnating it. It's sending it backwards as it not only poisons derivative works, but also any relevance of the field itself. At this point it's statistically wiser to dismiss studies in certain fields than it is to assume that their results are meaningful and replicable. That's just ridiculous, but true. And you can perform twin studies for problems like this. I don't think most people understand how these work - you don't need separated twins living in different areas or anything crazy like that. The whole 'trick' is that identical twins are more genetically much more similar than fraternal or non-identical twins. If the results between identicals in something is closer than it is between non-identicals, then there is probably a strong genetic link to the thing being studied. It's just a handy way of automatically controlling for environmental variables, so much as possible. Of course twin studies can also be misleading, but that's another issue.
sbov 8y ago
If the marshmallow test is truly valid, then we should see a difference for kids in low income households that waited for the second marshmallow vs kids in low income households that didn't.
fwdpropaganda 8y ago
> Is that a valid thing to do? If "will power" was a strongly inherited trait, and the theory is that will power leads to increase household income in adult life, then this controlling would remove most of the signal. I suspect you might misunderstand what "controling for" means. Suppose you want to predict A (here "increase household income in adult life") using variable X (here "will power"). You do this by regressing A against X. Saying that you "control for Y" (here "income of a child's household") just means that you regress A against both X and Y simultaneously. What sense does that make? Suppose that you regress A against X and conclude that "X explains some of A" (these are vague terms which I can make more precise if you want) and you hope that your correlation is capturing some narrative of causality. Well, someone could come along and say "yes, A and X are correlated, but actually the causality is Y causes A, and X just happens to be correlated with Y. So you're focusing your study on X when the interesting dynamics is on Y". This is valid criticism. One way to mitigate that criticism is that you study how A depends on X while controling for Y (i.e. you include both in the regression). The rationale here is that if the result is that "both X and Y explain some of A" and someone makes that previous criticism you can respond "Yes, you're right that Y has some effect on A, but in addition X also has some effect on A, independently of Y. Therefore, X has signal regardless of whatever is going on in the dynamics between A and Y." So, to answer your question: Yes, it's perfectly valid if you mean "controlling" in the sense that it is used in statistics.
- your-nanny 8y ago
  Yes. Leaving out likely predictors is worse. Collinearity issues can mean that the beta estimates for collinear variables are less robust, making them harder to interpret. Sometimes this is approached by some data reduction technique, eg PCA.
your-nanny 8y ago
Asking parents about child's self control generally more reliable than kid self report. Anyway there is a literature on the reliability of these measures. Feel free to consult it before discounting it.
comboy 8y ago
Marshmallows represent different value for kids with different background. Some can have them every day, for some it's something they only saw other kids eating. It's similar to how it would be easy to risk $50K for somebody with $100M net worth compared to somebody whose net worth is $100K.
- stochastic_monk 8y ago
  I grew up in a household without sugary cereals. My siblings and I would sing songs and dance with excitement when we had them. Socioeconomic status certainly explains the difference between cohorts, but perhaps it’s the easier signal to catch while the core mechanism is a matter of what is available in one’s household.
  - orbitur 8y ago
    Yeah, I'm a little torn on this. The OP's claim makes sense on its face, but then I think about my own household, where we're solidly upper-middle class, and my wife and I refuse to keep sugary cereals (and most other junks like Doritos or candy, etc) in our cupboard. My daughter has definitely experienced them at birthday parties and at her grandparents'. Would she have "willpower"? I'm leaning towards "yes" because her experience has been that she can rely on people. Perhaps a poor child is simply more likely to have unreliable caretakers, so there's less trust.
  - dghughes 8y ago
    My mother grew up on a farm with eight siblings. For Christmas they would get a single apple in their stocking from Santa! My mother said everyone would be very excited. There was an an apple orchard on the farm.
  - _red 8y ago
    Why did your parents seek to create a household which avoided sugary snacks? The fact is, that itself could be representative of the genetic predisposition towards self-control.
    - 8y ago
dahart 8y ago
> Is that a valid thing to do? If "will power" was a strongly inherited trait, and the theory is that will power leads to increase household income in adult life, then this controlling would remove most of the signal. I would think so. If what seems like “will power” is strongly correllated with your parents’ income, that seems to demonstrate that will power is not a strongly heritable trait, thus the hypothesis is likely false.
- ravar 8y ago
  Just because something correlates with parental income doesn't mean that it isn't heritable. Take an example that you agree with. Being white (or Asian) correlates with parental income. Those traits are also rather heritable.
  - dahart 8y ago
    I agree with that, which is precisely why you should control for it, no? You’re example is correct, but not what I was talking about. You’re only talking about one kind of correlation, and assuming that the data is otherwise ignored. What I imagined was will power being easy to test for heritability if it changes from generation to generation. Being white or Asian might correlate with average income for a population, but Asianness & whiteness don’t change with income. On the other hand, if “will power” changes from generation to generation and the sign and magnitude of the change correllates with the sign & magnitude change of income, then the likelihood that will power is heritable drops rapidly.
    - wavegeek 8y ago
      > which is precisely why you should control for it, no? No. If willpower is inherited then what is required is a serious attempt to deal with this. If parental income is a proxy for willpower, and willpower is substantially heritable (both quite likely) then naively controlling for parental income is actually controlling away a substantial portion of the child's own willpower! The model needs to specifically allow for the heritability for willpower, which theirs does not. So in this case, the study demonstrates roughly "the marshmallow test does not test willpower, when you control for (much of the child's) willpower". As such it is useless rubbish, like a lot of social science research.
ouid 8y ago
did you read the rest? all they did was stratify on income and the effect went away.
jakobegger 8y ago
> this controlling would remove most of the signal That's the whole point of controlling. If all of your signal can be explained by a correllation, then you aren't measuring what you think you are. If "willpower" (waiting for second marshmallow) was important, it would be important regardless of whether your parents are rich or not. This means that among poor kids, those with willpower would be more successful, and among rich kids, those with willpower would be more successful. The results of the follow up study show us that this is not the case. The important factor is your parents income, not your "willpower".
- eanzenberg 8y ago
  If willpower is the strongest indicator for income then controlling for parent’s income leads to some strange circular reasoning: willpower is not a strong influence of income when controlling for income.
  - ethbro 8y ago
    * Above is true, if and only if: - Willpower is non-socially inherited - Willpower is proportionally developed (to their final amount) in children at the time of testing I would assume the hypothesis of the researchers was that children were more tabula rasa at the age tested, and therefore controlling for variables wouldn't be circular.
    - xapata 8y ago
      That's not the issue. Controlling for a variable is a model like `y ~ x + z` where x is the explanatory variable and z is the control. The commenter complained that since `y ~ z` and `x ~ z` the interpretation of the effect is strange. Not so. This is the sole purpose of controlling for confounders. If parents income were either not correlated with willpower or child's income, it could be dropped from the model. The stronger the correlation of x and z, the more important the control.
      - eanzenberg 8y ago
Barrin92 8y ago
>If "will power" was a strongly inherited trait, and the theory is that will power leads to increase household income in adult life, then this controlling would remove most of the signal. If some underlying willpower was the cause for the behaviour and willpower would lead to better outcomes, then there would be differentation between children within the same income bracket who score different on this test, right? But that is not the case: This new paper found that among kids whose mothers had a college degree, those who waited for a second marshmallow did no better in the long run—in terms of standardized test scores and mothers’ reports of their children’s behavior—than those who dug right in.
- jabgrabdthrow 8y ago
  Wait a second, when did the discussion narrow to standardized test scores? I only read the article and did not follow through with sources, it seemed to imply “success” was measured more broadly, by their end socioeconomic status.

hydrox24 8y ago

They used a lot of different measures. I'll try and summarise the key ones here: Academic achievement was measured using the Woodcock-Johnson Psycho-Educational Battery Revised, a "commonly used measure of cognitive ability and achievement". They used various sub-tests from this battery for measuring maths/english seperately. Behavioural problems were measured by reports from mothers. (They used the 1991 "Child Behavior Checklist") This was apparently done to be consistent with the original report. > [child cognitive indicators were] measured at age 24 months by the Bayley Mental Development Index (MDI; Bayley, 1991) and at age 36 months by the Bracken Basic Concept Scale (BBCS; Bracken, 1984). And the child's home environment was measured by the HOME inventory (Caldwell & Bradley, 1984).

misja111 8y ago

>> Is that a valid thing to do? If "will power" was a strongly inherited trait, and the theory is that will power leads to increase household income in adult life, then this controlling would remove most of the signal. Exactly. The way to control for the income of a child's household while not filtering out a possible inherited trait, would be to do the test with only adopted children. Then it would be ok to control for parents' income.

ethbro 8y ago
> The way to control for the income of a child's household while not filtering out a possible inherited trait, would be to do the test with only adopted children. It's truisms like this that make me cut psychologists and economists slack when results fail to reproduce. If experiments are prohibitively expensive or impossible to recruit large enough populations, the right answer isn't to just toss hands up, but to do the best you can with what you have. So respect to all the people working in "messy" real life fields and still trying to push the frontier of knowledge!
- JumpCrisscross 8y ago
  I don't think it's a truism. I would have never wondered about the marshmallow test amongst adopted children had this study not been done.
- TangoTrotFox 8y ago
  Publishing results which are then used as the basis for other work, which are not replicable, is not 'pushing' the frontier of knowledge. It's not even stagnating it. It's sending it backwards as it not only poisons derivative works, but also any relevance of the field itself. At this point it's statistically wiser to dismiss studies in certain fields than it is to assume that their results are meaningful and replicable. That's just ridiculous, but true. And you can perform twin studies for problems like this. I don't think most people understand how these work - you don't need separated twins living in different areas or anything crazy like that. The whole 'trick' is that identical twins are more genetically much more similar than fraternal or non-identical twins. If the results between identicals in something is closer than it is between non-identicals, then there is probably a strong genetic link to the thing being studied. It's just a handy way of automatically controlling for environmental variables, so much as possible. Of course twin studies can also be misleading, but that's another issue.

sbov 8y ago

If the marshmallow test is truly valid, then we should see a difference for kids in low income households that waited for the second marshmallow vs kids in low income households that didn't.

fwdpropaganda 8y ago

> Is that a valid thing to do? If "will power" was a strongly inherited trait, and the theory is that will power leads to increase household income in adult life, then this controlling would remove most of the signal. I suspect you might misunderstand what "controling for" means. Suppose you want to predict A (here "increase household income in adult life") using variable X (here "will power"). You do this by regressing A against X. Saying that you "control for Y" (here "income of a child's household") just means that you regress A against both X and Y simultaneously. What sense does that make? Suppose that you regress A against X and conclude that "X explains some of A" (these are vague terms which I can make more precise if you want) and you hope that your correlation is capturing some narrative of causality. Well, someone could come along and say "yes, A and X are correlated, but actually the causality is Y causes A, and X just happens to be correlated with Y. So you're focusing your study on X when the interesting dynamics is on Y". This is valid criticism. One way to mitigate that criticism is that you study how A depends on X while controling for Y (i.e. you include both in the regression). The rationale here is that if the result is that "both X and Y explain some of A" and someone makes that previous criticism you can respond "Yes, you're right that Y has some effect on A, but in addition X also has some effect on A, independently of Y. Therefore, X has signal regardless of whatever is going on in the dynamics between A and Y." So, to answer your question: Yes, it's perfectly valid if you mean "controlling" in the sense that it is used in statistics.

your-nanny 8y ago
Yes. Leaving out likely predictors is worse. Collinearity issues can mean that the beta estimates for collinear variables are less robust, making them harder to interpret. Sometimes this is approached by some data reduction technique, eg PCA.

your-nanny 8y ago

Asking parents about child's self control generally more reliable than kid self report. Anyway there is a literature on the reliability of these measures. Feel free to consult it before discounting it.

comboy 8y ago

Marshmallows represent different value for kids with different background. Some can have them every day, for some it's something they only saw other kids eating. It's similar to how it would be easy to risk $50K for somebody with $100M net worth compared to somebody whose net worth is $100K.

stochastic_monk 8y ago
I grew up in a household without sugary cereals. My siblings and I would sing songs and dance with excitement when we had them. Socioeconomic status certainly explains the difference between cohorts, but perhaps it’s the easier signal to catch while the core mechanism is a matter of what is available in one’s household.
- orbitur 8y ago
  Yeah, I'm a little torn on this. The OP's claim makes sense on its face, but then I think about my own household, where we're solidly upper-middle class, and my wife and I refuse to keep sugary cereals (and most other junks like Doritos or candy, etc) in our cupboard. My daughter has definitely experienced them at birthday parties and at her grandparents'. Would she have "willpower"? I'm leaning towards "yes" because her experience has been that she can rely on people. Perhaps a poor child is simply more likely to have unreliable caretakers, so there's less trust.
- dghughes 8y ago
  My mother grew up on a farm with eight siblings. For Christmas they would get a single apple in their stocking from Santa! My mother said everyone would be very excited. There was an an apple orchard on the farm.
- _red 8y ago
  Why did your parents seek to create a household which avoided sugary snacks? The fact is, that itself could be representative of the genetic predisposition towards self-control.
  - 8y ago

dahart 8y ago

> Is that a valid thing to do? If "will power" was a strongly inherited trait, and the theory is that will power leads to increase household income in adult life, then this controlling would remove most of the signal. I would think so. If what seems like “will power” is strongly correllated with your parents’ income, that seems to demonstrate that will power is not a strongly heritable trait, thus the hypothesis is likely false.

ravar 8y ago
Just because something correlates with parental income doesn't mean that it isn't heritable. Take an example that you agree with. Being white (or Asian) correlates with parental income. Those traits are also rather heritable.
- dahart 8y ago
  I agree with that, which is precisely why you should control for it, no? You’re example is correct, but not what I was talking about. You’re only talking about one kind of correlation, and assuming that the data is otherwise ignored. What I imagined was will power being easy to test for heritability if it changes from generation to generation. Being white or Asian might correlate with average income for a population, but Asianness & whiteness don’t change with income. On the other hand, if “will power” changes from generation to generation and the sign and magnitude of the change correllates with the sign & magnitude change of income, then the likelihood that will power is heritable drops rapidly.
  - wavegeek 8y ago
    > which is precisely why you should control for it, no? No. If willpower is inherited then what is required is a serious attempt to deal with this. If parental income is a proxy for willpower, and willpower is substantially heritable (both quite likely) then naively controlling for parental income is actually controlling away a substantial portion of the child's own willpower! The model needs to specifically allow for the heritability for willpower, which theirs does not. So in this case, the study demonstrates roughly "the marshmallow test does not test willpower, when you control for (much of the child's) willpower". As such it is useless rubbish, like a lot of social science research.

ouid 8y ago

did you read the rest? all they did was stratify on income and the effect went away.

jakobegger 8y ago

> this controlling would remove most of the signal That's the whole point of controlling. If all of your signal can be explained by a correllation, then you aren't measuring what you think you are. If "willpower" (waiting for second marshmallow) was important, it would be important regardless of whether your parents are rich or not. This means that among poor kids, those with willpower would be more successful, and among rich kids, those with willpower would be more successful. The results of the follow up study show us that this is not the case. The important factor is your parents income, not your "willpower".

eanzenberg 8y ago
If willpower is the strongest indicator for income then controlling for parent’s income leads to some strange circular reasoning: willpower is not a strong influence of income when controlling for income.
- ethbro 8y ago
  * Above is true, if and only if: - Willpower is non-socially inherited - Willpower is proportionally developed (to their final amount) in children at the time of testing I would assume the hypothesis of the researchers was that children were more tabula rasa at the age tested, and therefore controlling for variables wouldn't be circular.
  - xapata 8y ago
    That's not the issue. Controlling for a variable is a model like `y ~ x + z` where x is the explanatory variable and z is the control. The commenter complained that since `y ~ z` and `x ~ z` the interpretation of the effect is strange. Not so. This is the sole purpose of controlling for confounders. If parents income were either not correlated with willpower or child's income, it could be dropped from the model. The stronger the correlation of x and z, the more important the control.
    - eanzenberg 8y ago

Barrin92 8y ago

>If "will power" was a strongly inherited trait, and the theory is that will power leads to increase household income in adult life, then this controlling would remove most of the signal. If some underlying willpower was the cause for the behaviour and willpower would lead to better outcomes, then there would be differentation between children within the same income bracket who score different on this test, right? But that is not the case: This new paper found that among kids whose mothers had a college degree, those who waited for a second marshmallow did no better in the long run—in terms of standardized test scores and mothers’ reports of their children’s behavior—than those who dug right in.

jabgrabdthrow 8y ago
Wait a second, when did the discussion narrow to standardized test scores? I only read the article and did not follow through with sources, it seemed to imply “success” was measured more broadly, by their end socioeconomic status.