4 ms·

> correlated with better academic achievement later Correlation usually doesn't indicate causation.

msla 6y ago
> Correlation usually doesn't indicate causation. I think this is taking "correlation doesn't always prove causation" a few steps too far, right off a cliff. There are difficulties with reasoning from correlation, but saying it "usually doesn't" indicate correlation is too bold, especially in experiments where confounders have been controlled for. [1] [1] (And don't get me started with the semi-clever types who ask if this or that variable has been controlled for when the study or the article about the study explicitly says it has. Save us all from halfwits trying to teach experienced researchers their business.)
- asdfasgasdgasdg 6y ago
  https://www.gwern.net/Causality https://www.gwern.net/Causality provides more empirical background on my view.
  - mlyle 6y ago
    OK, so we'll go with "pedantry". We have both very strong evidence of correlation here, and occasional measures showing causal relationships in randomized studies -- Head Start, children randomized to different amounts of Kindergarten by individual city programs, etc. And sure, we have very poor data on what happens to affluent kids with involved parents, because this is a difficult population to study (haha, randomize my kids out to a half-day of K? Nothanks.) But, hey, you thought you could dunk on some bro on the internet, so.. high five!
    - asdfasgasdgasdg 6y ago
      I agree that my topmost comment was a little loose, but that wasn't targeted at any individual person. Having the relationship between correlation and observed to you is not a dunk (I hope). The preponderance of evidence I've been exposed to does not show a strong positive clinically significant effect of mainstream kindergarten, the one Head Start study you referenced notwithstanding. For example, here is one study that finds in the opposite direction. https://journals.sagepub.com/doi/abs/10.3102/0034654309359185?journalCode=rera https://journals.sagepub.com/doi/abs/10.3102/003465430935918... This is the sum of what I was trying to get across, in so many words. Sorry if the way I said it made you feel bad. I don't really want to cite studies at each other all day. I honestly don't have time for it and it's a little too amenable to cherry-picking. If there is a well-regarded summary of the literature that you're aware of, I'm happy to read it. But this really isn't that big of a deal to me, so please don't feel obligated to respond.
      - mlyle 6y ago
        The cherry that you picked is a good example. You point to a study showing a small (but positive) marginal benefit of full-day kindergarten over half-day kindergarten to refute my original assertion that kindergarten in general is good? mkay. If you have academic library access, here's a Harvard meta-analysis of ECE (e.g. preschool/pre-K). 7 of the input studies were randomized to ECE vs. non-ECE. https://journals.sagepub.com/doi/pdf/10.3102/0013189X17737739 https://journals.sagepub.com/doi/pdf/10.3102/0013189X1773773...
        asdfasgasdgasdg 6y ago
        R.e. me cherry-picking my study: "But the association disappeared by third grade." Second sentence in the abstract? To me, an effect which disappears by third grade is not a small positive effect. It is no effect at all. R.e. your study: this table (https://imgur.com/a/VOqDjNo https://imgur.com/a/VOqDjNo) claims that ECE (i.e. pre-k and 3k) cause a twelve percentage point (20%) increase in the likelihood of graduating from high school. That does not pass the sniff test for me. The effect size is far too large -- it would be plain to everyone that ECE was a must-have if the effect were this big. No parent would opt not to send their kid to pre-K. The study authors acknowledge serious limitations to their methodology: > Limitations: Research is needed to address several important limitations of the work presented. First and most importantly, circumstances surrounding today’s ECE programs differ from those associated with many of the programs included in this analysis. Many programs in this analysis were implemented at a time when alternative care options were limited, mostly targeted particularly high-risk children, often included comprehensive “wrap-around” services and home visiting components, and frequently provided services for multiple years at a time. IMO, you cannot cite this study in opposition to my position ("kindergarten doesn't make a significant difference for most kids") in good conscience. It is a bad study, at least as it pertains to ECE for any non-at-risk kids.
        mlyle 6y ago
        > R.e. me cherry-picking my study: "But the association disappeared by third grade." Second sentence in the abstract? To me, an effect which disappears by third grade is not a small positive effect. It is no effect at all. Yes, so there was a positive effect that was no longer measurable by third grade, when we're looking at the dose response of kindergarten. (And there's other studies pointing the other way re: dose-response). And, we're talking about kindergarten in general, not the marginal benefits of higher doses of kindergarten. That is-- if 200 mg ibuprofen is the standard of care, and we find people taking 400mg of ibuprofen have better outcomes than 200mg, then we think that probably ibuprofen is good (and 200mg would probably be better than 0). If we find taking 4000mg of ibuprofen is worse than taking 200mg, we can't really form any conclusions about the overall merits of 200mg vs. 0. > That does not pass the sniff test for me OK, so you don't like it. That's definitive. > IMO, you cannot cite this study It's not a study, it's a meta-analysis. > in opposition to my position ("kindergarten doesn't make a significant difference for most kids") Which is at least a clear position rather than the default position of snark. > It is a bad study It summarizes all the high quality research we have on the subject, and mentions/criticizes the earlier meta-analyses (which included many more component studies and found similar magnitudes of effect). > at least as it pertains to ECE for any non-at-risk kids. Of course, some of the component studies were of lower risk populations.
    - dang 6y ago
      You broke the site guidelines badly in this thread. Please do not take HN further into flamewars, and please don't get involved in nasty tit-for-tat spats. Those are particularly tedious. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- astrange 6y ago
  > [1] (And don't get me started with the semi-clever types who ask if this or that variable has been controlled for when the study or the article about the study explicitly says it has. Save us all from halfwits trying to teach experienced researchers their business.) This is specifically called "shallow dismissal" in the HN rules (used to be "middle-brow dismissal"). Basically this is when you don't read the article and reply with something that sounds smart and applies in the general situation, but not here because the authors knew that too.
mlyle 6y ago
No. But we do have some evidence of causation-- e.g. children offered Head Start programs do better than children that were not. E.g. Ludwig and Miller (2007) showed a 3% increase in high school graduation rates and a 4% increase in college attendance in disadvantaged youth offered early education. That's a big difference for a treatment 15 years later.
- asdfasgasdgasdg 6y ago
  > . . . in disadvantaged youth offered early education. Head Start was a lot more than just early education. It was social services, nutrition, mental health services, and even the employment of some parents of the Head Start children. I should be clear that I do not mean to claim the total absence of benefits of kindergarten. I was exaggerating. The position I will stake is that I believe the benefits are likely very minor, even outside of COVID, and especially for affluent children.
  - mlyle 6y ago
    So, then, your intent in responding to my comment was just to be pedantic and snarky, and not to offer any actual information except to parrot the normal correlation != causation line? That was a good use of our time.
    - asdfasgasdgasdg 6y ago
      Your one observing a correlation between kindergarten and success? No, I think your comment was likely to engender in the typical reader a belief that kindergarten was known to have a significant and positive causal effect (both statistical and clinical) on outcomes as a general rule. I don't believe this has been established. My intent in responding to your comment was to prevent that belief from wrongly being engendered in the minds of the readers.
- spoonjim 6y ago
  Is there a possible confounder that the kids whose parents find out about these offerings and take their kids to them are more attentive overall?

msla 6y ago

> Correlation usually doesn't indicate causation. I think this is taking "correlation doesn't always prove causation" a few steps too far, right off a cliff. There are difficulties with reasoning from correlation, but saying it "usually doesn't" indicate correlation is too bold, especially in experiments where confounders have been controlled for. [1] [1] (And don't get me started with the semi-clever types who ask if this or that variable has been controlled for when the study or the article about the study explicitly says it has. Save us all from halfwits trying to teach experienced researchers their business.)

asdfasgasdgasdg 6y ago
https://www.gwern.net/Causality https://www.gwern.net/Causality provides more empirical background on my view.
- mlyle 6y ago
  OK, so we'll go with "pedantry". We have both very strong evidence of correlation here, and occasional measures showing causal relationships in randomized studies -- Head Start, children randomized to different amounts of Kindergarten by individual city programs, etc. And sure, we have very poor data on what happens to affluent kids with involved parents, because this is a difficult population to study (haha, randomize my kids out to a half-day of K? Nothanks.) But, hey, you thought you could dunk on some bro on the internet, so.. high five!
  - asdfasgasdgasdg 6y ago
    I agree that my topmost comment was a little loose, but that wasn't targeted at any individual person. Having the relationship between correlation and observed to you is not a dunk (I hope). The preponderance of evidence I've been exposed to does not show a strong positive clinically significant effect of mainstream kindergarten, the one Head Start study you referenced notwithstanding. For example, here is one study that finds in the opposite direction. https://journals.sagepub.com/doi/abs/10.3102/0034654309359185?journalCode=rera https://journals.sagepub.com/doi/abs/10.3102/003465430935918... This is the sum of what I was trying to get across, in so many words. Sorry if the way I said it made you feel bad. I don't really want to cite studies at each other all day. I honestly don't have time for it and it's a little too amenable to cherry-picking. If there is a well-regarded summary of the literature that you're aware of, I'm happy to read it. But this really isn't that big of a deal to me, so please don't feel obligated to respond.
    - mlyle 6y ago
      The cherry that you picked is a good example. You point to a study showing a small (but positive) marginal benefit of full-day kindergarten over half-day kindergarten to refute my original assertion that kindergarten in general is good? mkay. If you have academic library access, here's a Harvard meta-analysis of ECE (e.g. preschool/pre-K). 7 of the input studies were randomized to ECE vs. non-ECE. https://journals.sagepub.com/doi/pdf/10.3102/0013189X17737739 https://journals.sagepub.com/doi/pdf/10.3102/0013189X1773773...
      - asdfasgasdgasdg 6y ago
        R.e. me cherry-picking my study: "But the association disappeared by third grade." Second sentence in the abstract? To me, an effect which disappears by third grade is not a small positive effect. It is no effect at all. R.e. your study: this table (https://imgur.com/a/VOqDjNo https://imgur.com/a/VOqDjNo) claims that ECE (i.e. pre-k and 3k) cause a twelve percentage point (20%) increase in the likelihood of graduating from high school. That does not pass the sniff test for me. The effect size is far too large -- it would be plain to everyone that ECE was a must-have if the effect were this big. No parent would opt not to send their kid to pre-K. The study authors acknowledge serious limitations to their methodology: > Limitations: Research is needed to address several important limitations of the work presented. First and most importantly, circumstances surrounding today’s ECE programs differ from those associated with many of the programs included in this analysis. Many programs in this analysis were implemented at a time when alternative care options were limited, mostly targeted particularly high-risk children, often included comprehensive “wrap-around” services and home visiting components, and frequently provided services for multiple years at a time. IMO, you cannot cite this study in opposition to my position ("kindergarten doesn't make a significant difference for most kids") in good conscience. It is a bad study, at least as it pertains to ECE for any non-at-risk kids.
        mlyle 6y ago
        > R.e. me cherry-picking my study: "But the association disappeared by third grade." Second sentence in the abstract? To me, an effect which disappears by third grade is not a small positive effect. It is no effect at all. Yes, so there was a positive effect that was no longer measurable by third grade, when we're looking at the dose response of kindergarten. (And there's other studies pointing the other way re: dose-response). And, we're talking about kindergarten in general, not the marginal benefits of higher doses of kindergarten. That is-- if 200 mg ibuprofen is the standard of care, and we find people taking 400mg of ibuprofen have better outcomes than 200mg, then we think that probably ibuprofen is good (and 200mg would probably be better than 0). If we find taking 4000mg of ibuprofen is worse than taking 200mg, we can't really form any conclusions about the overall merits of 200mg vs. 0. > That does not pass the sniff test for me OK, so you don't like it. That's definitive. > IMO, you cannot cite this study It's not a study, it's a meta-analysis. > in opposition to my position ("kindergarten doesn't make a significant difference for most kids") Which is at least a clear position rather than the default position of snark. > It is a bad study It summarizes all the high quality research we have on the subject, and mentions/criticizes the earlier meta-analyses (which included many more component studies and found similar magnitudes of effect). > at least as it pertains to ECE for any non-at-risk kids. Of course, some of the component studies were of lower risk populations.
  - dang 6y ago
    You broke the site guidelines badly in this thread. Please do not take HN further into flamewars, and please don't get involved in nasty tit-for-tat spats. Those are particularly tedious. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
astrange 6y ago
> [1] (And don't get me started with the semi-clever types who ask if this or that variable has been controlled for when the study or the article about the study explicitly says it has. Save us all from halfwits trying to teach experienced researchers their business.) This is specifically called "shallow dismissal" in the HN rules (used to be "middle-brow dismissal"). Basically this is when you don't read the article and reply with something that sounds smart and applies in the general situation, but not here because the authors knew that too.

mlyle 6y ago

No. But we do have some evidence of causation-- e.g. children offered Head Start programs do better than children that were not. E.g. Ludwig and Miller (2007) showed a 3% increase in high school graduation rates and a 4% increase in college attendance in disadvantaged youth offered early education. That's a big difference for a treatment 15 years later.

asdfasgasdgasdg 6y ago
> . . . in disadvantaged youth offered early education. Head Start was a lot more than just early education. It was social services, nutrition, mental health services, and even the employment of some parents of the Head Start children. I should be clear that I do not mean to claim the total absence of benefits of kindergarten. I was exaggerating. The position I will stake is that I believe the benefits are likely very minor, even outside of COVID, and especially for affluent children.
- mlyle 6y ago
  So, then, your intent in responding to my comment was just to be pedantic and snarky, and not to offer any actual information except to parrot the normal correlation != causation line? That was a good use of our time.
  - asdfasgasdgasdg 6y ago
    Your one observing a correlation between kindergarten and success? No, I think your comment was likely to engender in the typical reader a belief that kindergarten was known to have a significant and positive causal effect (both statistical and clinical) on outcomes as a general rule. I don't believe this has been established. My intent in responding to your comment was to prevent that belief from wrongly being engendered in the minds of the readers.
spoonjim 6y ago
Is there a possible confounder that the kids whose parents find out about these offerings and take their kids to them are more attentive overall?