5 ms·

I'm tempted to repeat the oft said correlation is not causation. But really I have never thought that captures the problem I have with studies like this. Of cou

I'm tempted to repeat the oft said correlation is not causation. But really I have never thought that captures the problem I have with studies like this. Of course correlation does not necessarily imply causation. What I really find to be problematic is a small time series of data of a facet of a human behaviour across a population is probably going to correlate extremely well to many other time series across the same population. Or even other populations. How unusual should we find the correlation? What we might be seeing is nothing more than the average rate of change in a population to legislative changes. If it even shows that. If you take two time series that rise and then fall and overlay them on their peak I suspect most graphs look as convincing as the one presented.

gruseom 14y ago
This article cites research that goes way beyond the usual sort of correlation. How much stronger would the evidence need to get?
- greendestiny 14y ago
  I'm not sure, how strong should such a correlation be to be considered evidence? That's the real question. And I think when it comes to annual averages of broad trends its a real question how we should interpret correlations. To me this article is data porn, its a story with data. It may be some part of the truth, but I think it exists to entertain the data literate - so we can follow along with the numbers and pat ourselves on the back for a fact discovered. Its enough analysis to be dangerous. And in case hackernews has become so infected with the kind of reddit thinking that can only see in black and white - yes I think reducing lead in gasoline is a very good thing.
  - jforman 14y ago
    The strength of the correlation or its time-scale have no bearing on whether the correlation should be considered evidence of causation. The fatal flaw of a correlation is that it can be specious — it can appear to explain reality, but really there's a third variable responsible for driving the phenomenon you're interested in. In this case, lead exposure and crime rate are correlated, but maybe lead doesn't cause crime at all: maybe something else causes crime that also happens to correlate with lead exposure. Who knows, maybe a certain pesticide was used at the same time that lead gasoline came into vogue, and that's really the true cause of the rise in crime rates. In research like this, when you can't do a manually controlled experiment, you have to control for hidden variables by some other means. And that's precisely what the investigators in this article did: they varied the input data to try to "shake out" other variables that might be behind the correlation. They looked at different time scales, different geographies, and different demographics, in an attempt to control for hidden variables that might be related to any one of those things. Every time you vary the input data and keep finding a correlation, your evidence of a causative relationship goes up. The gold standard, of course, would be to expose two random, double-blind sets of infants to lead and to a control substance and see what happens. But since that would be unethical (as we have reason to think lead is bad for you), we're stuck with animal studies or retrospective studies. Personally, I find the evidence in this article impressive, but it would take quite a lot of looking into their specific methods to come to any real conclusion.
    - greendestiny 14y ago
      That's the high level view of the problem of correlation and causation. What I'm really saying is how unlikely should we consider these correlations - especially correlations over very broad average trends in human behaviour. Now I'm sure there are some statistical tools that would give some sense of how surprising the correlation of these trends is but I'd like to know out of all the things about human society we track how many of them correlate and how closely.
      - nitrogen 14y ago
        Presumably one can correlate lead exposure with reduced intelligence independent of time.
    - SideburnsOfDoom 14y ago
      > The fatal flaw of a correlation is that it can be specious — it can appear to explain reality, but really there's a third variable responsible for driving the phenomenon you're interested in. But in the case of lead, the causal chain from lead exposure to worse behaviour is fairly well understood.
  - sliverstorm 14y ago
    Would you at least agree the correlation seems strong enough to warrant further investigation?
    - greendestiny 14y ago
      Sure, it might even be strong evidence. It's just not clear and we see these time series correlations again and again.
- smsm42 14y ago
  Well, for starters to prove casuation one doesn't only need to show X moves in sync with Y. One also needs to show that it is not because both X and Y are caused by some reason Z which influences both even though X and Y are completely independent from each other. E.g. correlation between good lead data and good crime data may be caused by the fact that in cities with better law enforcement both criminal laws and environmental laws are enforced strictly, or because richer folks drive less polluting cars and commit less crimes. It does not mean less lead pollution causes people to be rich. The whole thing sound like classic fitting the preconceived conclusion. But there's also something like this: >>>> Not only does lead promote apoptosis, or cell death, in the brain, but the element is also chemically similar to calcium. How lead is chemically similar to calcium? They are on opposite ends of the periodic table and have very different chemical properties. That, of course, does not prevent lead from messing with ion channels, but saying lead is chemically similar to calcium is wrong.
  - pixl97 14y ago
    How lead is chemically similar to calcium? They are on opposite... Then you go on to explain how lead is like calcium in its bioavailability, so in relation to the operations of a human body, lead is chemically similar to calcium (only dangerous). http://brain.oxfordjournals.org/content/126/1/5.full http://brain.oxfordjournals.org/content/126/1/5.full For example, lead’s ability to pass through the blood–brain barrier (BBB) is due in large part to its ability to substitute for calcium ions (Ca2+)
    - smsm42 14y ago
      "Ability to replace calcium in certain biochemical processes" is not nearly the same as "chemically similar to calcium". It's like saying if I stole your login for facebook I would look like you and nobody would be able to distinguish between us without DNA test. Completely different things. Biochemical processes are known for accepting a lot of substitutes, that's how a lot of drugs work. That does not mean all those are chemically similar in general - they just play similar roles in certain processes.
  - gruseom 14y ago
SideburnsOfDoom 14y ago
> I'm tempted to repeat the oft said correlation is not causation I's be interested in seeing what you think on that score after reading TFA and possibly even the studies that it talks about.
- greendestiny 14y ago
  I haven't read any of the studies it talks about. I think the evidence for lead causing brain damage is very strong. Its effect on the crime rate is a different story. Admittedly it says it holds up over multiple countries, which seems like it should be stronger evidence. Honestly I don't know, but I do think there is a real problem in our understanding of how to interpret population level trends. The rigorous camp simply wants to throw out everything that isn't a double blind study. The optimistic are convinced of every similar looking graph. I think we should be doing more meta-analysis of these things and building up a Bayesian sort of set of priors to apply to these results.
  - sopooneo 14y ago
    I have two out of the air guesses on how brain damage might cause crime. First, part of your brain responsible for empathy might not work right. Second, if you're dumb you have a harder time holding down a job, so might be more likely to resort to crime. I throw these out only to play with ideas, not to imply I have any evidence they are correct, or that brain-damage even does cause crime increase.
    - walls 14y ago
      From the article: > A second study [23] found that high exposure to lead during childhood was linked to a permanent loss of gray matter in the prefrontal cortex—a part of the brain associated with aggression control as well as what psychologists call "executive functions": emotional regulation, impulse control, attention, verbal reasoning, and mental flexibility.
defactoserfdom 14y ago
I've always thought that we should be wary of correlation between two series that are positively correlated with time.
colomon 14y ago
Yeah, it just feels like the entire article is too convenient. 1) Everything seems to work out perfectly, no matter how it is considered. For instance, if lead contamination in the ground is such a big deal, why does lead in gas track so closely to the crime rate? Wouldn't it be the case that it would track on the upside but not the downside? 2) He keeps on emphasizing lead lowers IQ. But the Flynn Effect suggests that IQ started going up in the 1930s and has only (maybe) started to level off very recently. It's the exact opposite of what you'd expect based on lead exposure from leaded gas. 3) Long term crime rates have been mostly going down for centuries: http://marginalrevolution.com/marginalrevolution/2011/06/long-term-trend-in-homicide-rates.html http://marginalrevolution.com/marginalrevolution/2011/06/lon... Mind you, I'm fully in agreement that lead sucks and I sure as hell don't want my kid exposed to it. It's just that I'm far from convinced that the crime rates change mostly because of lead exposure. It seems far too tidy an explanation.
- jkimmel 14y ago
  As far as the Flynn effect is concerned, it's possible that the benefits of general increases in public health (to which the Flynn effect is oft attributed, though somewhat more controversially in recent years) negated the effects of lead on a large scale. However, this does not mean that the Flynn effect would not have had a greater observable magnitude had lead exposure not been so high. Part of me agrees with your overall conclusion about the corollary nature of the research as a whole, but those are my two cents on that particular point.
  - colomon 14y ago
    Sure, it's possible (even probable!) there are any number of things affecting IQ. I'm just pointing out that if you'd plotted raw IQ levels versus time, the graph would have the opposite sign from what you'd expect from Drum's lead hypothesis. There may be an explanation that makes it all fit together, but it somehow needs to be addressed, not just ignored or hand-waved away.
- tlb 14y ago
  You're assuming that the causation goes lead -> lower IQ -> more crime. Indeed, the Flynn effect doesn't support the lower IQ -> more crime relationship. Lead has a wider range of neurological effects than just reducing IQ. The causation may be more likely: lead -> unknown neurological changes -> more criminality & lower IQ.
crusso 14y ago
So many factors that could have contributed to the crime drop. Hate to drive by this interesting thread, but wasn't this same crime drop attributed to the legalization of abortion? http://en.wikipedia.org/wiki/The_Impact_of_Legalized_Abortion_on_Crime http://en.wikipedia.org/wiki/The_Impact_of_Legalized_Abortio...
- gruseom 14y ago
  (Duplicated from a comment I made to the original posting of this article at https://news.ycombinator.com/item?id=5002806... https://news.ycombinator.com/item?id=5002806...) Via http://election.princeton.edu/2012/12/22/scientific-american http://election.princeton.edu/2012/12/22/scientific-american, which cited Reyes' work on the lead theory, here's a link to a striking critique of the abortion theory: http://www.economist.com/node/5246700?story_id=5246700 http://www.economist.com/node/5246700?story_id=5246700 Someone inspected Donohue and Levitt's code and found a bug that meant they hadn't controlled for what they claimed they had. "Fixing that error reduces the effect of abortion on arrests by about half, using the original data, and two-thirds using updated numbers."

gruseom 14y ago

This article cites research that goes way beyond the usual sort of correlation. How much stronger would the evidence need to get?

greendestiny 14y ago
I'm not sure, how strong should such a correlation be to be considered evidence? That's the real question. And I think when it comes to annual averages of broad trends its a real question how we should interpret correlations. To me this article is data porn, its a story with data. It may be some part of the truth, but I think it exists to entertain the data literate - so we can follow along with the numbers and pat ourselves on the back for a fact discovered. Its enough analysis to be dangerous. And in case hackernews has become so infected with the kind of reddit thinking that can only see in black and white - yes I think reducing lead in gasoline is a very good thing.
- jforman 14y ago
  The strength of the correlation or its time-scale have no bearing on whether the correlation should be considered evidence of causation. The fatal flaw of a correlation is that it can be specious — it can appear to explain reality, but really there's a third variable responsible for driving the phenomenon you're interested in. In this case, lead exposure and crime rate are correlated, but maybe lead doesn't cause crime at all: maybe something else causes crime that also happens to correlate with lead exposure. Who knows, maybe a certain pesticide was used at the same time that lead gasoline came into vogue, and that's really the true cause of the rise in crime rates. In research like this, when you can't do a manually controlled experiment, you have to control for hidden variables by some other means. And that's precisely what the investigators in this article did: they varied the input data to try to "shake out" other variables that might be behind the correlation. They looked at different time scales, different geographies, and different demographics, in an attempt to control for hidden variables that might be related to any one of those things. Every time you vary the input data and keep finding a correlation, your evidence of a causative relationship goes up. The gold standard, of course, would be to expose two random, double-blind sets of infants to lead and to a control substance and see what happens. But since that would be unethical (as we have reason to think lead is bad for you), we're stuck with animal studies or retrospective studies. Personally, I find the evidence in this article impressive, but it would take quite a lot of looking into their specific methods to come to any real conclusion.
  - greendestiny 14y ago
    That's the high level view of the problem of correlation and causation. What I'm really saying is how unlikely should we consider these correlations - especially correlations over very broad average trends in human behaviour. Now I'm sure there are some statistical tools that would give some sense of how surprising the correlation of these trends is but I'd like to know out of all the things about human society we track how many of them correlate and how closely.
    - nitrogen 14y ago
      Presumably one can correlate lead exposure with reduced intelligence independent of time.
  - SideburnsOfDoom 14y ago
    > The fatal flaw of a correlation is that it can be specious — it can appear to explain reality, but really there's a third variable responsible for driving the phenomenon you're interested in. But in the case of lead, the causal chain from lead exposure to worse behaviour is fairly well understood.
- sliverstorm 14y ago
  Would you at least agree the correlation seems strong enough to warrant further investigation?
  - greendestiny 14y ago
    Sure, it might even be strong evidence. It's just not clear and we see these time series correlations again and again.
smsm42 14y ago
Well, for starters to prove casuation one doesn't only need to show X moves in sync with Y. One also needs to show that it is not because both X and Y are caused by some reason Z which influences both even though X and Y are completely independent from each other. E.g. correlation between good lead data and good crime data may be caused by the fact that in cities with better law enforcement both criminal laws and environmental laws are enforced strictly, or because richer folks drive less polluting cars and commit less crimes. It does not mean less lead pollution causes people to be rich. The whole thing sound like classic fitting the preconceived conclusion. But there's also something like this: >>>> Not only does lead promote apoptosis, or cell death, in the brain, but the element is also chemically similar to calcium. How lead is chemically similar to calcium? They are on opposite ends of the periodic table and have very different chemical properties. That, of course, does not prevent lead from messing with ion channels, but saying lead is chemically similar to calcium is wrong.
- pixl97 14y ago
  How lead is chemically similar to calcium? They are on opposite... Then you go on to explain how lead is like calcium in its bioavailability, so in relation to the operations of a human body, lead is chemically similar to calcium (only dangerous). http://brain.oxfordjournals.org/content/126/1/5.full http://brain.oxfordjournals.org/content/126/1/5.full For example, lead’s ability to pass through the blood–brain barrier (BBB) is due in large part to its ability to substitute for calcium ions (Ca2+)
  - smsm42 14y ago
    "Ability to replace calcium in certain biochemical processes" is not nearly the same as "chemically similar to calcium". It's like saying if I stole your login for facebook I would look like you and nobody would be able to distinguish between us without DNA test. Completely different things. Biochemical processes are known for accepting a lot of substitutes, that's how a lot of drugs work. That does not mean all those are chemically similar in general - they just play similar roles in certain processes.
- gruseom 14y ago

SideburnsOfDoom 14y ago

> I'm tempted to repeat the oft said correlation is not causation I's be interested in seeing what you think on that score after reading TFA and possibly even the studies that it talks about.

greendestiny 14y ago
I haven't read any of the studies it talks about. I think the evidence for lead causing brain damage is very strong. Its effect on the crime rate is a different story. Admittedly it says it holds up over multiple countries, which seems like it should be stronger evidence. Honestly I don't know, but I do think there is a real problem in our understanding of how to interpret population level trends. The rigorous camp simply wants to throw out everything that isn't a double blind study. The optimistic are convinced of every similar looking graph. I think we should be doing more meta-analysis of these things and building up a Bayesian sort of set of priors to apply to these results.
- sopooneo 14y ago
  I have two out of the air guesses on how brain damage might cause crime. First, part of your brain responsible for empathy might not work right. Second, if you're dumb you have a harder time holding down a job, so might be more likely to resort to crime. I throw these out only to play with ideas, not to imply I have any evidence they are correct, or that brain-damage even does cause crime increase.
  - walls 14y ago
    From the article: > A second study [23] found that high exposure to lead during childhood was linked to a permanent loss of gray matter in the prefrontal cortex—a part of the brain associated with aggression control as well as what psychologists call "executive functions": emotional regulation, impulse control, attention, verbal reasoning, and mental flexibility.

defactoserfdom 14y ago

I've always thought that we should be wary of correlation between two series that are positively correlated with time.

colomon 14y ago

Yeah, it just feels like the entire article is too convenient. 1) Everything seems to work out perfectly, no matter how it is considered. For instance, if lead contamination in the ground is such a big deal, why does lead in gas track so closely to the crime rate? Wouldn't it be the case that it would track on the upside but not the downside? 2) He keeps on emphasizing lead lowers IQ. But the Flynn Effect suggests that IQ started going up in the 1930s and has only (maybe) started to level off very recently. It's the exact opposite of what you'd expect based on lead exposure from leaded gas. 3) Long term crime rates have been mostly going down for centuries: http://marginalrevolution.com/marginalrevolution/2011/06/long-term-trend-in-homicide-rates.html http://marginalrevolution.com/marginalrevolution/2011/06/lon... Mind you, I'm fully in agreement that lead sucks and I sure as hell don't want my kid exposed to it. It's just that I'm far from convinced that the crime rates change mostly because of lead exposure. It seems far too tidy an explanation.

jkimmel 14y ago
As far as the Flynn effect is concerned, it's possible that the benefits of general increases in public health (to which the Flynn effect is oft attributed, though somewhat more controversially in recent years) negated the effects of lead on a large scale. However, this does not mean that the Flynn effect would not have had a greater observable magnitude had lead exposure not been so high. Part of me agrees with your overall conclusion about the corollary nature of the research as a whole, but those are my two cents on that particular point.
- colomon 14y ago
  Sure, it's possible (even probable!) there are any number of things affecting IQ. I'm just pointing out that if you'd plotted raw IQ levels versus time, the graph would have the opposite sign from what you'd expect from Drum's lead hypothesis. There may be an explanation that makes it all fit together, but it somehow needs to be addressed, not just ignored or hand-waved away.
tlb 14y ago
You're assuming that the causation goes lead -> lower IQ -> more crime. Indeed, the Flynn effect doesn't support the lower IQ -> more crime relationship. Lead has a wider range of neurological effects than just reducing IQ. The causation may be more likely: lead -> unknown neurological changes -> more criminality & lower IQ.

crusso 14y ago

So many factors that could have contributed to the crime drop. Hate to drive by this interesting thread, but wasn't this same crime drop attributed to the legalization of abortion? http://en.wikipedia.org/wiki/The_Impact_of_Legalized_Abortion_on_Crime http://en.wikipedia.org/wiki/The_Impact_of_Legalized_Abortio...

gruseom 14y ago
(Duplicated from a comment I made to the original posting of this article at https://news.ycombinator.com/item?id=5002806... https://news.ycombinator.com/item?id=5002806...) Via http://election.princeton.edu/2012/12/22/scientific-american http://election.princeton.edu/2012/12/22/scientific-american, which cited Reyes' work on the lead theory, here's a link to a striking critique of the abortion theory: http://www.economist.com/node/5246700?story_id=5246700 http://www.economist.com/node/5246700?story_id=5246700 Someone inspected Donohue and Levitt's code and found a bug that meant they hadn't controlled for what they claimed they had. "Fixing that error reduces the effect of abortion on arrests by about half, using the original data, and two-thirds using updated numbers."