7 ms·

Understanding Causality Is the Next Challenge for Machine Learning

125 points by headalgorithm 6y ago

arusahni 6y ago
... and humans, tbh.
turing_complete 6y ago
I recommend "The Book of Why" by Judea Pearl as an introduction to the topic. It is targeted at a general (but educated) audience.
- epistasis 6y ago
  For some shorter reads, I saw this recommendation for biology undergrads yesterday: > I would really love all biology students to read Elliott Sober's "Apportioning Causal Responsibility", Susan Oyama's "Causal Democracy and Causal Contributions in Developmental Systems Theory", and Richard Lewontin's "The Analysis of Variance and the Analysis of Causes". https://twitter.com/hanemaung/status/1321488068717719552?s=21 https://twitter.com/hanemaung/status/1321488068717719552?s=2... These are available through research gate for those without library access.
- jasonwatkinspdx 6y ago
  The Epilog to his book Causality is a reasonably short read and does a good job of introducing the concepts imo. It's basically the same content he used in his Turing award speech and similar presentations over the last decade or so: http://bayes.cs.ucla.edu/BOOK-2K/causality2-epilogue.pdf http://bayes.cs.ucla.edu/BOOK-2K/causality2-epilogue.pdf
NikolaeVarius 6y ago
I was under the impression understanding causality is like, the secret of space, time and existence.
- krapp 6y ago
  To understand causality, you must first have understood causality.
  - ganzuul 6y ago
    https://en.wikipedia.org/wiki/Unmoved_mover#First_cause https://en.wikipedia.org/wiki/Unmoved_mover#First_cause One should mention the anthropic principle in this context. This would then lead one to speculate about a connection between observing causal order and self-awareness.
clircle 6y ago
Why does anyone think AI can pick up on causation when humans can't even do it?
- nabla9 6y ago
  Humans can and we are really good at it. I suspect that you made that comment because you think in the terms of rigorous detection of causality, not everyday effective detection of causality heuristically.
  - shatnersbassoon 6y ago
    Yes - humans do it abductively, which isn't rigorous but serves our purpose most of the time. You can't trust a self driving system with abduction though - it's the style of reasoning that gave us rain dances and homeopathy.
    - nabla9 6y ago
      >You can't trust a self driving system with abduction though All wetware based self driving systems on the road use heuristics in their wetware system. You can't hope to have self driving without heuristics. That's what deep learning is.
  - chillacy 6y ago
    Nowadays yes. For most of history though (and still commonly today), causal explanations for plenty of things involved some flavor of the supernatural.
- shatnersbassoon 6y ago
  I'm sure there was a philosopher who said that his deepest wish was to know just one cause. Been trying to find out who said it for ages with no luck, so maybe I'll just attribute it to myself.
  - goatlover 6y ago
    David Hume perhaps? He reasoned that causality was not empirical and therefore was a habit of thinking humans acquired from constant conjunction of events. Kant was troubled by that so he elevated causality to a category of thought, like space and time, which the mind used to structure sensations.
mooneater 6y ago
Which book should you read? https://www.bradyneal.com/which-causal-inference-book https://www.bradyneal.com/which-causal-inference-book
- deleted 6y ago
  [deleted]
- cschmidt 6y ago
  It is worth mentioning that the "Causal Inference: What if" book in the flowchart is a free download. The dead tree version is still in the works. https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/ https://www.hsph.harvard.edu/miguel-hernan/causal-inference-...
mrfusion 6y ago
Anyone working on GPT-4? it might figure some of that out on its own?
blackbear_ 6y ago
The article is missing an important point: you cannot learn causality from observational data alone. It's not about shortcomings of this or that model, it's a theoretical impossibility. Reinforcement learning is uniquely positioned to build machines that understand cause-and-effect on their own because the algorithm is allowed to interact with the world, observe the results, gather more data, rule out hypotheses, and so on.
- dfmooreqqq 6y ago
  Agreed. But even RL is often difficult to transfer to other domains of knowledge. Though I agree that this is an important next challenge (and causality has been the "next challenge" for at least the last 5 years), it's often more easily solved these days with mixing in human expert knowledge to the equation (that is, using ML alongside of human expertise).
- jasonwatkinspdx 6y ago
  While CRTs are the gold standard for establishing causality, that doesn't mean all is lost in observational data alone. Causal inference is a rich field, with a lot of work in recent decades. There's so much more to causal inference than the old dismissive chesnut about correlation.
- jbay808 6y ago
  Can you elaborate on this? If agent A interacts with a system and can learn causal relationships, how could agent B who observes all of agent A's experiments not be capable of drawing the same conclusions? It seems that any theorem that rules out learning causality from observational data alone would also rule out learning causality from any kind of interactions. Unless you're assuming that agent A "knows" it has free will so its own actions have no cause, while agent B can't tell whether the environment caused agent A's actions or vice-versa. But if that's what the proof hinges on, it's pretty shallow, because agent A has no such guarantee that its own choices have no root cause.
  - blackbear_ 6y ago
    Sorry, I was quite sloppy in my previous comment. Agent B certainly can learn everything that A learns from the same observations. What I meant is that you cannot learn from "general" observational data, unless it is structured in a certain way (the randomized controlled trial mentioned in a sibling). RL is able to gather data on its own, while other ML methods must do with what they are given. This means that RL could eventually discover the causal relationships, while non-RL cannot (except if the data comes from a RCT).
wenc 6y ago
(First-order) causality actually isn't hard to determine in environments where first-order effects dominate. The way to determine causality here is through a combination of physical laws and controlled experimentation [1]. In fact, we have plenty of causal models (e.g. 1st principles physics-based models, or design-of-experiments models). Without these models, machines/control systems/etc would not work. The trouble is, outside of these 1st-order effect dominant, deterministic environments, causality becomes much harder. In complex systems, stochasticity, nonlinearity, feedback loops and higher-order effects dominate. There's also emergent behavior -- properties that are true in the small are not true in the large. Consider a complex system like human society -- can we truly determine causality of broad interventions? Likely not in a first-order way like in the physical sciences. We can do it imperfectly through tools like causal inference (Rubin) which makes much more modest claims about the "strength" of effects (average causal effect). Randomized Controlled Tests (RCT) is another tool for making causal claims. But in a complex world, 2nd, 3rd and higher order effects dominate and so the notion of root causes itself becomes ambiguous. Richard I. Cook once said "post-accident attribution to a 'root cause' is fundamentally wrong". Though humans are attracted to the idea of a chain of simple causes (which is why we have the myth of Mrs O'Leary's cow kick over a lantern and starting the Great Chicago Fire of 1871), there's typically no easily-identified root cause. First-order causal thinking assumes a Directed-Acyclic-Graph (DAG) idea of a causality chain which converge into a set of effects, but the reality is that such a DAG, if it can be represented, is likely to be infinitely complex in a complex environment. First-order causal thinking is an insufficient mental model in a complex environments. Instead, I think instead of aiming for a deep understanding of epistemic causality (where we try to know and represent causality), it's probably more useful to focus on instrumental causality (where we aim to know the main points of leverage that are effective in changing the system). I think we'll likely get very far just by finding the knobs that have the most effect on the variables we would like to change (that don't also simultaneously change variables that we wouldn't want to change). [1] to determine causality, we typically have to perturb the system -- determining causality through observational data is possible, e.g. via natural experiments, but there are many epistemic restrictions which limit the claims that can be made.
- jostmey 6y ago
  > to determine causality, we typically have to perturb the system Couldn't agree more! For computer models to understand causality, it must be able to interact with the environment and probe it. I think understanding causality is one and the same as reinforcement learning, where a computer model learns to interact with its environment
Animats 6y ago
It's good to see people working on this. They're focusing on the right part of the problem, too - predicting a few seconds ahead in a physical environment. Basic survival is about not really screwing up in the next 10 seconds. (People who do robotics are very aware of this.) Don't get tangled up in the philosophy of causality. That's not the immediate problem.
- ganzuul 6y ago
  As someone thoroughly tangled up in said philosophy, I concur. Here be dragons.
- ljw1001 6y ago
  Species survival requires looking much farther ahead, but it’s harder to do, and even harder to monetize.
- e_y_ 6y ago
  Focusing on the short term also seems like it'd make paperclip/Skynet scenarios less likely.
o_p 6y ago
Humans dont understand causality at all and yet they are considered intelligent though
nafizh 6y ago
The area is under-explored because AFAIK there is no introduction for someone adept in machine learning to learn the basics with coding projects in for example pytorch and showing how causal inference is bringing added benefit. If you want to advance causality research, lower the barrier to entry just like it has been done with deep learning.
princeb 6y ago
it wasn't too long ago that interpretable models are seen as unimportant in the field. the value of prediction is so much more valuable than any other result of machine learning application. explanatory models were considered fuddy-duddy econometric pseudo-science.
im3w1l 6y ago
I have a fun story about causality inference. I pressed a light switch and immediately thunder crashed. I pressed it again, no sound was heard. Followed by nervous laughter. I take this to mean that we have a notion of "effectiveness", and that consequences are attributed to preceding effective actions.
random_user456 6y ago
There have been a lot of advances recently. https://arxiv.org/abs/2010.12237v1 https://arxiv.org/abs/2010.12237v1
- sheepdestroyer 6y ago
  That's Deep Mind's "Algorithms for Causal Reasoning in Probability Trees" released on October 23rd. It got posted on HN a few days ago but I am surprised that it did not get more traction.

125 points by headalgorithm 6y ago

... and humans, tbh.

I recommend "The Book of Why" by Judea Pearl as an introduction to the topic. It is targeted at a general (but educated) audience.

epistasis 6y ago
For some shorter reads, I saw this recommendation for biology undergrads yesterday: > I would really love all biology students to read Elliott Sober's "Apportioning Causal Responsibility", Susan Oyama's "Causal Democracy and Causal Contributions in Developmental Systems Theory", and Richard Lewontin's "The Analysis of Variance and the Analysis of Causes". https://twitter.com/hanemaung/status/1321488068717719552?s=21 https://twitter.com/hanemaung/status/1321488068717719552?s=2... These are available through research gate for those without library access.
jasonwatkinspdx 6y ago
The Epilog to his book Causality is a reasonably short read and does a good job of introducing the concepts imo. It's basically the same content he used in his Turing award speech and similar presentations over the last decade or so: http://bayes.cs.ucla.edu/BOOK-2K/causality2-epilogue.pdf http://bayes.cs.ucla.edu/BOOK-2K/causality2-epilogue.pdf

NikolaeVarius 6y ago

I was under the impression understanding causality is like, the secret of space, time and existence.

krapp 6y ago
To understand causality, you must first have understood causality.
- ganzuul 6y ago
  https://en.wikipedia.org/wiki/Unmoved_mover#First_cause https://en.wikipedia.org/wiki/Unmoved_mover#First_cause One should mention the anthropic principle in this context. This would then lead one to speculate about a connection between observing causal order and self-awareness.

clircle 6y ago

Why does anyone think AI can pick up on causation when humans can't even do it?

nabla9 6y ago
Humans can and we are really good at it. I suspect that you made that comment because you think in the terms of rigorous detection of causality, not everyday effective detection of causality heuristically.
- shatnersbassoon 6y ago
  Yes - humans do it abductively, which isn't rigorous but serves our purpose most of the time. You can't trust a self driving system with abduction though - it's the style of reasoning that gave us rain dances and homeopathy.
  - nabla9 6y ago
    >You can't trust a self driving system with abduction though All wetware based self driving systems on the road use heuristics in their wetware system. You can't hope to have self driving without heuristics. That's what deep learning is.
- chillacy 6y ago
  Nowadays yes. For most of history though (and still commonly today), causal explanations for plenty of things involved some flavor of the supernatural.
shatnersbassoon 6y ago
I'm sure there was a philosopher who said that his deepest wish was to know just one cause. Been trying to find out who said it for ages with no luck, so maybe I'll just attribute it to myself.
- goatlover 6y ago
  David Hume perhaps? He reasoned that causality was not empirical and therefore was a habit of thinking humans acquired from constant conjunction of events. Kant was troubled by that so he elevated causality to a category of thought, like space and time, which the mind used to structure sensations.

mooneater 6y ago

Which book should you read? https://www.bradyneal.com/which-causal-inference-book https://www.bradyneal.com/which-causal-inference-book

deleted 6y ago
[deleted]
cschmidt 6y ago
It is worth mentioning that the "Causal Inference: What if" book in the flowchart is a free download. The dead tree version is still in the works. https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/ https://www.hsph.harvard.edu/miguel-hernan/causal-inference-...

mrfusion 6y ago

Anyone working on GPT-4? it might figure some of that out on its own?

blackbear_ 6y ago

The article is missing an important point: you cannot learn causality from observational data alone. It's not about shortcomings of this or that model, it's a theoretical impossibility. Reinforcement learning is uniquely positioned to build machines that understand cause-and-effect on their own because the algorithm is allowed to interact with the world, observe the results, gather more data, rule out hypotheses, and so on.

dfmooreqqq 6y ago
Agreed. But even RL is often difficult to transfer to other domains of knowledge. Though I agree that this is an important next challenge (and causality has been the "next challenge" for at least the last 5 years), it's often more easily solved these days with mixing in human expert knowledge to the equation (that is, using ML alongside of human expertise).
jasonwatkinspdx 6y ago
While CRTs are the gold standard for establishing causality, that doesn't mean all is lost in observational data alone. Causal inference is a rich field, with a lot of work in recent decades. There's so much more to causal inference than the old dismissive chesnut about correlation.
jbay808 6y ago
Can you elaborate on this? If agent A interacts with a system and can learn causal relationships, how could agent B who observes all of agent A's experiments not be capable of drawing the same conclusions? It seems that any theorem that rules out learning causality from observational data alone would also rule out learning causality from any kind of interactions. Unless you're assuming that agent A "knows" it has free will so its own actions have no cause, while agent B can't tell whether the environment caused agent A's actions or vice-versa. But if that's what the proof hinges on, it's pretty shallow, because agent A has no such guarantee that its own choices have no root cause.
- blackbear_ 6y ago
  Sorry, I was quite sloppy in my previous comment. Agent B certainly can learn everything that A learns from the same observations. What I meant is that you cannot learn from "general" observational data, unless it is structured in a certain way (the randomized controlled trial mentioned in a sibling). RL is able to gather data on its own, while other ML methods must do with what they are given. This means that RL could eventually discover the causal relationships, while non-RL cannot (except if the data comes from a RCT).

wenc 6y ago

(First-order) causality actually isn't hard to determine in environments where first-order effects dominate. The way to determine causality here is through a combination of physical laws and controlled experimentation [1]. In fact, we have plenty of causal models (e.g. 1st principles physics-based models, or design-of-experiments models). Without these models, machines/control systems/etc would not work. The trouble is, outside of these 1st-order effect dominant, deterministic environments, causality becomes much harder. In complex systems, stochasticity, nonlinearity, feedback loops and higher-order effects dominate. There's also emergent behavior -- properties that are true in the small are not true in the large. Consider a complex system like human society -- can we truly determine causality of broad interventions? Likely not in a first-order way like in the physical sciences. We can do it imperfectly through tools like causal inference (Rubin) which makes much more modest claims about the "strength" of effects (average causal effect). Randomized Controlled Tests (RCT) is another tool for making causal claims. But in a complex world, 2nd, 3rd and higher order effects dominate and so the notion of root causes itself becomes ambiguous. Richard I. Cook once said "post-accident attribution to a 'root cause' is fundamentally wrong". Though humans are attracted to the idea of a chain of simple causes (which is why we have the myth of Mrs O'Leary's cow kick over a lantern and starting the Great Chicago Fire of 1871), there's typically no easily-identified root cause. First-order causal thinking assumes a Directed-Acyclic-Graph (DAG) idea of a causality chain which converge into a set of effects, but the reality is that such a DAG, if it can be represented, is likely to be infinitely complex in a complex environment. First-order causal thinking is an insufficient mental model in a complex environments. Instead, I think instead of aiming for a deep understanding of epistemic causality (where we try to know and represent causality), it's probably more useful to focus on instrumental causality (where we aim to know the main points of leverage that are effective in changing the system). I think we'll likely get very far just by finding the knobs that have the most effect on the variables we would like to change (that don't also simultaneously change variables that we wouldn't want to change). [1] to determine causality, we typically have to perturb the system -- determining causality through observational data is possible, e.g. via natural experiments, but there are many epistemic restrictions which limit the claims that can be made.

jostmey 6y ago
> to determine causality, we typically have to perturb the system Couldn't agree more! For computer models to understand causality, it must be able to interact with the environment and probe it. I think understanding causality is one and the same as reinforcement learning, where a computer model learns to interact with its environment

Animats 6y ago

It's good to see people working on this. They're focusing on the right part of the problem, too - predicting a few seconds ahead in a physical environment. Basic survival is about not really screwing up in the next 10 seconds. (People who do robotics are very aware of this.) Don't get tangled up in the philosophy of causality. That's not the immediate problem.

ganzuul 6y ago
As someone thoroughly tangled up in said philosophy, I concur. Here be dragons.
ljw1001 6y ago
Species survival requires looking much farther ahead, but it’s harder to do, and even harder to monetize.
e_y_ 6y ago
Focusing on the short term also seems like it'd make paperclip/Skynet scenarios less likely.

o_p 6y ago

Humans dont understand causality at all and yet they are considered intelligent though

nafizh 6y ago

The area is under-explored because AFAIK there is no introduction for someone adept in machine learning to learn the basics with coding projects in for example pytorch and showing how causal inference is bringing added benefit. If you want to advance causality research, lower the barrier to entry just like it has been done with deep learning.

princeb 6y ago

it wasn't too long ago that interpretable models are seen as unimportant in the field. the value of prediction is so much more valuable than any other result of machine learning application. explanatory models were considered fuddy-duddy econometric pseudo-science.

im3w1l 6y ago

I have a fun story about causality inference. I pressed a light switch and immediately thunder crashed. I pressed it again, no sound was heard. Followed by nervous laughter. I take this to mean that we have a notion of "effectiveness", and that consequences are attributed to preceding effective actions.

random_user456 6y ago

There have been a lot of advances recently. https://arxiv.org/abs/2010.12237v1 https://arxiv.org/abs/2010.12237v1

sheepdestroyer 6y ago
That's Deep Mind's "Algorithms for Causal Reasoning in Probability Trees" released on October 23rd. It got posted on HN a few days ago but I am surprised that it did not get more traction.