Here's my usual rant about ML not doing it because we don't give it the right data. You can't learn causation from a table of floats. You can learn causation fr

Here's my usual rant about ML not doing it because we don't give it the right data. You can't learn causation from a table of floats. You can learn causation from a sufficiently annotated table of floats. If I tell you that here's a column of people's weights and a column of their diets, no model can learn the causal connection between the two. If I tell you that here's a column of peoples weights and here's a column of their diets which were randomly assigned under intervention, then suddenly a model can do it. All causal inference with observational data requires assumptions about conditional independence structure. It's so crucial that we always explain it all in prose in any writeup of any given causal investigation. We put none of that in tables themselves, despite it being entirely crucial. If we started making postgres tables that stopped looking like "height: float, weight: float" and started looking like "height: float, weight: do(float)" (as in pearl's do(...)) then we could start to automate causal inference much much more easily. Not to say the types would be nearly so simple. You'd need a full DAG for your database, and even then, it's not that simple: our AB testing platform (v3.1) intervened here according to this 1k line python script (git commit 191284794) that took in these columns and employed a model trained on this other entire table as of date X, before we migrated the db. Also this one column's meaning changed in november when we removed a button from the home page. But without some structured encoding of the structure that an analyst is going to need (structure they're absolutely going to be encoding in natural language in their writeup), we're trying to do it with one arm tied behind our backs.