The problem is that correlation does not necessarily imply causation. So if a ML process finds a counter-intuitive correlation we should seek to understand why

The problem is that correlation does not necessarily imply causation. So if a ML process finds a counter-intuitive correlation we should seek to understand why rather than assume the results are actually meaningful. Sadly I think this is the step that is being gratuitously skipped to the extent that “working” models are hardly interrogated.