The claim is that the models learn the inherent statistical properties of the training set, not the relationships and causality they were intended to learn. Whe

The claim is that the models learn the inherent statistical properties of the training set, not the relationships and causality they were intended to learn. When they removed a statistical property of the training set, their BERT model got more accurate.