"The high accuracy of predictions in tests often fools people into thinking the models are picking up on the 'true structure of the problem' in a human-like way

"The high accuracy of predictions in tests often fools people into thinking the models are picking up on the 'true structure of the problem' in a human-like way..." A form of confirmation bias?