Machine learning researchers are well aware of the problem of out-of-sample predictions, in fact most of the work goes into making systems appropriately general

Machine learning researchers are well aware of the problem of out-of-sample predictions, in fact most of the work goes into making systems appropriately generalizable by making architectures robust through dropout or regularization. The resulting systems might not correspond to our intuitive definition of "understanding", but they look more similar to what we know about the human brain than an elegant function with a small number of interpretable parameters.