Well, right. A binary classification system must assign a 0 or 1. But cross validation or other methods may reveal that it isn't a good fit, just a fit. As mlt

Well, right. A binary classification system must assign a 0 or 1. But cross validation or other methods may reveal that it isn't a good fit, just a fit. As mlthoughts pointed out in a different comment, any kind of regression technique faces issues about goodness of fit. The thing is, there are techniques to show you that the fit isn't very good. A simple linear regression will fit randomized noise, but there are outputs that can show you that the fit isn't good and the regression may not be reliable. The question I have here is whether ML techniques are failing in a different way, that it is fitting to randomized noise while appearing by various tests to be a very strong fit. If they're failing the same way that regression would (i.e.., someone applies it and fails to do basic tests for goodness of fit), that's a problem I suppose, but is it really a unique failing of ML or neural nets? It sounds like more like a standard misapplication of predictive modeling...