Well the majority of trained neural network models fail to solve most problems they're tested on, also. When you pick up a neural net paper (or really any machi

Well the majority of trained neural network models fail to solve most problems they're tested on, also. When you pick up a neural net paper (or really any machine learning paper) and look at the results of experiments the authors will invariably list the results of their best-performing model, chosen by careful cross-validation over multiple random seeds. The point of this testing is to estimate the true learning ability of a system by looking at what it can do in the best case scenario. Nobody reports the worst-case scenario, otherwise deep neural nets would look a lot less impressive than they do.