There is a good paper by Leo Breiman [1] which discusses the exact issue that you mentioned. E.g. Statisticians believe that there is a "True" model that genera

There is a good paper by Leo Breiman [1] which discusses the exact issue that you mentioned. E.g. Statisticians believe that there is a "True" model that generates the data, and the errors observed are merely noise. ML on the other hand does not assumes existence of a "True" model, the assumption is that only the Data is source of the truth and any model that predicts the data with lowest error is preferable subject to performance on test/cross-validation etc. This is a powerful approach and distinguishes ML as separate field from statistics. My favorite example is predicting prices of real estate. A statistician will build a multi-level model that takes into account various effects (Zip code level, city level, school district level, year/month of acquisition etc.) and then build a regression model. The errors then would then be simply noise, an ML approach would be simply using weighted K-Nearest Neighbors, with geographic location as part of distance metric. Sure there are no effects to adjust but the K-NN regression model can account for difficult to capture quirks of geography by being able to represent local non-linear decision surface. http://projecteuclid.org/euclid.ss/1009213726 http://projecteuclid.org/euclid.ss/1009213726