Statistically they must be right - most of the statistical improvement by an ML system is provided by the first "large" "good" data set, after that gains are su

Statistically they must be right - most of the statistical improvement by an ML system is provided by the first "large" "good" data set, after that gains are sure to be marginal. But, a lot of problems aren't really statistical in nature; they are scientific in that if you can discover a new, better, theory and use that then you can catch a few cases that you may often do the right thing on without knowing why, and sort those out. For example - you may be learning to diagnose a disease with one common driver, there may be a rare disease that responds to the same treatment but has a slightly different presentation, you often treat these patients because of false positives, but if you learn to properly classify them then you can almost always catch them; the statistical gain is negligible, the value is tangible.