If there is not enough diverse training data then perhaps you should not call it learning in first place. The whole notion of learning is about algorithm + corr

If there is not enough diverse training data then perhaps you should not call it learning in first place. The whole notion of learning is about algorithm + correct train data meant to make sense of certain kind of test data. All three components matter a lot and it is something that even the experts sometimes seem to miss. Running a crude algorithm some small set of training data specifically meant to amaze you on specific algorithm have very little information content.