It does introduce new information. This is how semi-supervised learning works. Many words may be missed by the weak model, but can be inferred correctly from th

It does introduce new information. This is how semi-supervised learning works. Many words may be missed by the weak model, but can be inferred correctly from their context. Then you have new labels to train the weak model on, to make it better at those words. The way I'm describing it is probably not the optimal way to do it. I don't know if there is a better way. But the point is it must be possible to take advantage of the vast quantities of unlabelled data we have. Humans brains somehow do something similar. Semi supervised learning is really cool. I saw one successful example where they labelled just a few emails as spam and not spam. Then they trained used their weak classifier to label thousands of unclassified emails, and then used those as training data for an even stronger model. It actually works: http://matpalm.com/semi_supervised_naive_bayes/semi_supervised_bayes.html http://matpalm.com/semi_supervised_naive_bayes/semi_supervis... https://en.wikipedia.org/wiki/Semi-supervised_learning https://en.wikipedia.org/wiki/Semi-supervised_learning