Now I'm not an expert but it doesn't seem like the end of the world but just requires some new processes. 1. Using large models to train small models is alread

Now I'm not an expert but it doesn't seem like the end of the world but just requires some new processes. 1. Using large models to train small models is already a thing. 2. AI can sometimes label data better than humans. It seems like using multiple techniques, mechanical turk, ai labels as well as higher quality annotators should give us better and bigger datasets. This one is more a question. Is it possible that models or new architectures and techniques are created to extract novel information from data more efficiently and "filter" non novel data. I don't think humans weigh all information they consume equally.