What I meant was that knowledge distillation from trained models is a very efficient process. That's why it "wants" to be free. Knowledge is easy to scrape, tr

What I meant was that knowledge distillation from trained models is a very efficient process. That's why it "wants" to be free. Knowledge is easy to scrape, transform and train on. Once it gets into the open datasets, all models trained on that data will inherit the skills. In language, more recently the most useful training data has been generated with GPT4. Its abilities get transferred very efficiently to lower models. I think the same will happen with images. We're going to generate synthetic datasets that would replace the original copyrighted works, but be more efficient and diverse.