The claim that most people training models make is that what they are doing is sufficiently transformative that it counts as fair use, and doesn't require a lic

The claim that most people training models make is that what they are doing is sufficiently transformative that it counts as fair use, and doesn't require a license. That means putting something in a software license that prohibits model training wouldn't do anything. In this case, what the model is doing is clearly (to me as an non-lawyer) not transformative enough to count as fair use, but it's possible that the co-pilot folks will be able to fix this kind of thing with better output filtering.