It is actually the other way round. The models would be even better if their overlords decided to let it train and learn anything unencumbered. These limitation

It is actually the other way round. The models would be even better if their overlords decided to let it train and learn anything unencumbered. These limitations are deliberate and ethical. One example is that stable diffusion (SD) (think GPT for images) does a pretty bad job of rendering humans. It also isn't trained on NSFW data. Now, people took these SD models and trained them on pornographic images. Turns out, this new model while excellent at generating NSFW images, also became really good at creating humans in general. There are similar gains that we are ethically leaving on the table. IMO, it's for the best. The field is moving fast enough as it is.