There is currently no law in the US or EU that I'm aware of that consider models trained on PII or copyright data to be considered as including that data. This

There is currently no law in the US or EU that I'm aware of that consider models trained on PII or copyright data to be considered as including that data. This is the core issue everyone is discussing about copyright around models and whether publicly scraped data is fair use for these purposes. Until this is decided, likely by a long series of trials in high courts, or passage of law through the appropriate legislative bodies (and then likely after many challenges to it) will GDPR and CCPA be able to apply to these language models. So no this is not currently a violation of any law.