You are splitting hairs, there is no doubt that transformers can incorporate and reproduce verbatim training data. Just ask ChatGPT to tell you the lyrics of a

You are splitting hairs, there is no doubt that transformers can incorporate and reproduce verbatim training data. Just ask ChatGPT to tell you the lyrics of a famous Madonna song. On that basis alone they are derivative works, regardless of how that data is represented internally.