> That said, OP is referring to whether the resulting model is able to produce verbatim copies of the data. Transformers are fundamentally large compression a

> That said, OP is referring to whether the resulting model is able to produce verbatim copies of the data. Transformers are fundamentally large compression algorithms where the target of compression is not just to minimize reconstruction loss + compressed file size. In fact, basically all of machine learning used today can be viewed through the lens of learning a compression algorithm with added goals other than the usual. By this logic if I create a lossy Jpeg of a copyrighted image it's not "copying" because the lossy compression.