That does raise the question how irreversibly you have to process something before it is no longer protected by copyright. Obviously zipping something is not e

That does raise the question how irreversibly you have to process something before it is no longer protected by copyright. Obviously zipping something is not enough, even lossy compression is (obviously) not enough. But then how does a language model differ from a lossy compression? Is it just the compression ratio? (are the weights even that much smaller than the data?) There are ways to train models that guarantee that the amount of information transferred per data point is limited, but to my knowledge those aren't used (and may be prohibitively expensive).