I wouldn't be so sure that there isn't "any" form of compression going on - that just seems silly. It seems that at some level it is *fantastic* lossy semantic

I wouldn't be so sure that there isn't "any" form of compression going on - that just seems silly. It seems that at some level it is *fantastic* lossy semantic compression. This is one of the more fascinating aspects of LLMs from a computer science perspective - almost all of human knowledge and experience in text from (several petabytes of data in a database) can be 'cleverly semantically compressed' to fit into a database of just a few hundred GB (NN params). That's some astounding lossy compression algorithm, unfortunately with a less than efficient decoder (temp=0). However, it isn't always 'plagiarism' either. 'Memorization' vs 'Plagiarism' vs 'understanding' is actually very tricky in LLMs, because having a good world model requires memorizing some facts, as well as some (potentially) "ontological" constructions. In addition, quoting a famous person is very useful for imparting some wisdom occasionally, which is also memorization, and can be seen as plagiarism if the author is left off. But sometimes this occurs (common sayings) and no one bats an eye. One good example of why explicit memorization is really needed is to provide exact references (list of authors, year, and article title) - so clearly making a NN not rewarded for memorizing is not ideal. Hence, *very occasionally*, you get some 'bad memorization'. But it is worth noting that the alignment achieved with these models for such an enormously complex task is quite good.