at some points it starts to resemble more of a lossy compression algorithm. The 'next most likely word' aspect of it all seem to encode entire sequences of text

at some points it starts to resemble more of a lossy compression algorithm. The 'next most likely word' aspect of it all seem to encode entire sequences of text. Especially if it is a bit of a niche domain. Then these texts can be 'retrieved' from the 'compressed' set of examples.