from a linguistic standpoint a text is a whole lot more than language: it is an externalisation of thought that is fixed onto a medium using writing utensils an

from a linguistic standpoint a text is a whole lot more than language: it is an externalisation of thought that is fixed onto a medium using writing utensils and most of all, cultural norms in the form of a wild variety of different genres and forms of text, ranging from something like a stream of consciousness to something like a speech act. furthermore, text can be conceptually written or spoken and with the internet we got an explosion of text that is conceptually spoken. those are the things OP might be refering to in regards to the "entropy" that encodes much more than just the tokens themselves.