4 ms·
This statement is true only if it is not possible to tell from the context if the noun refers to a specific/previously mentioned thing or not. It would be possi
by 112233 3y ago
This statement is true only if it is not possible to tell from the context if the noun refers to a specific/previously mentioned thing or not.
It would be possible to measure the amount of information contained in these articles, Shannon style, by taking a body of text, removing the articles, and then asking a bunch of english speakers ( that can possibly be approximated by a LLM ) to put back in the correct articles. Any uncertainty or variation would point to information being lost by the removal.
- tysam_and 3y agoI was thinking about Shannon entropy as well, as the OP completely forgets the word 'from' as well! "I go to house", "I go from house". Certainly, house contains more information, but the concepts of to and from as some kind of token do contain meaningful amounts of entropy as well.
- morsch 3y agoHuman languages are highly redundant, for the most part. The communication channel is lossy, so you add parity bits and error correction codes.