2 ms·
Is this that the n-grams are wrong, or that they are limited in what you can do/say with them? I find the data fun, but I'm not entirely sure what to make of i
by taeric 3y ago
Is this that the n-grams are wrong, or that they are limited in what you can do/say with them? I find the data fun, but I'm not entirely sure what to make of it. You will be doing a query on past books on today's lexicon. Which just feels wrong.
As an easy example that I know, if you search for "þe", you will not find a lot of hits. Which, is mostly fair, as historically we know that "þ" dropped off around the 1400s. That said, add in "ye" and you see a ton of its use.
Is that an intentional feature of n-grams? Feels more like an encoding mistake passed down through the ages. Would be like getting upset at the great vowel shift and not realizing that our phonetic symbols are not static universal truths.