3 ms·
I would attribute this result more to "substantially all corpora of human languages will exhibit a zipf distribution in word frequency" than to "kids these days
by patio11 17y ago
I would attribute this result more to "substantially all corpora of human languages will exhibit a zipf distribution in word frequency" than to "kids these days."
For example, off the top of my head I know you can do about 90% of jpop songs with 200 words, and I say without fear of contradiction that Wikipedia, the Bible, the contents of the Gutenberg free library, or any other substantial corpora will also evidence a zipf distribution.
Bonus points: the zipf distribution for the King James version of the Bible and the zipf distribution for text messages would be virtually indistinguishable if you rank ordered them and then swapped n% of the words across the two lists. I think n is likely to be about 10. A fun project for after Easter if they've published their dataset anywhere...
Sorry -- natural language processing is my first love.