4 ms·
Word-use is roughly Zipf-distributed [0] in most all modern languages, so there's not much difference in the number of commonly used words (as long as you ignor
by pash 12y ago
Word-use is roughly Zipf-distributed [0] in most all modern languages, so there's not much difference in the number of commonly used words (as long as you ignore basic differences like the existence of articles and the richness of word inflection).
An analysis of English and French texts on Project Gutenburg shows that 93 words account for half of English usage, versus 89 for French [1]. English has fewer words in the middle of the distribution (696 versus 795 at 70%) and in the near tail (6,428 versus 9,050 at 90%, and 14,736 versus 21,231 at 95%), but more in the extreme tail.
0. https://en.m.wikipedia.org/wiki/Zipf%27s_law https://en.m.wikipedia.org/wiki/Zipf%27s_law
1. http://1.1o1.in/en/webtools/semantic-depth http://1.1o1.in/en/webtools/semantic-depth
- pcrh 12y ago>50th percentile of word use at 93 words for English and 89 for French [1]. If I understand the meaning to that phrase correctly, it seems to be an extraordinarily useless measure
- murbard2 12y ago> 14,736 versus 21,231 at 95% So about 50% more words for French. The very extreme tail is less interesting because it captures the size of the dictionary, and words very rarely used. I think the exponent of the tail would be the most relevant metric, but I can't open those pdf. Can someone plot the inverse CDF and make a log-log plot?