3 ms·
You could compress the written language in UTF-8 or the IPA version of the languages words and potentially get very different results. Also, languages tend to
by Symmetry 8y ago
You could compress the written language in UTF-8 or the IPA version of the languages words and potentially get very different results. Also, languages tend to be spoken at about the same bitrate even though they may have different numbers of bits per syllable. Spanish and Japanese, limited to pure vowel sounds and neither tending to pile up the consonants, tend to have less information per syllable but be spoken rapidly. English, with its diverse vowels and relatively consonant-rich syllables, and Chinese, with its medium number of phonemes but added layer of tones, tend to be spoken more slowly.
- bediger4000 8y agoThe author of the article would have been wise to include this information. I did find "Clustering by Compression", https://arxiv.org/abs/cs/0312044 https://arxiv.org/abs/cs/0312044 The emphasis in this paper was not on "efficiency", but rather on phylogenetic grouping. The authors used the "Universal Declaration of Human Rights", which apparently has 52 translations as the subject text. See Fig 13 and section 5.2 for details. Instead of just compressing and normalizing, they developed a normalized compression distance, which involves concatenating two texts, compression, and division by the largest of the two texts, as compressed alone. I think that using the same encoding of all the languages and some kind of normalizing would wash the results of differences due to encodings.