5 ms·

Oh yes! It touches language and compression, both dear subjects to programmers. If the accepted answer stands, that's remarkable. I wonder how one could measur

Oh yes! It touches language and compression, both dear subjects to programmers. If the accepted answer stands, that's remarkable. I wonder how one could measure a language efficiency. Maybe syllable count ? But one would need a sort of assembly to translate to and verify that a sentence computes the intended information.

enkid 3y ago
The problem is that there are complex concepts that translate to a single syllable in some languages but requires a lot of context or explanation in other languages. Would these be "single instructions" or multiple? Then when you really break it down, many concepts are used grammatically but are not required based on their context. Should you count that as extraneous or not? An example is that some languages use a case system to indicate the epistemology of a statement, how the speaker knew that information (i.e. saw it first hand, heard it second hand, or saw evidence of it). Depending on the context, this may be vital or extraneous. Therefore the same statement that is "compressed" to remove useless information would be compressed two different ways. That doesn't seem feasible. All of this is to say, this sounds good in theory, but the complexities on practice are insurmountable.
- 93po 3y ago
  > the complexities on practice are insurmountable. “Darmok and Jalad at Tanagra” begs to differ
- alcover 3y ago
  > single syllable in some languages but requires a lot of context or explanation in other languages then don't these other langs do a 'bad job' at compression ? > That doesn't seem feasible In general maybe not. But for some restricted 'assembly' ? "The cat is on the table" has no ambiguity. And in some langs like Polish it compresses better : "Kot jest na stole" (Cat is on table), same info, better syllable-wise compression (5 vs 7).
  - enkid 3y ago
    "The cat" has different information in it than "kot." That same sentence also translates to "A cat is on the table," but no English speaker would say "the cat" and "a cat" have the same meaning. In fact, I can't think of a context where both sentences would be interchangeable. The listener either already knows which cat is "the cat" or would be confused. "A cat" implies an unknown cat. If you walk into your house and say "a cat is on the table," the assumption is its an unknown cat. If you say "the cat" it's most likely a pet. In some contexts that matters and some it doesn't, therefore you can't just say Polish compresses better.
    - mbg721 3y ago
      "The cat" is folksy/chummy in a similar way to "the wife".
      - dlainhart 3y ago
        From a data compression/language efficiency standpoint, both sentences in both languages actually rely on a (potentially large) amount of unstated context to sort out these ambiguities. In some languages, this context can be totally unspoken and merely known to both the speaker and the listener. This absolutely MUST be accounted for if a truly correct translation is to be made. For instance, your assumption that the definite "the cat" is being used idiomatically like so: this sentence, used in the manner you offer, might be used in conversation might occur in a farmhouse somewhere between an old man and woman who have lived together in this house for a long time, i.e. American Gothic. There's a vast amount of shared information and a perception of very little ambiguity held by both the speaker and the listener (whether correct or mistaken!). Any of those might fail. Furthermore, to use this sentence in English unadorned by context requires that both the speaker and listener have a shared reference to _what_ cat is being referred to by the definite article, "the". This very well might come with an unambiguous default in other languages! Translation only gets more complicated from this.
        mbg721 3y ago
JoBrad 3y ago
Yes, but it’s lossy compression, so beware.
chewxy 3y ago
You could measure the efficiency of a language by computing how many bits are required to store the semantics of a word/phrase/sentence losslessly. Assuming ideal encoding to bits of course. Think of things like perplexity and the like. However, traditional way of computing efficiency of compression would not be useful for a meaningful analysis of the efficiency of a language. Barring issues like having an ideal encoding to bits, or even having the concept of "efficiency" being rigorously defined, there are problems just from the outset. Take context for example. All useful compression methods have some sort of decompression key involved. This could be the dictionary, or the bitmap or the know-how (for cases like RLE). In natural langauges, the compression/decompression key is stored in a distributed fashion across the minds of a society. "Darmok and Jalad at Tanagra" is a VERY efficient compression for what is presumably a very long story about two hunters who met at an island and fought a beast together, but it is only efficient to the people who speak that language. The "local" efficiency (to the population who speak the language) is very high, but the "global" efficiency isn't. So we must account for efficiency in terms of the size of the compressed concept as well as the compression key. And from my experience, it's a sorta lumpy kinda world out there.
- jdmichal 3y ago
  > You could measure the efficiency of a language by computing how many bits are required to store the semantics of a word/phrase/sentence losslessly. Assuming ideal encoding to bits of course. Think of things like perplexity and the like. This has been done! The answer is about 39 bits a second. https://www.science.org/content/article/human-speech-may-have-universal-transmission-rate-39-bits-second https://www.science.org/content/article/human-speech-may-hav...
- tsukikage 3y ago
  It is customary, when comparing performance of compression algorithms, to include the size of the tool needed for decompression in the compression benchmarks, since otherwise one can simply smuggle the uncompressed data in the decompression tool. ISTM a similar principle would need to apply here: learning the "Darmok and Jalad at Tanagra" language would involve absorbing many volumes of history and mythology where for the usual sort of language a dictionary, grammar reference and maybe a book of common idioms would suffice. Whatever metric is used to compare languages for efficiency should reflect this.
jdmichal 3y ago
I'm just a hobby linguist, but I believe research has generally shown that speech information density is relatively fixed. Languages with more complex syllabic structures end up speaking slower, but they can also code more information. https://www.science.org/content/article/human-speech-may-have-universal-transmission-rate-39-bits-second https://www.science.org/content/article/human-speech-may-hav...
notpushkin 3y ago
Finally, my favourite pastime: natural language golfing!
yieldcrv 3y ago
Hackersnews don’t have to be programming or tech
quickthrower2 3y ago
Lossy compression though. The songs meaning was lost because “too many times” is very important to make sense of the lyrics that follows.
throw0101c 3y ago
> I wonder how one could measure a language efficiency. An area of active research: * https://www.theatlantic.com/international/archive/2016/06/complex-languages/489389/ https://www.theatlantic.com/international/archive/2016/06/co... * https://www.degruyter.com/document/doi/10.1515/lingvan-2020-0081/html?lang=en https://www.degruyter.com/document/doi/10.1515/lingvan-2020-... A constructed language: * https://en.wikipedia.org/wiki/Ithkuil https://en.wikipedia.org/wiki/Ithkuil * Via: https://www.reddit.com/r/linguistics/comments/zyi8q/what_is_the_most_efficient_language/ https://www.reddit.com/r/linguistics/comments/zyi8q/what_is_... Doing a search for "language information entropy" also gives back a number of results.
seer 3y ago
While some languages “compress” information into less syllables, sometimes redundancy can be a feature. For example I feel like languages like spanish have longer words, but since the intonation doesn’t vary too much, fluent speakers can speak way faster, or to put it more accurately, fast speech is more prevalent. Similar thing I’ve noticed with the south indian language - Malayalam, just try to pronounce the name of the city - Thiruvananthapuram, local speakers would pronounce it with roughly the same speed as “London”, and would enunciate every syllable - its crazy.
- canistel 3y ago
  Since you mentioned Malayalam, the sentence can be made into a single word in the language - vitacholliyirunnereyaval (വിടചൊല്ലിയിരുന്നേറെയവൾ). Sounds lyrical, but does it. No magic but plain agglutination, and I am sure this should be possible in languages like Finnish too...
tetha 3y ago
My first impulse - after playing some programming/optimization games over the morning - was: Is these just one kind of efficiency for a language? For example, mandarin or japanese can be very short on the character count. However, this increases character complexity and makes the languages harder to learn. On the other hand, large parts of english tend to be simple to learn.
klodolph 3y ago
If you translate it back into English, “she had said goodbye too much”, it’s clear that the real question is, “did the author mean something by those choices which the translation obliterated?” Translation is a process which both erases information and introduces new information. Any comparison of languages which tries to evaluate which languages are more compact has to work with some assumptions about what information should be conveyed. A statistical distribution of language-independent messages. But when you choose a distribution, you’re encoding your biases. Not saying that language efficiency is a bunk concept, just that it’s a thorny, difficult concept to quantify. Same is true of data compression algorithms—there is no such thing as an absolute scale for Kolmogorov complexity, for the same reasons.
- bombcar 3y ago
  It’s hard enough to translate when there’s one meaning - but good writers often use multiple meaning of the same words - either for deeper meaning or for humor or other layered meanings. That’s why the translations of Asterix are so impressive.
  - mbg721 3y ago
    The Spanish dubs of the Simpsons have a similarly good reputation.
- scarmig 3y ago
  > Translation is a process which both erases information and introduces new information. There's an essay I enjoyed by Douglas Hofstadter which is all about this, though from an artistic POV without much (any?) information science. Translator, Trader. The title itself is a fun bit of translational wordplay on "traduttore, traditore."
- deleted 3y ago
  [deleted]
- magnat 3y ago
  > Translation is a process which both erases information and introduces new information "Translation is like a woman. If it is beautiful, it is not faithful. If it is faithful, it is most certainly not beautiful."
Someone 3y ago
Brevity isn’t the only goal with language; you also want robustness under noise and ease of random access (e.g. if you skim-read a text or if you enter a conversation a bit late or have to leave early) In an optimally breve language the meaning of a text could completely flip when a single letter/syllable/phoneme is changed. That, in turn, means listeners have to hear every letter/syllable/phoneme perfectly. Interestingly, natural languages already have a bit of both. As an example, if you skim-read a text and restart at “He said she wasn’t there anymore”, there are 3 ‘back references’ in that sentence that require you to look back in the text to find the meaning of. Also, a paragraph’s meaning can change by adding the sentence “Just joking.” Or even a simple “Not.”.

enkid 3y ago

The problem is that there are complex concepts that translate to a single syllable in some languages but requires a lot of context or explanation in other languages. Would these be "single instructions" or multiple? Then when you really break it down, many concepts are used grammatically but are not required based on their context. Should you count that as extraneous or not? An example is that some languages use a case system to indicate the epistemology of a statement, how the speaker knew that information (i.e. saw it first hand, heard it second hand, or saw evidence of it). Depending on the context, this may be vital or extraneous. Therefore the same statement that is "compressed" to remove useless information would be compressed two different ways. That doesn't seem feasible. All of this is to say, this sounds good in theory, but the complexities on practice are insurmountable.

93po 3y ago
> the complexities on practice are insurmountable. “Darmok and Jalad at Tanagra” begs to differ
alcover 3y ago
> single syllable in some languages but requires a lot of context or explanation in other languages then don't these other langs do a 'bad job' at compression ? > That doesn't seem feasible In general maybe not. But for some restricted 'assembly' ? "The cat is on the table" has no ambiguity. And in some langs like Polish it compresses better : "Kot jest na stole" (Cat is on table), same info, better syllable-wise compression (5 vs 7).
- enkid 3y ago
  "The cat" has different information in it than "kot." That same sentence also translates to "A cat is on the table," but no English speaker would say "the cat" and "a cat" have the same meaning. In fact, I can't think of a context where both sentences would be interchangeable. The listener either already knows which cat is "the cat" or would be confused. "A cat" implies an unknown cat. If you walk into your house and say "a cat is on the table," the assumption is its an unknown cat. If you say "the cat" it's most likely a pet. In some contexts that matters and some it doesn't, therefore you can't just say Polish compresses better.
  - mbg721 3y ago
    "The cat" is folksy/chummy in a similar way to "the wife".
    - dlainhart 3y ago
      From a data compression/language efficiency standpoint, both sentences in both languages actually rely on a (potentially large) amount of unstated context to sort out these ambiguities. In some languages, this context can be totally unspoken and merely known to both the speaker and the listener. This absolutely MUST be accounted for if a truly correct translation is to be made. For instance, your assumption that the definite "the cat" is being used idiomatically like so: this sentence, used in the manner you offer, might be used in conversation might occur in a farmhouse somewhere between an old man and woman who have lived together in this house for a long time, i.e. American Gothic. There's a vast amount of shared information and a perception of very little ambiguity held by both the speaker and the listener (whether correct or mistaken!). Any of those might fail. Furthermore, to use this sentence in English unadorned by context requires that both the speaker and listener have a shared reference to _what_ cat is being referred to by the definite article, "the". This very well might come with an unambiguous default in other languages! Translation only gets more complicated from this.
      - mbg721 3y ago

JoBrad 3y ago

Yes, but it’s lossy compression, so beware.

chewxy 3y ago

You could measure the efficiency of a language by computing how many bits are required to store the semantics of a word/phrase/sentence losslessly. Assuming ideal encoding to bits of course. Think of things like perplexity and the like. However, traditional way of computing efficiency of compression would not be useful for a meaningful analysis of the efficiency of a language. Barring issues like having an ideal encoding to bits, or even having the concept of "efficiency" being rigorously defined, there are problems just from the outset. Take context for example. All useful compression methods have some sort of decompression key involved. This could be the dictionary, or the bitmap or the know-how (for cases like RLE). In natural langauges, the compression/decompression key is stored in a distributed fashion across the minds of a society. "Darmok and Jalad at Tanagra" is a VERY efficient compression for what is presumably a very long story about two hunters who met at an island and fought a beast together, but it is only efficient to the people who speak that language. The "local" efficiency (to the population who speak the language) is very high, but the "global" efficiency isn't. So we must account for efficiency in terms of the size of the compressed concept as well as the compression key. And from my experience, it's a sorta lumpy kinda world out there.

jdmichal 3y ago
> You could measure the efficiency of a language by computing how many bits are required to store the semantics of a word/phrase/sentence losslessly. Assuming ideal encoding to bits of course. Think of things like perplexity and the like. This has been done! The answer is about 39 bits a second. https://www.science.org/content/article/human-speech-may-have-universal-transmission-rate-39-bits-second https://www.science.org/content/article/human-speech-may-hav...
tsukikage 3y ago
It is customary, when comparing performance of compression algorithms, to include the size of the tool needed for decompression in the compression benchmarks, since otherwise one can simply smuggle the uncompressed data in the decompression tool. ISTM a similar principle would need to apply here: learning the "Darmok and Jalad at Tanagra" language would involve absorbing many volumes of history and mythology where for the usual sort of language a dictionary, grammar reference and maybe a book of common idioms would suffice. Whatever metric is used to compare languages for efficiency should reflect this.

jdmichal 3y ago

I'm just a hobby linguist, but I believe research has generally shown that speech information density is relatively fixed. Languages with more complex syllabic structures end up speaking slower, but they can also code more information. https://www.science.org/content/article/human-speech-may-have-universal-transmission-rate-39-bits-second https://www.science.org/content/article/human-speech-may-hav...

notpushkin 3y ago

Finally, my favourite pastime: natural language golfing!

yieldcrv 3y ago

Hackersnews don’t have to be programming or tech

quickthrower2 3y ago

Lossy compression though. The songs meaning was lost because “too many times” is very important to make sense of the lyrics that follows.

throw0101c 3y ago

> I wonder how one could measure a language efficiency. An area of active research: * https://www.theatlantic.com/international/archive/2016/06/complex-languages/489389/ https://www.theatlantic.com/international/archive/2016/06/co... * https://www.degruyter.com/document/doi/10.1515/lingvan-2020-0081/html?lang=en https://www.degruyter.com/document/doi/10.1515/lingvan-2020-... A constructed language: * https://en.wikipedia.org/wiki/Ithkuil https://en.wikipedia.org/wiki/Ithkuil * Via: https://www.reddit.com/r/linguistics/comments/zyi8q/what_is_the_most_efficient_language/ https://www.reddit.com/r/linguistics/comments/zyi8q/what_is_... Doing a search for "language information entropy" also gives back a number of results.

seer 3y ago

While some languages “compress” information into less syllables, sometimes redundancy can be a feature. For example I feel like languages like spanish have longer words, but since the intonation doesn’t vary too much, fluent speakers can speak way faster, or to put it more accurately, fast speech is more prevalent. Similar thing I’ve noticed with the south indian language - Malayalam, just try to pronounce the name of the city - Thiruvananthapuram, local speakers would pronounce it with roughly the same speed as “London”, and would enunciate every syllable - its crazy.

canistel 3y ago
Since you mentioned Malayalam, the sentence can be made into a single word in the language - vitacholliyirunnereyaval (വിടചൊല്ലിയിരുന്നേറെയവൾ). Sounds lyrical, but does it. No magic but plain agglutination, and I am sure this should be possible in languages like Finnish too...

tetha 3y ago

My first impulse - after playing some programming/optimization games over the morning - was: Is these just one kind of efficiency for a language? For example, mandarin or japanese can be very short on the character count. However, this increases character complexity and makes the languages harder to learn. On the other hand, large parts of english tend to be simple to learn.

klodolph 3y ago

If you translate it back into English, “she had said goodbye too much”, it’s clear that the real question is, “did the author mean something by those choices which the translation obliterated?” Translation is a process which both erases information and introduces new information. Any comparison of languages which tries to evaluate which languages are more compact has to work with some assumptions about what information should be conveyed. A statistical distribution of language-independent messages. But when you choose a distribution, you’re encoding your biases. Not saying that language efficiency is a bunk concept, just that it’s a thorny, difficult concept to quantify. Same is true of data compression algorithms—there is no such thing as an absolute scale for Kolmogorov complexity, for the same reasons.

bombcar 3y ago
It’s hard enough to translate when there’s one meaning - but good writers often use multiple meaning of the same words - either for deeper meaning or for humor or other layered meanings. That’s why the translations of Asterix are so impressive.
- mbg721 3y ago
  The Spanish dubs of the Simpsons have a similarly good reputation.
scarmig 3y ago
> Translation is a process which both erases information and introduces new information. There's an essay I enjoyed by Douglas Hofstadter which is all about this, though from an artistic POV without much (any?) information science. Translator, Trader. The title itself is a fun bit of translational wordplay on "traduttore, traditore."
deleted 3y ago
[deleted]
magnat 3y ago
> Translation is a process which both erases information and introduces new information "Translation is like a woman. If it is beautiful, it is not faithful. If it is faithful, it is most certainly not beautiful."

Someone 3y ago

Brevity isn’t the only goal with language; you also want robustness under noise and ease of random access (e.g. if you skim-read a text or if you enter a conversation a bit late or have to leave early) In an optimally breve language the meaning of a text could completely flip when a single letter/syllable/phoneme is changed. That, in turn, means listeners have to hear every letter/syllable/phoneme perfectly. Interestingly, natural languages already have a bit of both. As an example, if you skim-read a text and restart at “He said she wasn’t there anymore”, there are 3 ‘back references’ in that sentence that require you to look back in the text to find the meaning of. Also, a paragraph’s meaning can change by adding the sentence “Just joking.” Or even a simple “Not.”.