4 ms·
Working with word co-occurence clustering based on Kolmogorov complexity I have become convinced that there is a computational complexity element to every langu
by compbio 11y ago
Working with word co-occurence clustering based on Kolmogorov complexity I have become convinced that there is a computational complexity element to every language.
Words like "circumvent" and "environment" are close in regards to complexity. Words like "us" and "me" are close in regards to complexity.
The counting argument tells us that most strings are not compressible. It is then a wonderful feature of sensor data, natural language, DNA and computer code that it can be compressed quite a bit. This means there is a certain order in the language that compressors can use to keep the file size smaller.
There is a cognitive economy trade-off between the energy needed to keep a system running and increased complexity. Less complex language helps us save energy. We use short words for concepts that we use often. Very complex concepts and words like "disambiguation" may be described with shorter simpler words to someone who has not stored that word and general accepted meaning yet.
In this complexity view languages evolve to use as little energy/computational complexity to convey as much information as possible. The results found in this article can also be explained using this view. Parsing a sentence like "Throw the trash out" requires you to store in working memory the word "throw" 'till you get to the word "out" for the full concept "to throw out". Until you get to the word "out", the "throw" remains in a superstate (could become "throw in", "throw on" etc.). You need both words to form a mental picture of someone throwing out the trash. This requires more computational energy to the listener, and is hence ineffective. If you want your message to be heard, you have to communicate in clear simple-energy sentences. So using simpler less computationally intensive sentences benefits both the speaker and the listener.
This would readily explain why natural languages beat the random benchmark. Randomness has far less structure to use for compression by an intelligent agent. Randomness is not optimized communication, since it is more unpredictable.
In short: Simplicity and conveying information with little energy is a fitness factor that natural selection optimizes for. This is universal to all natural language speaking agents with a limited energy budget.
- rndn 11y agoIt’s probably more memetics (constrained by cognitive load) than genetics that select the features of languages, and there are likely many other factors that determine the complexity of languages and grammatical structures: For example, there is a sweet spot between minimal symbol count and minimal lengths of the words. In the extremes you have either short words but many symbols, or few symbols but long words. At the same time the number of distinguishable phonemes and therefore symbols are, of course, restricted by the sounds that are producible by the average vocal tract. Secondly, the communication channels thought→vocalization→hearing→thought or even thought→typing→reading→thought are inherently very noisy, so you end up with a lot of redundancy, like particles, introduction and transition phrases. And lastly, I think that there are always some words and phases that are not shaped by efficiency/cognitive load, but rather by whether it is fun or fashionable to talk in a certain way. There is certainly some cultural variance that can be orthogonal to efficiency.
- compbio 11y agoVery thought provoking answer! Thanks. As memes house in agents with an energy budget, I think that shorter simpler memes have more chance to take hold and reproduce ("Make something people want"). Words in a sentence are like models in an ensemble. Simple words are more general and have a high bias and low variance ("Make stuff users want"). Highly complex words and sentences have a lower bias, but a higher variance. You need to average a lot of them to get a clear picture. That's why the sentences in scientific papers are usually so long, they need to gradually cancel out the noise. > There is certainly some cultural variance that can be orthogonal to efficiency. Yes, agreed! Though same with memes, certain words or symbols without any redundancy may have cultural value. You may gain energy by speaking a certain language to a certain degree of sophistication. You may have to invest energy to gain access to the information contained in symbols (or have agents "unzip" these for themselves).