6 ms·
>Scientists started with written texts from 17 languages, including English, Italian, Japanese, and Vietnamese. They calculated the information density of each
by SketchySeaBeast 7y ago
>Scientists started with written texts from 17 languages, including English, Italian, Japanese, and Vietnamese. They calculated the information density of each language in bits—the same unit that describes how quickly your cellphone, laptop, or computer modem transmits information.
I feel like the whole "bits" calculation is a neat way to get into the media, but not actually related to "information density".
Edit: Been informed I'm deeply ignorant on Information Theory.
- paggle 7y agoIt's hard to translate to bits, but there is definitely different information densities between language. E.g. when you write a paragraph in Spanish, the total number of keystrokes you'll need is higher than writing the same thing in English.
- monocasa 7y agoBits is a base unit of information theory.
- sansnomme 7y agoTheoretically a summarisation algorithm (or neural network) should reduce sentences of similar content from different languages to similar sizes no?
- furgooswft13 7y agoHuman speech uses the extremely dense lossy compression method called "A Lifetime of Experience and Biases"
- burnte 7y agoIt's only lossy when the decoder's dictionary doesn't have a similar volume of data points to properly reinflate the data stream. Things that didn't make sense to me as a teen make far too much sense now because my experience can properly contextualize the old data.
- furgooswft13 7y agoOr lossy when the decoder has a different volume of data points and extracts totally different information or even info that never existed in the first place. Kinda like those anime/retro game upscaling filters, except with way higher variance on what comes out between decoders, or the same decoder at different times. Gives me an idea for a new JPEG decoder with a floating ADC as input.
- wahern 7y agoBit is the unit usually used for describing entropy, information density, and channel capacity, regardless of whether computer systems are involved. See https://pdfs.semanticscholar.org/d554/d933fd78e19d51d68ac4eabab6810cf0ba99.pdf https://pdfs.semanticscholar.org/d554/d933fd78e19d51d68ac4ea... and, more generally, https://en.wikipedia.org/wiki/Channel_capacity https://en.wikipedia.org/wiki/Channel_capacity and https://en.wikipedia.org/wiki/Information_theory https://en.wikipedia.org/wiki/Information_theory. The field of Information theory effectively began with Claude Shannon. The same formalisms he developed are used outside computer science--linguistics, physics, microbiology, etc.
- TacticalTable 7y agoThis has actually been a fairly popular idea within linguistics for a long time, and is absolutely relevant to information density. Problems like these need to be better understood for natural language understanding and translation, and 'bits' provide a good baseline measurement.
- rrss 7y agoThe authors use syllable conditional entropy to estimate information density, and entropy is measured in bits (or shannons, if you prefer).