19 ms·

Human speech may have a universal transmission rate: 39 bits per second

671 points by headalgorithm 7y ago

SketchySeaBeast 7y ago
>Scientists started with written texts from 17 languages, including English, Italian, Japanese, and Vietnamese. They calculated the information density of each language in bits—the same unit that describes how quickly your cellphone, laptop, or computer modem transmits information. I feel like the whole "bits" calculation is a neat way to get into the media, but not actually related to "information density". Edit: Been informed I'm deeply ignorant on Information Theory.
- paggle 7y ago
  It's hard to translate to bits, but there is definitely different information densities between language. E.g. when you write a paragraph in Spanish, the total number of keystrokes you'll need is higher than writing the same thing in English.
- monocasa 7y ago
  Bits is a base unit of information theory.
- sansnomme 7y ago
  Theoretically a summarisation algorithm (or neural network) should reduce sentences of similar content from different languages to similar sizes no?
- furgooswft13 7y ago
  Human speech uses the extremely dense lossy compression method called "A Lifetime of Experience and Biases"
  - burnte 7y ago
    It's only lossy when the decoder's dictionary doesn't have a similar volume of data points to properly reinflate the data stream. Things that didn't make sense to me as a teen make far too much sense now because my experience can properly contextualize the old data.
    - furgooswft13 7y ago
      Or lossy when the decoder has a different volume of data points and extracts totally different information or even info that never existed in the first place. Kinda like those anime/retro game upscaling filters, except with way higher variance on what comes out between decoders, or the same decoder at different times. Gives me an idea for a new JPEG decoder with a floating ADC as input.
- wahern 7y ago
  Bit is the unit usually used for describing entropy, information density, and channel capacity, regardless of whether computer systems are involved. See https://pdfs.semanticscholar.org/d554/d933fd78e19d51d68ac4eabab6810cf0ba99.pdf https://pdfs.semanticscholar.org/d554/d933fd78e19d51d68ac4ea... and, more generally, https://en.wikipedia.org/wiki/Channel_capacity https://en.wikipedia.org/wiki/Channel_capacity and https://en.wikipedia.org/wiki/Information_theory https://en.wikipedia.org/wiki/Information_theory. The field of Information theory effectively began with Claude Shannon. The same formalisms he developed are used outside computer science--linguistics, physics, microbiology, etc.
- TacticalTable 7y ago
  This has actually been a fairly popular idea within linguistics for a long time, and is absolutely relevant to information density. Problems like these need to be better understood for natural language understanding and translation, and 'bits' provide a good baseline measurement.
- rrss 7y ago
  The authors use syllable conditional entropy to estimate information density, and entropy is measured in bits (or shannons, if you prefer).
jumelles 7y ago
Being able to convey emotion with tone in speech likely adds a lot more "information".
- _nhynes 7y ago
  The increased information density in tonal language appears to be offset by a reduced speaking rate [0]. [0] http://ohll.ish-lyon.cnrs.fr/fulltext/pellegrino/Pellegrino_to%20appear_Language.pdf http://ohll.ish-lyon.cnrs.fr/fulltext/pellegrino/Pellegrino_...
  - pmontra 7y ago
    I think jumelles was referring to something like "don't do that" uttered as fun, angry, etc.
  - SopaDeMacaco 7y ago
    That's not what he said, sperglord.
- 8bitsrule 7y ago
  So very true. Said in the right tone, "I don't care" can mean -many- things. Emotional intelligence is acquired (or not). 'Bit rate' means nada. As well, the 'amount of information' conveyed depends on the environment, and the preparation of the speaker and the listener. Some speakers (not to mention any names) spew a lot of BS ( not information) to sort through. I'd argue that, in a medical environment, the word 'sponge' conveys less information than the word 'ebola'. For the prepared, the cascade of necessary reactions in the brain can take little or much time to process. Some authors/speakers (authoritative) can pack -a lot to think about- in a few words. Like 'Where is everybody?'
Mathnerd314 7y ago
> instead of being limited by how quickly we can process information by listening, we’re likely limited by how quickly we can gather our thoughts. That’s because, he says, the average person can listen to audio recordings sped up to about 120%—and still have no problems with comprehension. “It really seems that the bottleneck is in putting the ideas together.” Glad this paragraph was in the article, clears up their methodology. I wonder if it applies to writing too, or if skilled writers work faster.
- Aperocky 7y ago
  Really depends on language, if you're writing java, you'll be putting out a lot more than that due to how stupidly verbose it is.
  - jefftk 7y ago
    > if you're writing java, you'll be putting out a lot more than that due to how stupidly verbose it is Being "verbose" means that each letter you type communicates fewer bits of information. If the bottleneck is putting ideas together then you would expect someone writing in a more verbose language to type more letters per minute but still take a similar amount of time to communicate the idea. In practice most Java programmers are using IDEs with good auto-completion, though, so aren't actually needing to type as many letters as you'd think.
    - dual_basis 7y ago
      What might be more relevant, however, is the effect of verbosity on comprehension. Syntax highlighting can help, and humans will likely naturally perform input compression (chunking), but this must be learned.
    - the_af 7y ago
      Like the sibling comment mentioned, it seems verbosity hinders reading comprehension rather than writing. Many IDEs understand this and hide some of the boilerplate. This raises the question: if the IDE autocompletes the boilerplate for you, and also hides it, why is it needed in the first place?
jefftk 7y ago
Here's the actual paper: https://advances.sciencemag.org/content/5/9/eaaw2594 https://advances.sciencemag.org/content/5/9/eaaw2594
ilamont 7y ago
But, he says, instead of being limited by how quickly we can process information by listening, we’re likely limited by how quickly we can gather our thoughts. That’s because, he says, the average person can listen to audio recordings sped up to about 120%—and still have no problems with comprehension. Some years ago I worked on an accessibility project for an app and website designed for people with disabilities. One of the team members had low vision, and used a screen reader that must have been set to 3x or even higher. I usually listen to YouTube and podcasts at 1.5-2x and I could barely understand the audio. He seemed surprised, which indicated to me that 3x+ was the norm for people in his circle. I wonder if his ability was trained through years of using fast screen readers, vs. a lower visual processing load leads to better audio processing, or some other explanation.
- vlovich123 7y ago
  Yes. Took me a while but I can comfortably understand 2x speed and now 1x podcasts seem weird like they're talking super slow. I would imagine it's something you just train even more out of necessity.
- liability 7y ago
  I typically use TTS near 2.5x (I turn it up when I'm alert and down when I'm tired.) It's definitely a learned skill; a few years ago I started at 1x and struggled even with that. Every couple of months, take a moment to reflect on your comprehension. Is it currently easy for you to understand the audio? If yes, then crank it up a little bit until it's noticeably more difficult. Repeat this process periodically over a year or so and before you know it, it'll be set pretty damn quick.
- dual_basis 7y ago
  I can watch YouTube content at 3x without issue. I did this without much intentional effort - I simply downloaded an extension which allows me to speed up the video in increments of 0.1x using a keyboard shortcut. Whenever it felt slow I would speed it up, and whenever it felt too fast I would slow it down. Without paying much attention to the actual numbers I had reached over 3x within a month or so.
tkfu 7y ago
I'm a bit confused, here. (I went and looked at the original paper.) They estimated information density for each of the subject languages as a whole, on average: > In parallel, from independently available written corpora in these languages, we estimated each language’s information density (ID) as the syllable conditional entropy to take word-internal syllable-bigram dependencies into account. But the experiment uses the same text translated into each language! Why introduce this extra variable (and source of error) of estimated language-wide information density, if you are controlling your experiment such that you have the exact same information encoded in each language? That is to say, why use an _estimated_ information density when you could measure it exactly for the texts that are being spoken? Or, conversely, why go to all the trouble of having the speakers read the same text translated into each language, if you aren't going to make use of that symmetry?
- rocqua 7y ago
  Good question! I suppose that the experiment wants to capture the actual 'information density' of the language, and hence looks at the full language. Then, they want to avoid any modification in speech rate due to the semantics of the spoken text. This does not make sense for a hypothesis where the actual bit-rate of speech tends towards 39 b/s. That is, when your text happens to convey more bits, you slow down. However, for an alternative hypothesis, this design does make sense. The idea here is that a language naturally converges to a speech-rate that gives 39 b/s. The idea here is that the actual speech-rate is much more constant, and just drops until it becomes too fast. For that, I'd argue you don't want the mean bit-rate but something like the 90th percentile bit-rate. Because it seems to me that speech-rate that is 'too fast' more than 10% of the time would not really be natural.
- canjobear 7y ago
  Information depends on probability. If something is very probable then it doesn’t have much information (because you already saw it coming). If something is improbable then it has a lot of information. In the paper they want to know how much information is in a syllable in context. To do that they need to know the probability of each syllable given the previous syllable. To estimate that probability distribution, you need to look at a lot of text, much more than just the passages that the authors used to measure speech rate.
holy_city 7y ago
I'm not sure what the review process is for the source, but the paper [1] is pretty interesting. Lots of cool findings/references in there like: - There is a measurable difference in information density based on the sex of the speaker - Syllables were chosen as the base unit of measurement because morphemes (words) are too big/linguistically varied and phonemes (sound equivalent of letters) are too small and likely to be dropped in regular speech. I'd like to see the same analysis using phonemes to see how it changes, especially between dialects. [1] https://advances.sciencemag.org/content/5/9/eaaw2594 https://advances.sciencemag.org/content/5/9/eaaw2594
JimBrimble35 7y ago
I would love to see the impact of acronyms on information density. It would also be interesting to see how many bits per second the average human maximum is.
- liability 7y ago
  Some people you might expect to have a very high bps, like auctioneers, are actually using a sort of parlor trick to create the illusion of rapid speech (a form of intention skillful stuttering which, when done right, is perceived as very fast speech.) I expect some people are capable of some small multiple of this average, but probably not anything seriously dramatic. (Of course there seems to be no lower bound, as in the case of involuntary stuttering.)
- dredmorbius 7y ago
  YMMV, TLAs increase s/n but leverage ROI on prefamiliarisation. Language is symbolic, all words are pointers. Whether you collapse complexity through an Apollonian use of religious icons or through initialisations and acronyms likely matters little. But: TANSTAAFL.
Wistar 7y ago
I wonder how much face-to-face communication adds to the effective transmission rate where the physical gestures, body language, winks, and other non-verbal communication are taken into account.
soniman 7y ago
39.15/64 = phi
- rocqua 7y ago
  and 64 seconds is relevant here because?
  - soniman 7y ago
    It's 64 bits. The rate of information is measured in bits, and so 64 was the closest binary (2^6) denominator to 39.15 bits/sec.
- saagarjha 7y ago
  Of course, Φ is irrational and the occurrence of golden ratio conjugate is almost certainly just a coincidence…
jgalt212 7y ago
The Micro Machines commercial guy was transmitting at at least 900 baud. https://www.youtube.com/watch?v=yDBvEo1s6A4 https://www.youtube.com/watch?v=yDBvEo1s6A4
- rzzzt 7y ago
  John Moschitta held the title of World's Fastest Talker at one point: http://nymag.com/speed/2016/12/is-the-micro-machines-guy-still-the-fastest-talking-man-on-the-planet.html http://nymag.com/speed/2016/12/is-the-micro-machines-guy-sti...
6gvONxR4sf7o 7y ago
From the paper [0]: >Abstract >Language is universal, but it has few indisputably universal characteristics, with cross-linguistic variation being the norm. For example, languages differ greatly in the number of syllables they allow, resulting in large variation in the Shannon information per syllable. Nevertheless, all natural languages allow their speakers to efficiently encode and transmit information. We show here, using quantitative methods on a large cross-linguistic corpus of 17 languages, that the coupling between language-level (information per syllable) and speaker-level (speech rate) properties results in languages encoding similar information rates (~39 bits/s) despite wide differences in each property individually: Languages are more similar in information rates than in Shannon information or speech rate. These findings highlight the intimate feedback loops between languages’ structural properties and their speakers’ neurocognition and biology under communicative pressures. Thus, language is the product of a multiscale communicative niche construction process at the intersection of biology, environment, and culture. [0] https://advances.sciencemag.org/content/5/9/eaaw2594 https://advances.sciencemag.org/content/5/9/eaaw2594
SXX 7y ago
So here is my own experience. I was avid audio book fan for last 3 years and while ago some guy on reddit told me about how he listen books on Audible using high-speed option like 2.x. I never tried that before last summer since at higher speed speech become incomprehensible for me. What this guy told me is that it's just take time to adjust to it. So I basically started to listen for books at slightly higher speed. Then I gradually increased it and in a few days I could handle 2.0x speed no problem while listening for really complex fantasy (Malazan Book of the Fallen [1]). After two weeks I could handle 2.5x without a problem. In the beginning it was harder to comprehend at high speed while walking or crossing the street since I lost attention, but in a few months I could do anything while listening without missing any information or emotions of narrator. To give an example of how far this can go. This spring I was listening for The Expanse audiobook [2] at 4.0x speed. With some effort I could go even faster for like 5.x in case of these particular books, but obviously can not keep up for long. I still usually listen books at 2.0-3.0x depend on narrator and quality of audio and this skill dont go away even if I have extended time between books like a month or so. [1] https://www.audible.com/pd/Reapers-Gale-Audiobook/B00M4LRBY6 https://www.audible.com/pd/Reapers-Gale-Audiobook/B00M4LRBY6 [2] https://www.audible.co.uk/pd/Abaddons-Gate-Audiobook/B00T6NZFWK https://www.audible.co.uk/pd/Abaddons-Gate-Audiobook/B00T6NZ... UPD: Edit. s/can keep up/can not keep up/
- bitwize 7y ago
  Blind people use screenreader software sped up so fast as to be indecipherable to untrained ears. The screenreader can give them near instantaneous feedback about where on the screen they are and what's there when it's so sped up, and with a bit of practice perceiving the sped-up speech imposes no burden at all.
  - SXX 7y ago
    I can tell from experience that when I lay down in my bed with my eyes closed I could comprehend speech at much higher speed than I would while walking on the street. No surprise blind people can handle it better even though I have no clue how exactly it work in relations to the brain. I was always curios to make actual research / paper on this kind of thing, but as non-scientist I simply have no time to do so. So I happy someone actually doing it.
adampk 7y ago
This reminds me of The User Illusion. The number "40 bits/s" is quoted as the processing rate of human consciousness. Maybe humans can only transmit information as fast as they can think of it? https://www.goodreads.com/book/show/106732.The_User_Illusion https://www.goodreads.com/book/show/106732.The_User_Illusion
hsnewman 7y ago
So what the title says is: It may or may not have that rate. Moving on, nothing to see here.
JackFr 7y ago
So basically human speech processing is a noisy channel which has limited bandwidth. If the language we encode in has a higher rate of compression, we need to transmit with fewer errors, but if it has a low rate of compression (or even redundancy) we can support a high rate of transmission and correct for more errors. Which is kind of neat. Thank you Claude Shannon!
impostir 7y ago
I would be interested if Rapid Serial Visual Presentation breaks this rule for any language. My guess is that it wouldn't, but I would be fascinated to find out. I would guess that languages with more general density of information per word would force the reader to slow their words per minute rate. Anecdotally, I can read with RSVP at around 620 words per minute in English and retain general comprehension. Sadly, I don't know another language well enough to compare that information with anything. https://en.m.wikipedia.org/wiki/Rapid_serial_visual_presentation https://en.m.wikipedia.org/wiki/Rapid_serial_visual_presenta...
n1231231231234 7y ago
This is really cool. I am working in a related area and I think most of us have assumed that on average, the information rate is 'about the same' for the languages across the world. So it's exciting to see that their results confirm this assumption. Two qualifying remarks. 1) The 'about the same' is important. Even in their data, there is still quite some variance. They found an average of 39bits, with a stdev of 5. That means that about 1/3 of the data falls outside of the range of 34-44bits. 2) Which brings me to the the uniform information density (UID) hypothesis. According to the UID, the language signal should be pretty smooth wrt how information is spread across it. For many years, the UID was thought to be pretty absolute: Even across a unit like a sentence, it was thought that information will spread pretty evenly. Now, there is an increasing amount of research that shows that esp. in spontaneous spoken language, there is a lot more variance within in the signal, with considerable peaks and troughs spread across longer sequences.
- godelski 7y ago
  Why did everyone assume it would be the same on average? This seems weird to me. Also, can you explain more about how the information density was calculated? Anything at the bit level seems crazy small to me. Words convey a lot of information. They cause your brain to create images, sounds, emotions, smells, etc. I guess we're calling language a compression of that? But even still, bits seems small.
  - n1231231231234 7y ago
    > Why did everyone assume it would be the same on average? This seems weird to me. (see edit below; but i leave this up; it might be interesting, also) you mean that even for smaller sequences, the UID holds, right? the assumption was that even for a single sentence, there are a lot of ways to reduce or increase information density so that you get a smoother signal. e.g.: "It is clear that we have to help them to move on.", you could contract it to "it's clear we gotta help them move on" and contract it even further in the actual speech signal ('help'em'). or you could stretch it: "it is clear to us that we definitely have to help them in some way to move on", or alike. the assumption was that such increases / decreases would even be done to 'iron out' the very local peaks and troughs, particularly in speech. bits: yeah, that took me a while to get used to, as well. the authors used (conditional) entropy as a way to measure information density (which is a good measure in this instance imv). and bits is just per definition the unit that comes out of information theoretical entropy: https://en.wikipedia.org/wiki/Entropy_(information_theory) https://en.wikipedia.org/wiki/Entropy_(information_theory) . btw: while technically possible, i don't think that the comparison in the summary article between 39 bits in language and a xy bit modem is a helpful comparison. bits in the context of entropy are all about occurence and expectation in a given context. bits of a modem/in CS, they represent a low level information content for which we do not check context and expectation. edit: ah, i realise you are asking why most in our community assumed that this universal rate applied across languages, right? i guess the intuition was that all of us humans, no matter what language we speak, use the speech signal to transmit and receive information and that all of us have the same cognitive abilities. so the rate at which we convey information should be about the same. sure, there are probably differences according to some factors (spoken vs written language, differences in knowledge between speakers, etc.). but when the only factor that differs is English vs Hausa, esp. in spontaneous spoken language, then the information rate should be about the same.
mattr47 7y ago
In my initial Army job I had to learn how to copy Morse code. I got up to a respectable 23 groups per minute (a group is 5 characters). If I remember right, beyond about 13-15 GPM, once the code stopped transmitting I would still write for another 20 seconds or so. It was not something I consciously did, it just backed up in my brain.
deleted 7y ago
[deleted]
nickjj 7y ago
I think it also really comes down to what videos or audio you listen to. In other words, the person delivering the audio and its quality. I listen to most non-entertainment videos / podcasts at 1.5x and I would say about 80% of them are completely ok to comprehend from a "can I easily listen to this without struggling to figure out what they are saying at a language level". But as soon as I try 2x, that drops down to maybe 30-50% because the person speaking isn't speaking clear enough, they have an accent that overpowers any type of comprehension ability on my part or the audio quality is too poor and it introduces too many artifacts. Sometimes I ask my friends to watch the same video at 2x to see if they can comprehend it and often times they can't (but sometimes they can). We're all in the same area. I generally find a neutral accent and very clear annunciation helps the most. I've had a bunch of people say they've watched my videos / courses at 2x without issues because apparently I have no accent which is something I've heard from a number of people in different countries where English isn't their native language. I find it interesting because I've also heard a decent amount of people say I speak very fast at 1.0x speed, so I do believe accents and annunciation has at least some role in this. Does anyone know of any software where you can feed it an English audio sample and it spits back the number of syllables per second? Seems like a pretty cool potential ML project.
- ckosidows 7y ago
  Where are you from if people think you don't have an accent, if you don't mind me asking?
  - nickjj 7y ago
    Long Island, New York. I'm not trying to plug my channel but here's my latest public video from the other week: https://www.youtube.com/watch?v=Kq_khHWovl4 https://www.youtube.com/watch?v=Kq_khHWovl4 I do believe audio quality plays a -huge- role in this. For comparison, here's the most recent Railscasts video from a few years ago by another screencast author: https://www.youtube.com/watch?v=urPi4qZJeOE https://www.youtube.com/watch?v=urPi4qZJeOE I can deal with him at 2x but it's mentally taxing because his audio has a metallic wispy sound at that speed and it makes his words sound blended together. I think he also talks slightly slower than me at 1.0x as well, so it's not just base talking speed. Does anyone else notice that metallic sound too? Here's another sample of Joe Rogan and Bill Burr on a podcast: https://www.youtube.com/watch?v=cS1KWv0das8 https://www.youtube.com/watch?v=cS1KWv0das8 Listening to them talk at 2x feels like a joy. They are talking a little slower since it's a casual conversation but the audio is crystal clear and both of them have very good word annunciation (not surprising since they are on stage talking for a living).
bscphil 7y ago
I took a peak at the paper and discovered the following: among the languages they studied, English was second only to French in the number of bits of information per second conveyed on average. (English also had significantly fewer syllables per second than French.) Thus, if you're choosing a language to communicate in on the basis of how fast it is to get an idea across, English and French are likely your best choices! (Among languages in the survey.)
bane 7y ago
I recall a moment when, as an adult, I thought it might be fun to watch some cartoons that were my favorite as a child. I actually ended up giving up fairly quickly because the rate of the speech in many cartoons is apparently geared to be very slow. I remember firing up the first episode of the classic Thundercats show and being really struck at how much slower the characters spoke than my memory of the show. On the flip side, I found noted radio show host Diane Rehm to be virtually unlistenable her rate of speech is sooooo incredibly slow. Her guests sound like they are all at 2x speed compared to her. I'm pretty sure that this number changes as we age and our processing faculties gear up and then down.
- stormbeard 7y ago
  Diane Rehm has spasmodic dysphonia, which affects the way she speaks. She's not just old.
  - bane 7y ago
    True, but spasmodic dysphonia doesn't cause slow speech [1]. It's the source of other speech issues she's worked hard to over come. Her pattern of speech is in general among the slowest I've ever heard -- sometimes approaching single digit words per minute. She's not always so slow, there's interstitials and other moments when her speaking is just kind of slow not unbearable. Despite my personal feelings, I think her pace of speaking is part of what her appeal was. After being assaulted by other media all day, her show can also be a very relaxing listen and was nearly always a very intelligent conversation. 1 - https://www.youtube.com/watch?v=SqzfsKMaLqk https://www.youtube.com/watch?v=SqzfsKMaLqk
ShinyObject 7y ago
The researchers obviously have to keep the scope narrow in order to get numbers at all. That said, we should be aware that a tech nerd audience will find simple answers to complex non-tech questions appealing, and we should not over-estimate our understanding here just because we have a number. There is a large amount of data transmitted through sub-communication and context, particularly during an in-person interaction, which is what people are wired for. Overall tone, body language, eye contact, and various social cues make up the bulk of data being transferred in many interactions. There's a reason why talking to some people feels exhausting and others invigorating, and it's not just the transcript.
- d-sc 7y ago
  What are the things that make the difference between invigorating and exhausting?
- knzhou 7y ago
  We can avoid reading too much into the study by just remembering the error bars. It's not like 39 is a universal constant. It's more like 39 with a standard deviation of 6. That's a wide spread, but it's less wide than the spread you get from syllable rate alone, and that's all the study quantitatively tells us.
deleted 7y ago
[deleted]
sasaf5 7y ago
Interesting, I did a similar analysis for music scores before and arrived at 30 bits per second.
mchinen 7y ago
Seems reasonable, from what is known about the capacity of speech, although extra cool if there is really some universal number we converge on. Kuyk and Kleijn found the upper bound of the speech channel to be 100 bits per second, in agreement with the lexical rate of about half that in "ON THE INFORMATION RATE OF SPEECH COMMUNICATION" (https://ieeexplore.ieee.org/document/7953233 https://ieeexplore.ieee.org/document/7953233) using an interesting method based on mutual information. Some highlights from the introduction that are relevant: Broadly speaking, two approaches to measuring the information rate of speech exist: the linguistic approach, and the acoustic approach. The linguistic approach describes speech as a sequence of discrete perceptual units such as phonemes, words, or sentences. Taking the average talking speed as 12 phonemes per second [3], and using the English phoneme probabilities tabulated in [4], the lexical information rate is approximately 50 b/s. When the dependencies of the phonemes are accounted for the rate will be decreased further. The lexical information rate does not include information about talker identification, emotional state, and prosody. However, these variables vary relatively slowly in time and contribute little to the overall information rate. As an example, [5] estimated that the total amount of talker-specific information (e.g., age, accent, sex) was of the order of 30 bits
iconjack 7y ago
I once saw research that said English speakers utter about 4 words per second. Shannon once empirically determined each English word carried about 6 bits of information. For what it's worth.
fargle 7y ago
Mm.Hmm... Not sure the researchers have yet encountered the Deep South (US). I estimate approximately 17 bits of information in "Mm.hmm..." based on the 128K possible meanings. It takes about 5-10 seconds for proper gestation, when uttered expertly during the course of a lively conversation. So more like approximately 1.7 to 3.4 bits/s.
- fargle 7y ago
  y'all
poormystic 7y ago
I wish I could provide references... I recall a tutor stating that human speech can be encoded at a very similar rate. This was in 1985 I think.
Merrill 7y ago
Typing at 60 words/minute, 5 characters/word and 8 bits/character gives a gross bit rate of 40 bits/second. Of course, the information rate is a lot less, since there are fewer than 8 bits of information per character in English. The paper says "from 4.8 bits per syllable for Basque to 8.0 bits per syllable for Vietnamese" and there are multiple characters per syllable. So the typing information bit rate is probably somewhere around 10 to 15 bits/second.
- aidenn0 7y ago
  IIRC 1.3 bits per character is the value used as a rule-of-thumb for english text entropy, which would yield 6.5bps at 60WPM and an average stenographer can go at 4x that rate for approximately 26bps.
drdeadringer 7y ago
This has at-least tangential interest to me as one of my side projects is to create a custom "Number Station". Maybe out of a Raspberry Pi or similar, I'm still very much in the preliminaries. Since I'm in the US I am aware of the FCC.
ineedasername 7y ago
I didn't know the but rate, but I learned this studying in computational linguistics. Languages that seem "fast" tend to have more phonemes per semantic unit than "slow" languages. In general, languages with a lower lexicon of phonemes require more of them to make a "word", they get spoken faster. So the semantic units per period of time remains fairly constant across languages.
joeyrosztoczy1 7y ago
I feel like there's an important follow up study in how much humans can compress data with respect to those bits (cat -> large -> mane ~= lion).
akjetma 7y ago
Scientists started with written texts from 17 languages, including English, Italian, Japanese, and Vietnamese. They calculated the information density of each language in bits—the same unit that describes how quickly your cellphone, laptop, or computer modem transmits information. They found that Japanese, which has only 643 syllables, had an information density of about 5 bits per syllable, whereas English, with its 6949 syllables, had a density of just over 7 bits per syllable. Vietnamese, with its complex system of six tones (each of which can further differentiate a syllable), topped the charts at 8 bits per syllable. how can you encode 643 syllables using 5 bits? same for 6949 syllabes/7 bits?
- geomark 7y ago
  If I understand this correctly, it isn't that they are uniquely encoding each syllable. It's that they are encoding the information in each syllable. Many syllables have very low information content and must be combined with other syllables to convey information. Many other syllables are redundant.
lenepp 7y ago
Apologies for being harsh, but this kind of thing is the phrenology of our time. I know it's utterly conventional to think this way about language in some circles that present themselves as doing legitimate science, but the view that you can calculate the amount of information in human speech, except in a super-technical sense that doesn't match any of the reporting on this study or the way people are interpreting it, has to be called out for the total nonsense that it is. It doesn't bear a moment's honest reflection. And yes, I know information theory. It's language that these folks - many of them prominent and celebrated within their utterly normalized professions, just like in the days of phrenology - are fundamentally mistaken about. What quantity of information do you think there is in the word "trump," for instance? Is it the same over time, to bring up just one feature of how this funny thing called context informs human speech? Wittgenstein's Philosophical Investigations is a good place to start if anyone's interested in understanding this issue.
- bagacrap 7y ago
  They aren't talking about the semantic information of the word "trump". They explain the methodology for calculating information, and it's per syllable (based on the number of distinct syllables that are part of the language's phonetics). So, for English speakers, 'trump' has exactly 7 bits in it. That exact syllable may or may not exist in another language, but if so the same singly syllabic word "trump" would have a different number of bits to a speaker of that language. Maybe next time RTA?
  - bagacrap 7y ago
    s/exactly 7/a little more than 7/
  - cortesoft 7y ago
    In other words, they aren't factoring in compression.
  - EpicEng 7y ago
    >Maybe next time RTA? I think it's you that has missed the point. Syllables have a very loose correlation to information. So great; we can stream out 39bits worth of syllables / second. In what way does that describe how information dense those syllables are? Context matters here.
alexandercrohde 7y ago
I don't get the methodology. If you have a short-story, and translate it into 10 languages (and translate very well), it should have the same amount of information in it in each translation, no? Therefore the transmission rate will simply be proportional to the time to read the story. This idea contradicts what their study found, no?
akjetma 7y ago
i wonder if there's a constant average reading speed (in terms of bits per second) across written languages. chinese is obviously more dense per symbol than english, but is each character processed more slowly as a result, preserving constant data rate across languages? also interesting to consider how much harder real time, verbal translation would be if data rates differed.
yarg 7y ago
I doubt that there's a universal rate - a universal mean (within the context of a shared language) makes more sense. Every conversation acts as its own handshaking algorithm from which context is derived, and contexts will vary greatly in terms of amount of language required to convey concepts. Jargon rich conversations between experts have the potential to transfer information at a rate far greater than average.
ta1234567890 7y ago
> After noting how long the speakers took to get through their readings, the researchers calculated an average speech rate per language, measured in syllables/second. > No matter how fast or slow, how simple or complex, each language gravitated toward an average rate of 39.15 bits per second So does this mean that we are "understanding" only those 39 bits of syllables per second, or more like we are using those 39 bits to index something like an internal address space? And if the latter is the case, how big would that address space be? It would also be cool to see this complemented with the data rate (bits/second) of emotion communicated per second and see if that increases the total effective rate of communication between people.
- skosch 7y ago
  The article explicitly concludes that the speaker is the bottleneck, not the listener. You can understand sped-up Youtube videos just fine.
  - ta1234567890 7y ago
    Yes. And they say that we can listen at a rate of up to 120% more, which means 46.8 bits/second of syllables. So then, does it mean we "understand" 46.8 bits of information, or that we are using those bit to address some other, maybe bigger/more complex or detailed, memory space?
beefman 7y ago
See also this story from 8 years ago, which was about a previous study by the same group https://news.ycombinator.com/item?id=2976044 https://news.ycombinator.com/item?id=2976044
Abishek_Muthian 7y ago
I bet, elon musk is going use this data during his next talk about neuralink on human brain is limited to 39bit/sec during output of speech and BCI would surpass this limitation by output to smartphone speakers directly.
anotheryou 7y ago
the "text" portion? or already with intonation etc?
metrxqin 7y ago
So speech is a very ineffective way to communicate?
codeulike 7y ago
Early on when Information Theory was emerging, there were attempts to measure the bandwidth of consciousness. They reckoned about 18 bits per second or less, which sounds very low. Tor Norretranders book, The User Illusion, mentions some of the research: W R Garner and Harold W Lake "The Amount of Information in Absolute Judgements" - Psychological Review 58 (1951) - they attempted to measure people's ability to distinguish stimuli (such as light and sound) in bits. Result: 2.2 to 3.2 bits per second. W E Hick "On the Rate of Gain of Information" - Quarterly Journal of Experimental Psychology 4 (1952) - this experiment measured how much information a person could pass on if they acted as a link in a communication channel. That is, faced with a series of flashing lights, subjects had to press the right keys. Result: 5.5 bits per second. Henry Quastler "Studies of Human Channel Capacity" - Information Theory, Proceedings of the Third London Symposium (1956). Measured how many bits of information are expressed by a pianist while pressing keys on a piano. Result: 25 bits per second. J R Pierce "Symbols, Signals and Noise" (Harper 1961) - used experiments involving letters and symbols. Result: 44 bits per second. Discussion of the research, Tor Norretranders book, and what the research may have missed here: http://memebake.blogspot.com/2008/08/straw-dogs-and-bandwidth-of.html http://memebake.blogspot.com/2008/08/straw-dogs-and-bandwidt...
fsiefken 7y ago
The constructed language Ithkuil is, among other things, designed to have a higher transmission rate. Here are documents and ideas about the upcoming new 2.0 language version: http://www.ithkuil.net/new_morpho-phonology_v_0_8.pdf http://www.ithkuil.net/new_morpho-phonology_v_0_8.pdf http://www.ithkuil.net/phonotaxis_v_0_4.pdf http://www.ithkuil.net/phonotaxis_v_0_4.pdf http://www.ithkuil.net/roots_v_0_1.pdf http://www.ithkuil.net/roots_v_0_1.pdf http://www.ithkuil.net/VxCs_Affixes_v_0_2.pdf http://www.ithkuil.net/VxCs_Affixes_v_0_2.pdf http://www.ithkuil.net/script_presentation_0_2.pdf http://www.ithkuil.net/script_presentation_0_2.pdf https://www.reddit.com/r/Ithkuil/comments/bvk1t5/tnil_numbering_system/ https://www.reddit.com/r/Ithkuil/comments/bvk1t5/tnil_number... https://www.reddit.com/r/Ithkuil/comments/29wzfq/interesting_video_touches_on_information_density/ https://www.reddit.com/r/Ithkuil/comments/29wzfq/interesting...
- kaoD 7y ago
  If your language is complex enough (and it's my understanding that Ithkuil is[0]) that the emitter takes more time to translate thoughts into it, the transmission rate is not going to raise regardless of information density (remember rate is over time). I guess if we consider non-real time communication, but in that case (e.g. in English, which is limited by the medium's rate) the reception rate is the main factor, which is probably not too far off the transmission rate. I'd say Ithkuil is designed for information density and my guess is its actual max rate is pretty similar to the submission's 39bps. [0] IIRC not even its creator is a fluent speaker.
windlessstorm 7y ago
But we convey a lot in a lot higher bandwidth, because we use parallel channels each with low bitrate limitations.
stackingup 7y ago
As an interesting side note, ham radio operators heavily use a digital mode called PSK31. It stands for "Phase Shift Keying, 31 Baud". As I understand it, the 31 bit/s transmission rate was chosen because it is close to the entropy that operators can generate by typing on their keyboards. PSK31 does not transmit 8 bit bytes, but instead uses what they call a Varicode, a kind of Fibonacci code. More frequent characters are encoded using fewer bits, thus the encoded bit rate is an approximation of the entropy in the text stream.
AtOmXpLuS 7y ago
Everything in AtOmXpLuS.CoM
bsza 7y ago
> Each participant read aloud 15 identical passages that had been translated into their mother tongue I doubt there is an objective way to ensure that no information is lost or smuggled in when you translate a text into another language. For example, English has more than 100 words for 'walk', whereas Toki Pona (a constructed language known for its extreme simplicity) has only one. But does 'stroll' encode the same amount of information as 'tawa'? Depends on what you want to use that information for, I guess. If you only want to know where I went this morning, they are equally good. If you want to know that my act of going there was a recreational activity, possibly part of my morning routine, they are not.
- TacticalTable 7y ago
  It's my interpretation that by deciding the bits of each word, they included subtext/classification that would differentiate 'walk' and 'stroll'.
Yuval_Halevi 7y ago
My girlfriend has a transmission rate of at least 78 bits
tripzilch 7y ago
Claude Shannon (of the information theory fame) did a similar research with his 1951 paper "Prediction and Entropy of Printed English". He used a particularly clever idea, leaving out words or letters from English text, he then measured how accurately people could predict the missing text. And then used those numbers for statistical information-theoretical analysis to estimate the information density at about 9-11 bits (IIRC) per letter.
- mizaru 7y ago
  Looked it up, 11.82 bits per word, 2.62 bits per letter.
  - tripzilch 7y ago
    Oh! Yes you're right. My mistake. That makes a lot more sense too, it would be weird if a letter was more bits than a byte, hm? :-)
auiya 7y ago
Are there weights applied for regional disparities? For instance sped up for American New Englanders and slowed down for American South Easterners?
cellular 7y ago
YouTube at 2x with captions can speed this to less than 78 seconds. "Less than " to account for advertising.
humble_engineer 7y ago
Curiously, this is also why memes are much better at conveying complicated political or sociological ideas faster and more efficiently than a few paragraphs on the topic.

671 points by headalgorithm 7y ago

>Scientists started with written texts from 17 languages, including English, Italian, Japanese, and Vietnamese. They calculated the information density of each language in bits—the same unit that describes how quickly your cellphone, laptop, or computer modem transmits information. I feel like the whole "bits" calculation is a neat way to get into the media, but not actually related to "information density". Edit: Been informed I'm deeply ignorant on Information Theory.

paggle 7y ago
It's hard to translate to bits, but there is definitely different information densities between language. E.g. when you write a paragraph in Spanish, the total number of keystrokes you'll need is higher than writing the same thing in English.
monocasa 7y ago
Bits is a base unit of information theory.
sansnomme 7y ago
Theoretically a summarisation algorithm (or neural network) should reduce sentences of similar content from different languages to similar sizes no?
furgooswft13 7y ago
Human speech uses the extremely dense lossy compression method called "A Lifetime of Experience and Biases"
- burnte 7y ago
  It's only lossy when the decoder's dictionary doesn't have a similar volume of data points to properly reinflate the data stream. Things that didn't make sense to me as a teen make far too much sense now because my experience can properly contextualize the old data.
  - furgooswft13 7y ago
    Or lossy when the decoder has a different volume of data points and extracts totally different information or even info that never existed in the first place. Kinda like those anime/retro game upscaling filters, except with way higher variance on what comes out between decoders, or the same decoder at different times. Gives me an idea for a new JPEG decoder with a floating ADC as input.
wahern 7y ago
Bit is the unit usually used for describing entropy, information density, and channel capacity, regardless of whether computer systems are involved. See https://pdfs.semanticscholar.org/d554/d933fd78e19d51d68ac4eabab6810cf0ba99.pdf https://pdfs.semanticscholar.org/d554/d933fd78e19d51d68ac4ea... and, more generally, https://en.wikipedia.org/wiki/Channel_capacity https://en.wikipedia.org/wiki/Channel_capacity and https://en.wikipedia.org/wiki/Information_theory https://en.wikipedia.org/wiki/Information_theory. The field of Information theory effectively began with Claude Shannon. The same formalisms he developed are used outside computer science--linguistics, physics, microbiology, etc.
TacticalTable 7y ago
This has actually been a fairly popular idea within linguistics for a long time, and is absolutely relevant to information density. Problems like these need to be better understood for natural language understanding and translation, and 'bits' provide a good baseline measurement.
rrss 7y ago
The authors use syllable conditional entropy to estimate information density, and entropy is measured in bits (or shannons, if you prefer).

jumelles 7y ago

Being able to convey emotion with tone in speech likely adds a lot more "information".

_nhynes 7y ago
The increased information density in tonal language appears to be offset by a reduced speaking rate [0]. [0] http://ohll.ish-lyon.cnrs.fr/fulltext/pellegrino/Pellegrino_to%20appear_Language.pdf http://ohll.ish-lyon.cnrs.fr/fulltext/pellegrino/Pellegrino_...
- pmontra 7y ago
  I think jumelles was referring to something like "don't do that" uttered as fun, angry, etc.
- SopaDeMacaco 7y ago
  That's not what he said, sperglord.
8bitsrule 7y ago
So very true. Said in the right tone, "I don't care" can mean -many- things. Emotional intelligence is acquired (or not). 'Bit rate' means nada. As well, the 'amount of information' conveyed depends on the environment, and the preparation of the speaker and the listener. Some speakers (not to mention any names) spew a lot of BS ( not information) to sort through. I'd argue that, in a medical environment, the word 'sponge' conveys less information than the word 'ebola'. For the prepared, the cascade of necessary reactions in the brain can take little or much time to process. Some authors/speakers (authoritative) can pack -a lot to think about- in a few words. Like 'Where is everybody?'

Mathnerd314 7y ago

> instead of being limited by how quickly we can process information by listening, we’re likely limited by how quickly we can gather our thoughts. That’s because, he says, the average person can listen to audio recordings sped up to about 120%—and still have no problems with comprehension. “It really seems that the bottleneck is in putting the ideas together.” Glad this paragraph was in the article, clears up their methodology. I wonder if it applies to writing too, or if skilled writers work faster.

Aperocky 7y ago
Really depends on language, if you're writing java, you'll be putting out a lot more than that due to how stupidly verbose it is.
- jefftk 7y ago
  > if you're writing java, you'll be putting out a lot more than that due to how stupidly verbose it is Being "verbose" means that each letter you type communicates fewer bits of information. If the bottleneck is putting ideas together then you would expect someone writing in a more verbose language to type more letters per minute but still take a similar amount of time to communicate the idea. In practice most Java programmers are using IDEs with good auto-completion, though, so aren't actually needing to type as many letters as you'd think.
  - dual_basis 7y ago
    What might be more relevant, however, is the effect of verbosity on comprehension. Syntax highlighting can help, and humans will likely naturally perform input compression (chunking), but this must be learned.
  - the_af 7y ago
    Like the sibling comment mentioned, it seems verbosity hinders reading comprehension rather than writing. Many IDEs understand this and hide some of the boilerplate. This raises the question: if the IDE autocompletes the boilerplate for you, and also hides it, why is it needed in the first place?

jefftk 7y ago

Here's the actual paper: https://advances.sciencemag.org/content/5/9/eaaw2594 https://advances.sciencemag.org/content/5/9/eaaw2594

ilamont 7y ago

But, he says, instead of being limited by how quickly we can process information by listening, we’re likely limited by how quickly we can gather our thoughts. That’s because, he says, the average person can listen to audio recordings sped up to about 120%—and still have no problems with comprehension. Some years ago I worked on an accessibility project for an app and website designed for people with disabilities. One of the team members had low vision, and used a screen reader that must have been set to 3x or even higher. I usually listen to YouTube and podcasts at 1.5-2x and I could barely understand the audio. He seemed surprised, which indicated to me that 3x+ was the norm for people in his circle. I wonder if his ability was trained through years of using fast screen readers, vs. a lower visual processing load leads to better audio processing, or some other explanation.

vlovich123 7y ago
Yes. Took me a while but I can comfortably understand 2x speed and now 1x podcasts seem weird like they're talking super slow. I would imagine it's something you just train even more out of necessity.
liability 7y ago
I typically use TTS near 2.5x (I turn it up when I'm alert and down when I'm tired.) It's definitely a learned skill; a few years ago I started at 1x and struggled even with that. Every couple of months, take a moment to reflect on your comprehension. Is it currently easy for you to understand the audio? If yes, then crank it up a little bit until it's noticeably more difficult. Repeat this process periodically over a year or so and before you know it, it'll be set pretty damn quick.
dual_basis 7y ago
I can watch YouTube content at 3x without issue. I did this without much intentional effort - I simply downloaded an extension which allows me to speed up the video in increments of 0.1x using a keyboard shortcut. Whenever it felt slow I would speed it up, and whenever it felt too fast I would slow it down. Without paying much attention to the actual numbers I had reached over 3x within a month or so.

tkfu 7y ago

I'm a bit confused, here. (I went and looked at the original paper.) They estimated information density for each of the subject languages as a whole, on average: > In parallel, from independently available written corpora in these languages, we estimated each language’s information density (ID) as the syllable conditional entropy to take word-internal syllable-bigram dependencies into account. But the experiment uses the same text translated into each language! Why introduce this extra variable (and source of error) of estimated language-wide information density, if you are controlling your experiment such that you have the exact same information encoded in each language? That is to say, why use an _estimated_ information density when you could measure it exactly for the texts that are being spoken? Or, conversely, why go to all the trouble of having the speakers read the same text translated into each language, if you aren't going to make use of that symmetry?

rocqua 7y ago
Good question! I suppose that the experiment wants to capture the actual 'information density' of the language, and hence looks at the full language. Then, they want to avoid any modification in speech rate due to the semantics of the spoken text. This does not make sense for a hypothesis where the actual bit-rate of speech tends towards 39 b/s. That is, when your text happens to convey more bits, you slow down. However, for an alternative hypothesis, this design does make sense. The idea here is that a language naturally converges to a speech-rate that gives 39 b/s. The idea here is that the actual speech-rate is much more constant, and just drops until it becomes too fast. For that, I'd argue you don't want the mean bit-rate but something like the 90th percentile bit-rate. Because it seems to me that speech-rate that is 'too fast' more than 10% of the time would not really be natural.
canjobear 7y ago
Information depends on probability. If something is very probable then it doesn’t have much information (because you already saw it coming). If something is improbable then it has a lot of information. In the paper they want to know how much information is in a syllable in context. To do that they need to know the probability of each syllable given the previous syllable. To estimate that probability distribution, you need to look at a lot of text, much more than just the passages that the authors used to measure speech rate.

holy_city 7y ago

I'm not sure what the review process is for the source, but the paper [1] is pretty interesting. Lots of cool findings/references in there like: - There is a measurable difference in information density based on the sex of the speaker - Syllables were chosen as the base unit of measurement because morphemes (words) are too big/linguistically varied and phonemes (sound equivalent of letters) are too small and likely to be dropped in regular speech. I'd like to see the same analysis using phonemes to see how it changes, especially between dialects. [1] https://advances.sciencemag.org/content/5/9/eaaw2594 https://advances.sciencemag.org/content/5/9/eaaw2594

JimBrimble35 7y ago

I would love to see the impact of acronyms on information density. It would also be interesting to see how many bits per second the average human maximum is.

liability 7y ago
Some people you might expect to have a very high bps, like auctioneers, are actually using a sort of parlor trick to create the illusion of rapid speech (a form of intention skillful stuttering which, when done right, is perceived as very fast speech.) I expect some people are capable of some small multiple of this average, but probably not anything seriously dramatic. (Of course there seems to be no lower bound, as in the case of involuntary stuttering.)
dredmorbius 7y ago
YMMV, TLAs increase s/n but leverage ROI on prefamiliarisation. Language is symbolic, all words are pointers. Whether you collapse complexity through an Apollonian use of religious icons or through initialisations and acronyms likely matters little. But: TANSTAAFL.

Wistar 7y ago

I wonder how much face-to-face communication adds to the effective transmission rate where the physical gestures, body language, winks, and other non-verbal communication are taken into account.

soniman 7y ago

39.15/64 = phi

rocqua 7y ago
and 64 seconds is relevant here because?
- soniman 7y ago
  It's 64 bits. The rate of information is measured in bits, and so 64 was the closest binary (2^6) denominator to 39.15 bits/sec.
saagarjha 7y ago
Of course, Φ is irrational and the occurrence of golden ratio conjugate is almost certainly just a coincidence…

jgalt212 7y ago

The Micro Machines commercial guy was transmitting at at least 900 baud. https://www.youtube.com/watch?v=yDBvEo1s6A4 https://www.youtube.com/watch?v=yDBvEo1s6A4

rzzzt 7y ago
John Moschitta held the title of World's Fastest Talker at one point: http://nymag.com/speed/2016/12/is-the-micro-machines-guy-still-the-fastest-talking-man-on-the-planet.html http://nymag.com/speed/2016/12/is-the-micro-machines-guy-sti...

6gvONxR4sf7o 7y ago

From the paper [0]: >Abstract >Language is universal, but it has few indisputably universal characteristics, with cross-linguistic variation being the norm. For example, languages differ greatly in the number of syllables they allow, resulting in large variation in the Shannon information per syllable. Nevertheless, all natural languages allow their speakers to efficiently encode and transmit information. We show here, using quantitative methods on a large cross-linguistic corpus of 17 languages, that the coupling between language-level (information per syllable) and speaker-level (speech rate) properties results in languages encoding similar information rates (~39 bits/s) despite wide differences in each property individually: Languages are more similar in information rates than in Shannon information or speech rate. These findings highlight the intimate feedback loops between languages’ structural properties and their speakers’ neurocognition and biology under communicative pressures. Thus, language is the product of a multiscale communicative niche construction process at the intersection of biology, environment, and culture. [0] https://advances.sciencemag.org/content/5/9/eaaw2594 https://advances.sciencemag.org/content/5/9/eaaw2594

SXX 7y ago

So here is my own experience. I was avid audio book fan for last 3 years and while ago some guy on reddit told me about how he listen books on Audible using high-speed option like 2.x. I never tried that before last summer since at higher speed speech become incomprehensible for me. What this guy told me is that it's just take time to adjust to it. So I basically started to listen for books at slightly higher speed. Then I gradually increased it and in a few days I could handle 2.0x speed no problem while listening for really complex fantasy (Malazan Book of the Fallen [1]). After two weeks I could handle 2.5x without a problem. In the beginning it was harder to comprehend at high speed while walking or crossing the street since I lost attention, but in a few months I could do anything while listening without missing any information or emotions of narrator. To give an example of how far this can go. This spring I was listening for The Expanse audiobook [2] at 4.0x speed. With some effort I could go even faster for like 5.x in case of these particular books, but obviously can not keep up for long. I still usually listen books at 2.0-3.0x depend on narrator and quality of audio and this skill dont go away even if I have extended time between books like a month or so. [1] https://www.audible.com/pd/Reapers-Gale-Audiobook/B00M4LRBY6 https://www.audible.com/pd/Reapers-Gale-Audiobook/B00M4LRBY6 [2] https://www.audible.co.uk/pd/Abaddons-Gate-Audiobook/B00T6NZFWK https://www.audible.co.uk/pd/Abaddons-Gate-Audiobook/B00T6NZ... UPD: Edit. s/can keep up/can not keep up/

bitwize 7y ago
Blind people use screenreader software sped up so fast as to be indecipherable to untrained ears. The screenreader can give them near instantaneous feedback about where on the screen they are and what's there when it's so sped up, and with a bit of practice perceiving the sped-up speech imposes no burden at all.
- SXX 7y ago
  I can tell from experience that when I lay down in my bed with my eyes closed I could comprehend speech at much higher speed than I would while walking on the street. No surprise blind people can handle it better even though I have no clue how exactly it work in relations to the brain. I was always curios to make actual research / paper on this kind of thing, but as non-scientist I simply have no time to do so. So I happy someone actually doing it.

adampk 7y ago

This reminds me of The User Illusion. The number "40 bits/s" is quoted as the processing rate of human consciousness. Maybe humans can only transmit information as fast as they can think of it? https://www.goodreads.com/book/show/106732.The_User_Illusion https://www.goodreads.com/book/show/106732.The_User_Illusion

hsnewman 7y ago

So what the title says is: It may or may not have that rate. Moving on, nothing to see here.

JackFr 7y ago

So basically human speech processing is a noisy channel which has limited bandwidth. If the language we encode in has a higher rate of compression, we need to transmit with fewer errors, but if it has a low rate of compression (or even redundancy) we can support a high rate of transmission and correct for more errors. Which is kind of neat. Thank you Claude Shannon!

impostir 7y ago

I would be interested if Rapid Serial Visual Presentation breaks this rule for any language. My guess is that it wouldn't, but I would be fascinated to find out. I would guess that languages with more general density of information per word would force the reader to slow their words per minute rate. Anecdotally, I can read with RSVP at around 620 words per minute in English and retain general comprehension. Sadly, I don't know another language well enough to compare that information with anything. https://en.m.wikipedia.org/wiki/Rapid_serial_visual_presentation https://en.m.wikipedia.org/wiki/Rapid_serial_visual_presenta...

n1231231231234 7y ago

This is really cool. I am working in a related area and I think most of us have assumed that on average, the information rate is 'about the same' for the languages across the world. So it's exciting to see that their results confirm this assumption. Two qualifying remarks. 1) The 'about the same' is important. Even in their data, there is still quite some variance. They found an average of 39bits, with a stdev of 5. That means that about 1/3 of the data falls outside of the range of 34-44bits. 2) Which brings me to the the uniform information density (UID) hypothesis. According to the UID, the language signal should be pretty smooth wrt how information is spread across it. For many years, the UID was thought to be pretty absolute: Even across a unit like a sentence, it was thought that information will spread pretty evenly. Now, there is an increasing amount of research that shows that esp. in spontaneous spoken language, there is a lot more variance within in the signal, with considerable peaks and troughs spread across longer sequences.

godelski 7y ago
Why did everyone assume it would be the same on average? This seems weird to me. Also, can you explain more about how the information density was calculated? Anything at the bit level seems crazy small to me. Words convey a lot of information. They cause your brain to create images, sounds, emotions, smells, etc. I guess we're calling language a compression of that? But even still, bits seems small.
- n1231231231234 7y ago
  > Why did everyone assume it would be the same on average? This seems weird to me. (see edit below; but i leave this up; it might be interesting, also) you mean that even for smaller sequences, the UID holds, right? the assumption was that even for a single sentence, there are a lot of ways to reduce or increase information density so that you get a smoother signal. e.g.: "It is clear that we have to help them to move on.", you could contract it to "it's clear we gotta help them move on" and contract it even further in the actual speech signal ('help'em'). or you could stretch it: "it is clear to us that we definitely have to help them in some way to move on", or alike. the assumption was that such increases / decreases would even be done to 'iron out' the very local peaks and troughs, particularly in speech. bits: yeah, that took me a while to get used to, as well. the authors used (conditional) entropy as a way to measure information density (which is a good measure in this instance imv). and bits is just per definition the unit that comes out of information theoretical entropy: https://en.wikipedia.org/wiki/Entropy_(information_theory) https://en.wikipedia.org/wiki/Entropy_(information_theory) . btw: while technically possible, i don't think that the comparison in the summary article between 39 bits in language and a xy bit modem is a helpful comparison. bits in the context of entropy are all about occurence and expectation in a given context. bits of a modem/in CS, they represent a low level information content for which we do not check context and expectation. edit: ah, i realise you are asking why most in our community assumed that this universal rate applied across languages, right? i guess the intuition was that all of us humans, no matter what language we speak, use the speech signal to transmit and receive information and that all of us have the same cognitive abilities. so the rate at which we convey information should be about the same. sure, there are probably differences according to some factors (spoken vs written language, differences in knowledge between speakers, etc.). but when the only factor that differs is English vs Hausa, esp. in spontaneous spoken language, then the information rate should be about the same.

mattr47 7y ago

In my initial Army job I had to learn how to copy Morse code. I got up to a respectable 23 groups per minute (a group is 5 characters). If I remember right, beyond about 13-15 GPM, once the code stopped transmitting I would still write for another 20 seconds or so. It was not something I consciously did, it just backed up in my brain.

deleted 7y ago

[deleted]

nickjj 7y ago

I think it also really comes down to what videos or audio you listen to. In other words, the person delivering the audio and its quality. I listen to most non-entertainment videos / podcasts at 1.5x and I would say about 80% of them are completely ok to comprehend from a "can I easily listen to this without struggling to figure out what they are saying at a language level". But as soon as I try 2x, that drops down to maybe 30-50% because the person speaking isn't speaking clear enough, they have an accent that overpowers any type of comprehension ability on my part or the audio quality is too poor and it introduces too many artifacts. Sometimes I ask my friends to watch the same video at 2x to see if they can comprehend it and often times they can't (but sometimes they can). We're all in the same area. I generally find a neutral accent and very clear annunciation helps the most. I've had a bunch of people say they've watched my videos / courses at 2x without issues because apparently I have no accent which is something I've heard from a number of people in different countries where English isn't their native language. I find it interesting because I've also heard a decent amount of people say I speak very fast at 1.0x speed, so I do believe accents and annunciation has at least some role in this. Does anyone know of any software where you can feed it an English audio sample and it spits back the number of syllables per second? Seems like a pretty cool potential ML project.

ckosidows 7y ago
Where are you from if people think you don't have an accent, if you don't mind me asking?
- nickjj 7y ago
  Long Island, New York. I'm not trying to plug my channel but here's my latest public video from the other week: https://www.youtube.com/watch?v=Kq_khHWovl4 https://www.youtube.com/watch?v=Kq_khHWovl4 I do believe audio quality plays a -huge- role in this. For comparison, here's the most recent Railscasts video from a few years ago by another screencast author: https://www.youtube.com/watch?v=urPi4qZJeOE https://www.youtube.com/watch?v=urPi4qZJeOE I can deal with him at 2x but it's mentally taxing because his audio has a metallic wispy sound at that speed and it makes his words sound blended together. I think he also talks slightly slower than me at 1.0x as well, so it's not just base talking speed. Does anyone else notice that metallic sound too? Here's another sample of Joe Rogan and Bill Burr on a podcast: https://www.youtube.com/watch?v=cS1KWv0das8 https://www.youtube.com/watch?v=cS1KWv0das8 Listening to them talk at 2x feels like a joy. They are talking a little slower since it's a casual conversation but the audio is crystal clear and both of them have very good word annunciation (not surprising since they are on stage talking for a living).

bscphil 7y ago

I took a peak at the paper and discovered the following: among the languages they studied, English was second only to French in the number of bits of information per second conveyed on average. (English also had significantly fewer syllables per second than French.) Thus, if you're choosing a language to communicate in on the basis of how fast it is to get an idea across, English and French are likely your best choices! (Among languages in the survey.)

bane 7y ago

I recall a moment when, as an adult, I thought it might be fun to watch some cartoons that were my favorite as a child. I actually ended up giving up fairly quickly because the rate of the speech in many cartoons is apparently geared to be very slow. I remember firing up the first episode of the classic Thundercats show and being really struck at how much slower the characters spoke than my memory of the show. On the flip side, I found noted radio show host Diane Rehm to be virtually unlistenable her rate of speech is sooooo incredibly slow. Her guests sound like they are all at 2x speed compared to her. I'm pretty sure that this number changes as we age and our processing faculties gear up and then down.

stormbeard 7y ago
Diane Rehm has spasmodic dysphonia, which affects the way she speaks. She's not just old.
- bane 7y ago
  True, but spasmodic dysphonia doesn't cause slow speech [1]. It's the source of other speech issues she's worked hard to over come. Her pattern of speech is in general among the slowest I've ever heard -- sometimes approaching single digit words per minute. She's not always so slow, there's interstitials and other moments when her speaking is just kind of slow not unbearable. Despite my personal feelings, I think her pace of speaking is part of what her appeal was. After being assaulted by other media all day, her show can also be a very relaxing listen and was nearly always a very intelligent conversation. 1 - https://www.youtube.com/watch?v=SqzfsKMaLqk https://www.youtube.com/watch?v=SqzfsKMaLqk

ShinyObject 7y ago

The researchers obviously have to keep the scope narrow in order to get numbers at all. That said, we should be aware that a tech nerd audience will find simple answers to complex non-tech questions appealing, and we should not over-estimate our understanding here just because we have a number. There is a large amount of data transmitted through sub-communication and context, particularly during an in-person interaction, which is what people are wired for. Overall tone, body language, eye contact, and various social cues make up the bulk of data being transferred in many interactions. There's a reason why talking to some people feels exhausting and others invigorating, and it's not just the transcript.

d-sc 7y ago
What are the things that make the difference between invigorating and exhausting?
knzhou 7y ago
We can avoid reading too much into the study by just remembering the error bars. It's not like 39 is a universal constant. It's more like 39 with a standard deviation of 6. That's a wide spread, but it's less wide than the spread you get from syllable rate alone, and that's all the study quantitatively tells us.

deleted 7y ago

[deleted]

sasaf5 7y ago

Interesting, I did a similar analysis for music scores before and arrived at 30 bits per second.

mchinen 7y ago

Seems reasonable, from what is known about the capacity of speech, although extra cool if there is really some universal number we converge on. Kuyk and Kleijn found the upper bound of the speech channel to be 100 bits per second, in agreement with the lexical rate of about half that in "ON THE INFORMATION RATE OF SPEECH COMMUNICATION" (https://ieeexplore.ieee.org/document/7953233 https://ieeexplore.ieee.org/document/7953233) using an interesting method based on mutual information. Some highlights from the introduction that are relevant: Broadly speaking, two approaches to measuring the information rate of speech exist: the linguistic approach, and the acoustic approach. The linguistic approach describes speech as a sequence of discrete perceptual units such as phonemes, words, or sentences. Taking the average talking speed as 12 phonemes per second [3], and using the English phoneme probabilities tabulated in [4], the lexical information rate is approximately 50 b/s. When the dependencies of the phonemes are accounted for the rate will be decreased further. The lexical information rate does not include information about talker identification, emotional state, and prosody. However, these variables vary relatively slowly in time and contribute little to the overall information rate. As an example, [5] estimated that the total amount of talker-specific information (e.g., age, accent, sex) was of the order of 30 bits

iconjack 7y ago

I once saw research that said English speakers utter about 4 words per second. Shannon once empirically determined each English word carried about 6 bits of information. For what it's worth.

fargle 7y ago

Mm.Hmm... Not sure the researchers have yet encountered the Deep South (US). I estimate approximately 17 bits of information in "Mm.hmm..." based on the 128K possible meanings. It takes about 5-10 seconds for proper gestation, when uttered expertly during the course of a lively conversation. So more like approximately 1.7 to 3.4 bits/s.

fargle 7y ago
y'all

poormystic 7y ago

I wish I could provide references... I recall a tutor stating that human speech can be encoded at a very similar rate. This was in 1985 I think.

Merrill 7y ago

Typing at 60 words/minute, 5 characters/word and 8 bits/character gives a gross bit rate of 40 bits/second. Of course, the information rate is a lot less, since there are fewer than 8 bits of information per character in English. The paper says "from 4.8 bits per syllable for Basque to 8.0 bits per syllable for Vietnamese" and there are multiple characters per syllable. So the typing information bit rate is probably somewhere around 10 to 15 bits/second.

aidenn0 7y ago
IIRC 1.3 bits per character is the value used as a rule-of-thumb for english text entropy, which would yield 6.5bps at 60WPM and an average stenographer can go at 4x that rate for approximately 26bps.

drdeadringer 7y ago

This has at-least tangential interest to me as one of my side projects is to create a custom "Number Station". Maybe out of a Raspberry Pi or similar, I'm still very much in the preliminaries. Since I'm in the US I am aware of the FCC.

ineedasername 7y ago

I didn't know the but rate, but I learned this studying in computational linguistics. Languages that seem "fast" tend to have more phonemes per semantic unit than "slow" languages. In general, languages with a lower lexicon of phonemes require more of them to make a "word", they get spoken faster. So the semantic units per period of time remains fairly constant across languages.

joeyrosztoczy1 7y ago

I feel like there's an important follow up study in how much humans can compress data with respect to those bits (cat -> large -> mane ~= lion).

akjetma 7y ago

Scientists started with written texts from 17 languages, including English, Italian, Japanese, and Vietnamese. They calculated the information density of each language in bits—the same unit that describes how quickly your cellphone, laptop, or computer modem transmits information. They found that Japanese, which has only 643 syllables, had an information density of about 5 bits per syllable, whereas English, with its 6949 syllables, had a density of just over 7 bits per syllable. Vietnamese, with its complex system of six tones (each of which can further differentiate a syllable), topped the charts at 8 bits per syllable. how can you encode 643 syllables using 5 bits? same for 6949 syllabes/7 bits?

geomark 7y ago
If I understand this correctly, it isn't that they are uniquely encoding each syllable. It's that they are encoding the information in each syllable. Many syllables have very low information content and must be combined with other syllables to convey information. Many other syllables are redundant.

lenepp 7y ago

Apologies for being harsh, but this kind of thing is the phrenology of our time. I know it's utterly conventional to think this way about language in some circles that present themselves as doing legitimate science, but the view that you can calculate the amount of information in human speech, except in a super-technical sense that doesn't match any of the reporting on this study or the way people are interpreting it, has to be called out for the total nonsense that it is. It doesn't bear a moment's honest reflection. And yes, I know information theory. It's language that these folks - many of them prominent and celebrated within their utterly normalized professions, just like in the days of phrenology - are fundamentally mistaken about. What quantity of information do you think there is in the word "trump," for instance? Is it the same over time, to bring up just one feature of how this funny thing called context informs human speech? Wittgenstein's Philosophical Investigations is a good place to start if anyone's interested in understanding this issue.

bagacrap 7y ago
They aren't talking about the semantic information of the word "trump". They explain the methodology for calculating information, and it's per syllable (based on the number of distinct syllables that are part of the language's phonetics). So, for English speakers, 'trump' has exactly 7 bits in it. That exact syllable may or may not exist in another language, but if so the same singly syllabic word "trump" would have a different number of bits to a speaker of that language. Maybe next time RTA?
- bagacrap 7y ago
  s/exactly 7/a little more than 7/
- cortesoft 7y ago
  In other words, they aren't factoring in compression.
- EpicEng 7y ago
  >Maybe next time RTA? I think it's you that has missed the point. Syllables have a very loose correlation to information. So great; we can stream out 39bits worth of syllables / second. In what way does that describe how information dense those syllables are? Context matters here.

alexandercrohde 7y ago

I don't get the methodology. If you have a short-story, and translate it into 10 languages (and translate very well), it should have the same amount of information in it in each translation, no? Therefore the transmission rate will simply be proportional to the time to read the story. This idea contradicts what their study found, no?

akjetma 7y ago

i wonder if there's a constant average reading speed (in terms of bits per second) across written languages. chinese is obviously more dense per symbol than english, but is each character processed more slowly as a result, preserving constant data rate across languages? also interesting to consider how much harder real time, verbal translation would be if data rates differed.

yarg 7y ago

I doubt that there's a universal rate - a universal mean (within the context of a shared language) makes more sense. Every conversation acts as its own handshaking algorithm from which context is derived, and contexts will vary greatly in terms of amount of language required to convey concepts. Jargon rich conversations between experts have the potential to transfer information at a rate far greater than average.

ta1234567890 7y ago

> After noting how long the speakers took to get through their readings, the researchers calculated an average speech rate per language, measured in syllables/second. > No matter how fast or slow, how simple or complex, each language gravitated toward an average rate of 39.15 bits per second So does this mean that we are "understanding" only those 39 bits of syllables per second, or more like we are using those 39 bits to index something like an internal address space? And if the latter is the case, how big would that address space be? It would also be cool to see this complemented with the data rate (bits/second) of emotion communicated per second and see if that increases the total effective rate of communication between people.

skosch 7y ago
The article explicitly concludes that the speaker is the bottleneck, not the listener. You can understand sped-up Youtube videos just fine.
- ta1234567890 7y ago
  Yes. And they say that we can listen at a rate of up to 120% more, which means 46.8 bits/second of syllables. So then, does it mean we "understand" 46.8 bits of information, or that we are using those bit to address some other, maybe bigger/more complex or detailed, memory space?

beefman 7y ago

See also this story from 8 years ago, which was about a previous study by the same group https://news.ycombinator.com/item?id=2976044 https://news.ycombinator.com/item?id=2976044

Abishek_Muthian 7y ago

I bet, elon musk is going use this data during his next talk about neuralink on human brain is limited to 39bit/sec during output of speech and BCI would surpass this limitation by output to smartphone speakers directly.

anotheryou 7y ago

the "text" portion? or already with intonation etc?

metrxqin 7y ago

So speech is a very ineffective way to communicate?

codeulike 7y ago

Early on when Information Theory was emerging, there were attempts to measure the bandwidth of consciousness. They reckoned about 18 bits per second or less, which sounds very low. Tor Norretranders book, The User Illusion, mentions some of the research: W R Garner and Harold W Lake "The Amount of Information in Absolute Judgements" - Psychological Review 58 (1951) - they attempted to measure people's ability to distinguish stimuli (such as light and sound) in bits. Result: 2.2 to 3.2 bits per second. W E Hick "On the Rate of Gain of Information" - Quarterly Journal of Experimental Psychology 4 (1952) - this experiment measured how much information a person could pass on if they acted as a link in a communication channel. That is, faced with a series of flashing lights, subjects had to press the right keys. Result: 5.5 bits per second. Henry Quastler "Studies of Human Channel Capacity" - Information Theory, Proceedings of the Third London Symposium (1956). Measured how many bits of information are expressed by a pianist while pressing keys on a piano. Result: 25 bits per second. J R Pierce "Symbols, Signals and Noise" (Harper 1961) - used experiments involving letters and symbols. Result: 44 bits per second. Discussion of the research, Tor Norretranders book, and what the research may have missed here: http://memebake.blogspot.com/2008/08/straw-dogs-and-bandwidth-of.html http://memebake.blogspot.com/2008/08/straw-dogs-and-bandwidt...

fsiefken 7y ago

The constructed language Ithkuil is, among other things, designed to have a higher transmission rate. Here are documents and ideas about the upcoming new 2.0 language version: http://www.ithkuil.net/new_morpho-phonology_v_0_8.pdf http://www.ithkuil.net/new_morpho-phonology_v_0_8.pdf http://www.ithkuil.net/phonotaxis_v_0_4.pdf http://www.ithkuil.net/phonotaxis_v_0_4.pdf http://www.ithkuil.net/roots_v_0_1.pdf http://www.ithkuil.net/roots_v_0_1.pdf http://www.ithkuil.net/VxCs_Affixes_v_0_2.pdf http://www.ithkuil.net/VxCs_Affixes_v_0_2.pdf http://www.ithkuil.net/script_presentation_0_2.pdf http://www.ithkuil.net/script_presentation_0_2.pdf https://www.reddit.com/r/Ithkuil/comments/bvk1t5/tnil_numbering_system/ https://www.reddit.com/r/Ithkuil/comments/bvk1t5/tnil_number... https://www.reddit.com/r/Ithkuil/comments/29wzfq/interesting_video_touches_on_information_density/ https://www.reddit.com/r/Ithkuil/comments/29wzfq/interesting...

kaoD 7y ago
If your language is complex enough (and it's my understanding that Ithkuil is[0]) that the emitter takes more time to translate thoughts into it, the transmission rate is not going to raise regardless of information density (remember rate is over time). I guess if we consider non-real time communication, but in that case (e.g. in English, which is limited by the medium's rate) the reception rate is the main factor, which is probably not too far off the transmission rate. I'd say Ithkuil is designed for information density and my guess is its actual max rate is pretty similar to the submission's 39bps. [0] IIRC not even its creator is a fluent speaker.

windlessstorm 7y ago

But we convey a lot in a lot higher bandwidth, because we use parallel channels each with low bitrate limitations.

stackingup 7y ago

As an interesting side note, ham radio operators heavily use a digital mode called PSK31. It stands for "Phase Shift Keying, 31 Baud". As I understand it, the 31 bit/s transmission rate was chosen because it is close to the entropy that operators can generate by typing on their keyboards. PSK31 does not transmit 8 bit bytes, but instead uses what they call a Varicode, a kind of Fibonacci code. More frequent characters are encoded using fewer bits, thus the encoded bit rate is an approximation of the entropy in the text stream.

AtOmXpLuS 7y ago

Everything in AtOmXpLuS.CoM

bsza 7y ago

> Each participant read aloud 15 identical passages that had been translated into their mother tongue I doubt there is an objective way to ensure that no information is lost or smuggled in when you translate a text into another language. For example, English has more than 100 words for 'walk', whereas Toki Pona (a constructed language known for its extreme simplicity) has only one. But does 'stroll' encode the same amount of information as 'tawa'? Depends on what you want to use that information for, I guess. If you only want to know where I went this morning, they are equally good. If you want to know that my act of going there was a recreational activity, possibly part of my morning routine, they are not.

TacticalTable 7y ago
It's my interpretation that by deciding the bits of each word, they included subtext/classification that would differentiate 'walk' and 'stroll'.

Yuval_Halevi 7y ago

My girlfriend has a transmission rate of at least 78 bits

tripzilch 7y ago

Claude Shannon (of the information theory fame) did a similar research with his 1951 paper "Prediction and Entropy of Printed English". He used a particularly clever idea, leaving out words or letters from English text, he then measured how accurately people could predict the missing text. And then used those numbers for statistical information-theoretical analysis to estimate the information density at about 9-11 bits (IIRC) per letter.

mizaru 7y ago
Looked it up, 11.82 bits per word, 2.62 bits per letter.
- tripzilch 7y ago
  Oh! Yes you're right. My mistake. That makes a lot more sense too, it would be weird if a letter was more bits than a byte, hm? :-)

auiya 7y ago

Are there weights applied for regional disparities? For instance sped up for American New Englanders and slowed down for American South Easterners?

cellular 7y ago

YouTube at 2x with captions can speed this to less than 78 seconds. "Less than " to account for advertising.

humble_engineer 7y ago

Curiously, this is also why memes are much better at conveying complicated political or sociological ideas faster and more efficiently than a few paragraphs on the topic.