14 ms·

The unbearable slowness of being: Why do we live at 10 bits/s?

296 points by sebg 2y ago

codedokode 2y ago
The 10 bit/s processing rate doesn't explain why a human talks better than a LLM that consumed terabytes of data traffic during learning.
cebert 2y ago
> Plausible explanations exist for the large neuron numbers in the outer brain, but not for the inner brain, and we propose new research directions to remedy this. Going on a limb here, but perhaps we shouldn’t modify biological composition of the human brain.
- mtlmtlmtlmtl 2y ago
  They're not talking about changing the brain. They're talking about remedying the lack of a plausible explanation.
  - thegeomaster 2y ago
    I think the parent commenter was making a joke.
ayongpm 2y ago
Interesting summary, where can I get the full text?
- sebg 2y ago
  Oops - my bad - here’s the arxiv link: https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234
nusl 2y ago
It's 1⁢0^9 bits/s. Your title is wrong.
- ceejayoz 2y ago
  > The information throughput of a human being is about 10 bits/s. In comparison, our sensory systems gather data at ∼1⁢0^9 bits/s. The title appears to be accurate?
  - mxfh 2y ago
    Just for playing any sport the accuracy to instruct 100s of muscles to work in a certain way is certainly above that 10bits, Pointing out positions in a 10cm x 10cm x 10cm cubic volume seems to possible significantly faster than 1/s. The slower examples listed in the table all have some externalities like a motor/object manipulation feedback loop overhead (speed cubing) and or redundacy and are not optimized for pure information density, so I have no idea why they settled on that average, and not the optimum? Object Recognition and Reading are already at ~50 bits. https://arxiv.org/html/2408.10234v2#S3 https://arxiv.org/html/2408.10234v2#S3
    - ceejayoz 2y ago
      > Just for playing any sport the accuracy to instruct 100s of muscles to work in a certain way is certainly above that 10bits But significant portions of that process are not done by the conscious brain, and some aren't done by the brain at all (reflex and peripheral nervous system). We don't consciously think about each of the 100 muscles we're switching on and off at rapid speed.
deleted 2y ago
[deleted]
luka598 2y ago
arxiv: https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234
- tetris11 2y ago
  Really fun paper. I especially enjoyed this section: > Based on the research reviewed here regarding the rate of human cognition, we predict that Musk’s brain will communicate with the computer at about 10 bits/s. Instead of the bundle of Neuralink electrodes, Musk could just use a telephone, whose data rate has been designed to match human language, which in turn is matched to the speed of perception and cognition
  - capitainenemo 2y ago
    It might be though that even though our processing rate is limited to 10 bits per second, shortening the communication loop between the helper AI and the human might allow the human to switch subjects more productively by getting faster feedback. The human would be in an executive approval role like the lead character in Accelerando with their agents, assuming they trusted their agents to be delegated to.
lccerina 2y ago
This is such a bad paper. Almost all calculations and equations look like some back of envelope calculation. A decent researcher would have provided some tests to their hypotheses.
- GeoAtreides 2y ago
  The numbers cited and used in calculations are supported by citations. The purpose of this paper is not to test a hypothesis, or to gather new data, but to think about existing data and new directions of research. This is spelled out in the paper's abstract, which is kind of summary of the whole paper, useful to get a very quick idea about the paper's purpose -- expanded further in the paper's introduction and re-visited again in the paper's conclusion.
  - lccerina 2y ago
    Thank you for explaining what an abstract is... The fact that those number come from a citation doesn't make them true. This is a badly written paper that a decent researcher wouldn't have written (and I know that the author has many papers, I am speaking about this one) and a decent reviewer would have rejected. A paragraph about Elon Musk? Guesstimates on information rates? As a blog post would have been okay-ish, as a scientific paper is quite bad.
    - GeoAtreides 2y ago
      >The fact that those number come from a citation doesn't make them true it does make them the citated paper problem, though. the guesstimates are explained as guestimates, and used as illustration for possible upper limits.
      - lccerina 2y ago
        The problem is that the PR machine of caltech then spits out articles like https://medicalxpress.com/news/2024-12-brain-paradox-quantifies-human-thought.html https://medicalxpress.com/news/2024-12-brain-paradox-quantif... or https://www.discovermagazine.com/mind/the-information-entering-our-brains-dwarfs-the-amount-coming-out-why https://www.discovermagazine.com/mind/the-information-enteri... or https://www.scientificamerican.com/article/the-human-brain-operates-at-a-stunningly-slow-pace/ https://www.scientificamerican.com/article/the-human-brain-o... with words like "measure" and "quantifies", "fill that quantitative gap". There are no measurements here, I can guess the weight of an apple based on some prior (which my brain stores as some continuous distribution, not bits), but I am not measuring it. It's incredibly tiring that bad science is sold as good science only because it comes from some fancy university. This paper is crap and should be treated as such.
nis0s 2y ago
It seems this analysis is incorrectly assuming a serial communication mode for neuronal transmission, which isn’t what happens.
- GeoAtreides 2y ago
  I have read the paper and your thesis, that the analysis is assuming a serial communication mode for neuronal transmission, is incorrect.
  - nis0s 2y ago
    They say this directly in section 7.2, > In contrast, central processing appears to be strictly serial… and then they proceed to give misinterpretated evidence of serialization because they’re making assumptions about lower level biochemical behavior based on higher level tissue performance. In fact, that tissue-level behavior isn’t correctly described either.
    - GeoAtreides 2y ago
      Be honest. The whole paragraph is: "In contrast, central processing appears to be strictly serial: When faced with two tasks in competition, individuals consistently encounter a “psychological refractory period” before being able to perform the second task broadbent_perception_1958, pashler_dual-task_1994. Even in tasks that do not require any motor output, such as thinking, we can pursue only one strand at a time." Clearly they're not talking about "neuronal transmission", but tasks, and further more, they cite their sources.
      - nis0s 2y ago
        I wasn’t being “dishonest”, I couldn’t copy/paste the entire text on my phone. I addressed the rest of that statement in my comment by noting that you can’t make the same assumptions about biochemical reactions and emergent behaviors of tissues. Secondly, even from a neurophysiology perspective, their cited evidence is misinterpreted. Any basic dual N-back task proves their central thesis incorrect.
cabirum 2y ago
> thinker can access about 220 ≈ 1 million possible items in the few seconds allotted Huh, no? No one is able to think about million items in a few seconds. The 20q thinking process involves bringing an incomplete set of abstract categories and asking a question that divides these categories into two halves (binary search). You don't even start from scratch, using previous experience (cache) to reuse whatever worked best the last time.
aithrowawaycomm 2y ago
I think the authors are using information theory to inappropriately flatten the complexity of the problem. On one hand we have “bits” of pre-processed sensory measurement data, then on the other hand we have “bits” of post-processed symbolic data: in many cases directly so via human language, but that would also include “the Terran unit moved a short distance” as a compact summary of a bunch of pixels updating in StarCraft. This even extends to the animal examples: the 10 bits/s figure applies to higher-level cognition. The crucial difference is that the sensory bits can be interpreted via the same “algorithm” in a context-independent way, whereas the higher-level cognition bits need their algorithms chosen very carefully (perhaps being modified at runtime). So I am just not sure why 10 bits/s of symbolic data processing is especially slow in the first place. We don’t have a relevant technological comparison because none of our technology actually processes data in that fashion.
- nis0s 2y ago
  When compared directly to the 10^9 bits/s for sensory information, which uses the same type of information, it is slow.
- aeonik 2y ago
  It's more like quantum information theory isn't it? https://en.wikipedia.org/wiki/Quantum_information https://en.wikipedia.org/wiki/Quantum_information
- kbelder 2y ago
  I'm running a chat LLM on my local pc. It spits out text just slightly faster than I can type, but it is using much of my CPU and redlining my GPU. Is it processing at a dozen bits per second, or hundreds of millions? If the text the LLM generates is "that is true", can I consider that one bit of information? I agree, they're artificially simplifying the framing of the question to generate a lower number than is sensible.
gmuslera 2y ago
Why not? Processing an input of 10^9 bits, making sense of all of that, and contrast it against all your existing knowledge have an output speed 10 bits/s? It is not so bad. At least if we were really processing all that information in the same way. It had to be enough to let us survive, in the context of the challenges we faced through most of our evolution. We took a lot of shortcuts and trims there, that is why we have a system 1 and system 2 in place, with a lot of builtin cognitive biases because of that.
crazygringo 2y ago
Where do they get 10 bits/second? Heck, I can type way faster than 10 bits per second, even after gzipping the output. And when I consider the amount of sensory information that I consciously process (not that comes in, but that I conceptually analyze), it's got to be way higher. 10 bits/s doesn't pass the smell test.
- GeoAtreides 2y ago
  The response to the question of "where do they get 10 bits/second" can be found in the paper, in great detail if I might add.
  - crazygringo 2y ago
    I don't have access. Nor do most of us here probably. Can you share for us then?
    - GeoAtreides 2y ago
      this thread has 20 comments at the time of writing my comment. About two of them contain a link to the full paper, please take a look.
      - ziddoap 2y ago
        It would be a lot less abrasive to say "It's linked elsewhere, but here it is: https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234" or some variation, instead of saying "it's here somewhere, go find it".
        GeoAtreides 2y ago
        with all due respect, it was meant to be slightly abrasive. it's understandable (?) not finding something when the thread has hundreds of comments, not so much when the thread had like 15-20 comments.
GeoAtreides 2y ago
I beg you, please read the paper before commenting. It's very interesting and it answers a lot of questions that might arise from just skimming the title.
- michaelt 2y ago
  Buddy, I followed the link and they want $35.95 to read the paper. This is... not a recipe for a successful discussion between people who have read the paper.
  - greyface- 2y ago
    https://news.ycombinator.com/item?id=42449845 https://news.ycombinator.com/item?id=42449845
    - lgas 2y ago
      Here's the link that link links to, to save people time: https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234
- MrMcCall 2y ago
  That might be the funniest comment I've ever seen on HN! A plea to reason, that is probably not outside the posting guidelines, but is certainly in a gray area :-)
  - GeoAtreides 2y ago
    I honestly don't understand why it would be funny or in a gray area to recommend people to actual read the paper?
    - MrMcCall 2y ago
      Asking people to read the article before commenting? A commonsense suggestion that needs to be made makes me smirk inside, not the least because I am guilty of this, too, around here. (But not this time, thank you, kind Sir.) As to being in a "gray area", have you read the posting guidelines? ;-) I'm pretty sure it says we shouldn't say things like "read the article" or "you haven't read the article, have you?" in our comments. Anyway, I'm laughing at this community (myself included) and the fact that your innocent and well-intentioned comment needs to be said here. And it did and does, my friend!
newswasboring 2y ago
Found the pre print if you don't have access https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234
MrMcCall 2y ago
The only time 'bits' will ever be an appropriate measure of human processing is when we are processing or producing diginal information artifacts, e.g. a typewritten document. Our bodies' systems are biochemical wetware that will never be aptly described using a boolean basis. That is one of the primary problems of society's obsessions with classical notions of gender. No one is male OR female. We are, every single one of us, a combination of male and female hormones. The more "male" a person is is the result of that balance favoring the male hormones; and vice versa. What humanity is now struggling with is that there are plenty of folks with lots of both or little of either and all kinds of combinations. Of course, my not being a biochemist means my categorization of hormones into "male" and "female" is, itself, likely to be a poorly booleanized representation of their natures. We are much more akin to Boltzmann's statistical mechanics description of reality, than to digital logic's boolean description.
- beezlebroxxxxxx 2y ago
  > That is one of the primary problems of society's obsessions with classical notions of gender. What you go on to discuss is sex, and sexual dimorphism, which is a remarkably robust way of classification. The "classical" notions of gender (tbh, "classical" doesn't make much sense here) as sex based is fairly reasonable all things considered. Consider the arguments presented in this essay [0]. That, however, doesn't really mean much for how we should treaty people in public who desire to express their gender in different ways, which is, of course, respecting of their dignity and desires, in most cases. [0]: https://philosophersmag.com/unexceptional-sex/ https://philosophersmag.com/unexceptional-sex/
  - MrMcCall 2y ago
    Well said. Yeah, what I mean by classical would boil down to just genitalia, which doesn't really hold up in how we must respect the person and how they feel and choose to express themselves. Yes, so long as their expressions are not harming others, then we must respect their human right to choose who they are. I've got to give a huge hat tip to Suzi (Eddie) Izzard, who -- beyond their being just a brilliant comic and generally good human being -- taught me and my fam about how the spectrum of human configuration is way more complex than just male and female. Cheers, friend.
FL33TW00D 2y ago
The title is a reference to "The unbearable lightness of being" by Milan Kundera for those unaware.
wigster 2y ago
perhaps the 10 bits/s is throttled at the simulation level ;-)
tonetheman 2y ago
[dead]
mjburgess 2y ago
> If the questions are properly designed, each will reveal 1 bit of information about the mystery thing. If the guesser wins routinely, this suggests that the thinker can access about 2^20 ≈ 1 million possible items in the few seconds allotted. So the speed of thinking – with no constraints imposed – corresponds to 20 bits of information over a few seconds: a rate of 10 bits per second or less. This is an extrinsic definition of "information" which is task relative, and has little to do with any intrinsic processing rate (if such a thing can even be defined for the imagination). The question of why does biological hardware capable of very high "intrinsic rates" deliver problem solving at "very low extrinsic rates" seems quite trivial. Its even a non-sequitur to compare them: properties of the parts are not properties of wholes. "Why does a gas move at 1 m/s, when its molecules move at 1000s m/s..." All the 'intrinsic processing' of intelligence is concerned with deploying a very large array of cognitive skills (imagination, coordination, planning, etc.) that are fully general. Any given task has requires all of those top be in operation, and so we expect a much slower rate of 'extrinsic information processing'. Consider how foolish the paper is to compare the intrinsic processing of a wifi network with the extrinsic task-specific processing of a human: it is likewise the case that if we set a computer the challenge of coordinating the solution of a task (eg., involving several LLMs) across a network, it's task-specific performance would drop off a cliff -- having a much slower 'solution rate' than 10bit/second. These 'task-specific bits' represent a vast amount of processing work to solve a problem. And are at least as much to do with the problem, than the system solving it. It seems to me all this paper does is define tasks in a highly abstract way that imposes a uniform cost to process '1 bit of task information'. Do the same for computers, and you'd likewise find tiny bitrates. The rate at which a problem is solved is 'one part of that problem per second' for a suitable definiton of 'part'
- psb217 2y ago
  Another relevant point is the common anecdote about, eg, some master engineer who gets paid big bucks to come fix some problem that's been blocking up a factory for weeks. The engineer walks around, listens to a few of the machines, and then walks up to one of the machines and knocks it with his elbow Fonzi style and the factory starts running again. The factory boss is happy his factory is working, but annoyed that he paid so much for such an "easy" solution. Ie, the amount of input and processing required to produce the "right" 10 bits might be far larger than 10 bits. Another obvious example is chess. The amount of bits conveyed by each move is small but, if you want to make the right move, you should probably put some deeper thought into it. Humans are essentially organisms that collect and filter information, boil it down to a condensed soup of understanding, and emit a light sprinkle of carefully chosen bits intended to reshape the future towards their desires.
cess11 2y ago
I see there are some people in the thread that doubt the low bandwidth between conscious thought and the rest of the central nervous system. Do you also doubt that you're actually living half a second in the past, with the brain compensating for this lag between initial perception and conscious reception of the indirect effects of it?
m3kw9 2y ago
Our awareness of time seem arbitrary. If feels time went by fast/slow, how does that work anyways?
bennettnate5 2y ago
> Why can we only think about one thing at a time? Maybe this is just a perception thing. Sure, you can only really keep up one stream of thought, visualization or inner dialogue (whatever you want to call it) at a time, but perhaps that's because we learn all our lives that direct communication is a one-channel, linear thing--speaking and listening focused on one topic at a time. Our brain does plenty of thinking in the background that leads to "a-ha!" moments even when the direct focus of our thoughts isn't on that topic. What if the mind could maintain multiple threads of thoughts at once, but our language coerces our thought patterns into being linear and non-concurrent?
- thmsths 2y ago
  I am not qualified to judge whether you're right or wrong but I love that concept!
- jdbxhdd 2y ago
  Also I do not agree with the premise that we can only think about one thing at a time. We routinely communicate with multiple people at once and also communicate with the same persons in multiple threads of conversations. Of cause this means that we switch between those tasks and do not really do them in parallel. At most we listen to one person, answer a second via speech, a third via text while thinking about what to respond to a fourth We just switch our focus of attention quite fast
  - imzadi 2y ago
    This is the part that bothers me. I can definitely think of multiple things at a time. It really just depends on the complexity of the tasks. I can listen to and process and audiobook while driving to work every morning, for instance. I definitely can have multiple thoughts in parallel. I remember when I used to recite prayers, I would be reciting the memorized prayer while thinking about other things. Both things were happening at the same time. The memorized task takes less processing power, but it still requires some thought to execute.
- Enginerrrd 2y ago
  As someone without an inner monologue, and someone that's spent a LOT of time meditating, it's not the language. It's the attention mechanisms themselves. Buddhist scholars insist that while we can have multiple threads of attention in our awareness, like strings with pearls of experience/thoughts we can only actually hold one little pearl of information from that stream in our attention at a time, and that we flit between them quite rapidly. Personally, I sort of agree, but I notice that there seems to be a time-compression thing happening where the pearl delivered to attention can contain a compressed summary of continuous perception. This seems to work for 2 things at once in awareness. When you start monitoring 3+ streams, there are gaps. And even maintaining the 2 streams continuously is exhausting so the mind tends to relax a little and leave gaps on a normal basis, but it seems like it can monitor dual feeds when its particularly important. My understanding is that neuroscience largely seems to agree with the above. (Actually, I'll note that the USUAL mode of being doesn't even monitor one stream continuously. A lot of the weird effects (and deeply interesting ones!) they talk about in meditative arts seem to pop up when you progress to being able to hold truly continuous attention.)
linuxdude314 2y ago
FWIW the title of this article is a play on the title of Milan Kundera's famous book "The Unbearable Lightness of Being".
PittleyDunkin 2y ago
> Why can we only think about one thing at a time? This sort of reasoning seems to be a symptom of inadequate communication/jargon/diction describing mental faculties. Many times during serious thought there's no discrete "number of thoughts" occuring at all: there's just a hazy mental process that resolves to some result and often many results. This reminds me of the "80% of people have no inner monologue!!!" bullshit that went around recently.
VoodooJuJu 2y ago
Bits are a unit of measurement we use in relation to computers. Humans are not computers. Do not use bits to measure anything in relation to humans. Stop thinking of humans as computers. It's dehumanizing, unhealthy, and a very narrow way to think of people. Using a computer model for humans is useless at best and misleading at worst.
- skibz 2y ago
  This paper isn't proposing that humans can be understood using a "computer model".
tsimionescu 2y ago
It seems very odd that the article seems to be measuring the information content of specific tasks that the brain is doing or specific objects that it is perceiving. But the brain is a general-purpose computer, not a speed-card computer, or English text computer, or binary digit computer, or Rubik's cube computer. When you look at a Rubik's cube, you don't just pick out specific positions of colored squares relative to each other. You also pick up the fact that it's a Rubik's cube and not a bird or a series of binary digits or English text. If an orange cat lunged at your Rubik's cube while you were studying it, you wouldn't process it as "face 3 has 4 red squares on the first row, then an orange diagonal with sharp claws", you'd process it as "fast moving sharp clawed orange cat attacking cube". Which implies that every time you loom at the cube you also notice that it's still a cube and not any of the millions of other objects you can recognize, adding many more bits of information. Similarly, when you're typing English text, you're not just encoding information from your brain into English text, you're also deciding that this is the most relevant activity to keep doing at the moment, instead of doing math or going out for a walk. Not to mention the precise mechanical control of your muscles to achieve the requisite movements, which we're having significant trouble programming into a robot.
- gtirloni 2y ago
  My thoughts exactly. It makes no sense to me that what I'm thinking and perceiving in real-time is the equivalent of 10 bit/s of data.
  - sigmoid10 2y ago
    Has anyone here even read more than the title? The article literally explains that you perceive at a rate of 10^9 bits per second. But after filtering and preprocessing in the outer brain you are only left with about 10 bits per second for conscious processing for things like motor functions. Yes, you can see a Rubic's cube and perceive all sorts of facts about it and the environment at the same time. But try solving it with your hands while someone shows you a bunch of objects and asks you visual comprehension questions at the same time. You might still perceive those other objects, but consciously classifying them verbally is gonna be really difficult. It's no surprise that the feature space that your deeper brain can actively work on is quite limited.
flerchin 2y ago
On average, people can read at least 200 WPM, but much higher at the top end. This is orders of magnitude higher than 10 bps.
- linearlayer 2y ago
  No it's not
l33tman 2y ago
The optical nerve has an information density of around 10 MBit/s (ref https://pmc.ncbi.nlm.nih.gov/articles/PMC1564115/ https://pmc.ncbi.nlm.nih.gov/articles/PMC1564115/) Concentrating on only the symbolic thinking speed seems to be unnecessarily restrictive..
- nabla9 2y ago
  The only limit is the max size of Hacker News title allows. :)
- a_wild_dandan 2y ago
  An additional citation substantiating that 10 MB/s figure: the 3rd sentence of TFA.
gchamonlive 2y ago
The authors of the article are smuggling in the assumption that 10bits/s is slow. It's slow when compared to general computing system that we implemented in silicon substrate. But this assumption doesn't translate linearly to the brain throughput and the perception of existence. In my opinion the hypothesis is meaningless. That is not to say the article is meaningless. Actually being able to measure brain information throughput is amazing. It's only that slowness isn't absolute.
Zondartul 2y ago
Ask stupid questions, receive stupid answers.
codedokode 2y ago
> In particular, our peripheral nervous system is capable of absorbing information from the environment at much higher rates, on the order of gigabits/s. This defines a paradox: The vast gulf between the tiny information throughput of human behavior, and the huge information inputs on which the behavior is based. This enormous ratio – about 100,000,000 – remains largely unexplained The GPU is capable of performing billions of operations per second, yet Cyberpunk barely runs at 60 fps. And there is no paradox at all. By the way, the brain seems to perform better than a GPU at tasks like image recognition. Probably because it does even more operations per second than the GPU.
- bee_rider 2y ago
  Is the brain better than a GPU at image recognition nowadays? Actually I’m not sure how that’s measured. Certainly a GPU could be tied to a database with a lot more things in it, like you can get some pretty impressive facial recognition demos where it’ll recognize a ton of random people. But humans can see objects they’ve never seen before and sometimes guess what they might be used for, which is sort of like object recognition but better. (Or sometimes I see an object I’m technically familiar with, like an old tool of my grandpa’s, and remembering what he used it for feels more like imagining… maybe it is).
- codedokode 2y ago
  There is also another comparison. Imagine if your goal is to calculate an integral over 100 dimensional space (or solve a quantum system) and answer whether it is larger or less than zero. This will take enourmous time but produces a single bit of information.
mulippy 2y ago
Seems like onsciousness is the bottleneck. It has to integrate over all the perceptions. Of course this will be slower!
kidel001 2y ago
These types of articles are so fundamentally flawed... it beggars belief. Why not ask the opposite question: if bandwidth works the way they describe, why can't H100 GPUs (3TB/s bandwidth) perform sensorimotor tasks 24 trillion times faster than a human? (Spoiler alert: they can not). <s> Could it be... there is a bit of a straw man argument here? About how much information it actually takes to input and output a complete sensorimotor task? I dare say! </s
metalman 2y ago
I tried to read the article, but celldotcom has a presumably very high bit rate robot that promptly questioned my humanity, so I did the dishes and ate lunch, but that didn't get through somehow as proof.(of humanity) And so my recourse is to then read the coments here, to try and get the gist of the argument,but even well fed, doing 11 of maybe 12 bits per second, there does not seem to be any point in quibling with reality. Maybe after a bit of shock-o-late icecream, (B and G chockolate therapy with esspresso beans added)
Helmut10001 2y ago
I read somewhere that the eye transmits 10 Million bits per second to the brain. I think all of this is a matter of perspective.
titzer 2y ago
> More generally, the information throughput of human behavior is about 10 bits/s. I'm sorry, I just can't take this article seriously. They make a fundamental mistake of encoding and assume that information is discretized into word-sized or action-sized chunks. A good example is a seemingly discrete activity such as playing a musical instrument, like a guitar. A guitar has frets and strings, a seemingly small number of finite notes it can play. So it would seem a perfect candidate for discretization along the lines of the musical scale. But any guitar player or listener knows that a guitar is not a keyboard or midi synth: 1. The attack velocity and angle of the pick intones aggression and emotion, not just along a few prescribed lines like "an angry or sad or loud or quiet". 2. Timing idiosyncracies like being slightly before or after a beat, or speeding up or slowing down, or even arhythmic; the entire expression of a piece of music is changed by subtleties in phrasing. 3. Microbends. The analog nature of strings cannot be hidden entirely behind frets. Differences in the amount of pressure, how close to the fret the fingers are, and slight bending of the strings, intentional or unintentional, static or dynamic, change the pitch of the note. 4. Non-striking sounds like the amount of palming, pick scraping, tapping, and sympathetic vibrations. Of course there are lots of other things. All of these things make the difference between a master guitar player, say Hendrix, and someone just playing the same notes. And yes of course we can consider the encoding of the audio coming out of the guitar to be information--at a much higher bitrate, but what about the facial expressions, body language, etc? There are tons of channels coming off a musician, particularly live performances. This entire article just misses these in picking a quantized encoding of information that of course has a low bitrate. In short, they are missing bazillions of channels, not the least of which is expression and timing.
constantcrying 2y ago
It seems the authors conflate a problem bring easy to state with little processing power being needed to solve it. This obviously isn't true, very complex mathematical problems can be stated in very few bits. Human interactions often can be extremely complex, even though they are relatively slow. Reading a text isn't about matching symbols to words. It is about taking these words and putting them into a social context, potentially even doubting their content or imagining the inner world of the author. Obviously that is what the "inner" brain (which existence seems very dubious to me) has to do. I see absolutely no paradox at all.
devenson 2y ago
Thinking is emulated -- therefore much slower.
zelon88 2y ago
Human's can transfer up to 39 bit/s during normal speech, so I highly doubt that it's accurate to describe human "throughput" as being only 10 bit/s. https://www.science.org/content/article/human-speech-may-have-universal-transmission-rate-39-bits-second https://www.science.org/content/article/human-speech-may-hav...
- mrguyorama 2y ago
  How long a time can you continuously generate that 40bps? If you can only do that 1/4 the time, that's 10bps
nis0s 2y ago
I think the discussion on serial vs parallel processing is incomplete in the linked paper, and is one reason I think the 10 bits/s bottleneck is an incomplete or misinterpreted result. Here’s a review with sources on serial processing, https://journalofcognition.org/articles/10.5334/joc.185 https://journalofcognition.org/articles/10.5334/joc.185 > Cognitive psychology has mainly focused on structural and functional limitations of cognitive processes when facing multitasking requirements. Structural limitations assume strict serial processing for at least one processing stage, while functional limitations assume flexible, parallel processing only limited by the number of available resources. Human movement science, on the other hand, emphasizes the plasticity of cognition and training possibilities. As both approaches have provided ample empirical evidence for their views but have predominantly worked in isolation, this example clearly illustrates the need for a more integrative approach to multitasking. A challenge for the contemporary research on multitasking is to bring together the issues of structure, flexibility, and plasticity in human multitasking, offering a new integrative theoretical framework that accounts for this fundamental aspect of human behaviour. From one of the papers cited by the above reference (Hommel 2020), > A closer look reveals that the questions being asked in dual-task research are not particularly interesting or realistic, and the answers being given lack mechanistic detail. In fact, present theorizing can be considered mere empirical generalization, which has led to merely labeling processing bottlenecks rather than describing how they operate and how they actually produce the bottleneck. So, while I applaud the authors on generating buzz and discussion, I think their promising work will benefit from more serious consideration of the underlying neurophysiology.
stainablesteel 2y ago
there's probably lots of local computation and i think it's a mistake to simplify it all down to just one substrate or receptor
krzat 2y ago
Related question: what's the physical/biological difference between conscious and unconscious?
vok 2y ago
There's interesting discussion with the authors here: https://www.alphaxiv.org/abs/2408.10234 https://www.alphaxiv.org/abs/2408.10234
kubasienki 2y ago
https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234
_yb2s 2y ago
What would it really even mean for our brain to be fast or slow? Relative to what? Inherent speed does not matter and is ill defined, it only matters relative to the environmental processes we have to react to. We’re already orders of magnitude faster than processes like geology and weather, so it is really only other animals where speed matters, and since we’re all using the same basic hardware, we’re on a relatively level playing field there (except for house flies it seems, lol). Time as we understand it may be as much as a cultural construct as anything else (see, for example Timeless physics). Some neurodivergent people, and people from very different cultures don’t experience anything like what most people in our culture refer to as time. As for thinking about one thing at a time- I am absolutely certain this is false. Our subconscious operates massively parallel, and we only have one conscious thought at a time. But it had an interrupt system that lets it either act instantly itself in an emergency, and to bring things to consciousness when they are important. I’m sure everyone has had the experience of reacting quickly to an emergency with no conscious thought, or suddenly knowing the solution to a problem you hadn’t even been consciously thinking about.
Barrin92 2y ago
I skimmed over the paper so sorry if I didn't see it, but isn't the most obvious answer that (at least conscious) human decision making happens at a high level of abstraction? Information output or processing in terms of bits, say the numbers of words in a sentence, isn't equivalent to the meaning or effect that has in the world, which is what matters. If a general orders an army to go to war that's a few bits of text on a piece of paper, but obviously that bears no relation to the consequence of what that puts into motion in the real world. It's not like we spend most of our days speedcubing or writing and reading the dictionary. We aren't low level data processors even though that might happen unconsciously somewhere in the sensory system, but the conscious mind is a reasoning system. Even in artificial systems is that visible, from Deepmind on Alpha Zero (https://deepmind.google/discover/blog/alphazero-shedding-new-light-on-chess-shogi-and-go/ https://deepmind.google/discover/blog/alphazero-shedding-new...) "For each move, AlphaZero searches only a small fraction of the positions considered by traditional chess engines. In Chess, for example, it searches only 60 thousand positions per second in chess, compared to roughly 60 million for Stockfish." Not too draw to many parallels between the human brain and these systems, but they do obviously share the similarity that higher order conceptual decision making compared to just data processing will result in lower rates of decision making at at least the top level. That's for me what you'd expect to happen, not a paradox.
0xbadcafebee 2y ago
We process information at the rate we do because that's what works for adaptation to our environment. That's how all biological things develop. Some things are remnants of old adaptations, but generally speaking, we are the way we are because that's what enables our survival. If it seems "slow", it's because it doesn't need to be faster. Better questions would be, why are we so weak? Why are we defenseless? Why are we hairless? Why do we move so slow? Most other mammals our size are much better equipped for survival. Obviously other beings "perform better" in many ways. Yet our fleshy, weak, slow, gangly, shivery bodies are suited for thinking, for adapting, for communicating and collaborating. However unoptimal or "slow" these authors think our brains are, they are obviously perfectly capable for what they need to do, seeing as we dominate the entire planet. In any system design, every decision tends to be a tradeoff. You can gain CPU power, but it will cost you energy, heat, and probably stability or longevity. You often do not know what a "feature" will do until you add it. So it's wiser to only add the features you need to accomplish your immediate goals. If at some point in the future, our survival is existentially threatened by our lack of "processing power", our brains will adapt. Until then, leave well enough alone.
peterfirefly 2y ago
10 baud, perhaps. Definitely not 10 bits/s!
luxuryballs 2y ago
I’m interested in the phenomenon that no matter how messed up (or absent) my sensory input becomes it doesn’t seem to impact my “inner observer”. The only thing impacting it is sleep or loss of consciousness, yet even then there can be moments during these states that this observer can become lucid. It’s like the science of the soul and one notable implication is dementia, when people become unable to function and recognize things, I wonder if that “inner observer” is still just as in-tact as it is on a huge dose of LSD?
rbanffy 2y ago
I prefer to think of the inherent slowness of my organic brain as a leisurely stroll through time. I’m taking the scenic route though life.
kirkules 2y ago
The back of the envelope computations are shockingly shallow and meaningless. (20 Questions, from the intro) Trying to think of a thing for the game is not a search over a set of known things. Just saying the possibility set has size 2^N doesn't mean that choosing something in the set consists of processing the set. But even if that were the case, and if you do consider each of 2^N options, the consideration process itself is not trivial and probably varies wildly. (English typing) Touch typists do not (only) simply convert an existing/known string to a sequence of hand actions by mapping character to action. There are whole words and sequences that become units/tokens from the standpoint of muscle memory and processing (this will be relevant to the rubik's cube topic as well). When i type, there's a sort of planning and queueing of actions, but also there's monitoring of actions that allows fast error correction with pressing delete a number of times or holding it and costly determining when the error has been reached, and resuming afterward. Of course the process likely varies from person to person, but there's such a host of other things going on that should count as part of the information processed in this simple behavior that the example and numbers used in the paper for it are utterly useless even as estimates. (Rubik's cube blind speed solving) Again we see reference to the entire possibility space (from the perspective of possible configurations of the puzzle). But solvers do not identify the configuration they encounter with reference to the space, nor do they search the space for it. They look for patterns and ignore what they cannot use for the strategy they have practiced. The cuber often does not commit to memory the whole configuration, but will often convert it to a custom and bespoke mnemonic. It's just utter nonsense to refer to the number of possible configurations, it has nothing directly to do with what the human is doing. If I memorize a 30 word passage, i have not "processed the set of possible 30 word passages".
Jensson 2y ago
The answer is that we defined second after the time period it takes for us to think about something, if our thinking was faster the second would also be faster and we would still feel like we only think about about one thing per second.
byyoung3 2y ago
"Why can we only think about one thing at a time?" - this is not true? There is a difference between thinking and conscious thinking.
tim333 2y ago
Free access link https://arxiv.org/pdf/2408.10234 https://arxiv.org/pdf/2408.10234
kykeonaut 2y ago
So the human brain is a great operating system that is desperately missing a good text editor...
zogomoox 2y ago
"It is by will alone I set my mind in motion. It is by the juice of Sapho that thoughts acquire speed, the lips acquire stains, the stains become a warning. It is by will alone I set my mind in motion."
weddpros 2y ago
You can compress 10Gbit/s down to 10bit/s but then you can't do the reverse operation... Yeah, we know. To me it's like saying "I've summarised this book to 5 words, so why would you even need a few days to read it entirely?".
- deleted 2y ago
  [deleted]
ckemere 2y ago
Definitely a neat article insofar as it summarizes many observations about task related “bandwidth” of human output. Summarizing other comments - calculating bandwidth in a task-specific way neglects metacognition: the Rubik’s cube solver can rapidly recover if someone shoves them when their blindfold is on; the memory champion can notice a pie being aimed at their face and duck. I think that there’s still something interesting around our inability to multitask conscious behaviors, but the overarching conclusion is flawed.
effed3 2y ago
10"very-ultra-extreme-high-level"Bit/s (maybe)
leflores 2y ago
At what speed is the brain processing when a pair of Chinese ping pong masters are playing? All of this requires seeing the ball coming from the opponent, subconsciously calculating its speed, angle, and rotation to decide how to move your body, legs, arms, hands, and even fingers to hit the ball at the desired location, with the desired angle, and at the desired speed. A standard ping pong table is 2.74 meters long, and in high-speed professional games, the ball can traverse this distance in as little as 90–140 milliseconds. This means players have less than a tenth of a second to react. It's not just the immense image-processing speed required but also the mental speed to transform that visual data into precise positioning and movement of hundreds of muscles in less than a second. Ten bits per second? No way.
deleted 2y ago
[deleted]

296 points by sebg 2y ago

The 10 bit/s processing rate doesn't explain why a human talks better than a LLM that consumed terabytes of data traffic during learning.

cebert 2y ago

> Plausible explanations exist for the large neuron numbers in the outer brain, but not for the inner brain, and we propose new research directions to remedy this. Going on a limb here, but perhaps we shouldn’t modify biological composition of the human brain.

mtlmtlmtlmtl 2y ago
They're not talking about changing the brain. They're talking about remedying the lack of a plausible explanation.
- thegeomaster 2y ago
  I think the parent commenter was making a joke.

ayongpm 2y ago

Interesting summary, where can I get the full text?

sebg 2y ago
Oops - my bad - here’s the arxiv link: https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234

nusl 2y ago

It's 1⁢0^9 bits/s. Your title is wrong.

ceejayoz 2y ago
> The information throughput of a human being is about 10 bits/s. In comparison, our sensory systems gather data at ∼1⁢0^9 bits/s. The title appears to be accurate?
- mxfh 2y ago
  Just for playing any sport the accuracy to instruct 100s of muscles to work in a certain way is certainly above that 10bits, Pointing out positions in a 10cm x 10cm x 10cm cubic volume seems to possible significantly faster than 1/s. The slower examples listed in the table all have some externalities like a motor/object manipulation feedback loop overhead (speed cubing) and or redundacy and are not optimized for pure information density, so I have no idea why they settled on that average, and not the optimum? Object Recognition and Reading are already at ~50 bits. https://arxiv.org/html/2408.10234v2#S3 https://arxiv.org/html/2408.10234v2#S3
  - ceejayoz 2y ago
    > Just for playing any sport the accuracy to instruct 100s of muscles to work in a certain way is certainly above that 10bits But significant portions of that process are not done by the conscious brain, and some aren't done by the brain at all (reflex and peripheral nervous system). We don't consciously think about each of the 100 muscles we're switching on and off at rapid speed.

deleted 2y ago

[deleted]

luka598 2y ago

arxiv: https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234

tetris11 2y ago
Really fun paper. I especially enjoyed this section: > Based on the research reviewed here regarding the rate of human cognition, we predict that Musk’s brain will communicate with the computer at about 10 bits/s. Instead of the bundle of Neuralink electrodes, Musk could just use a telephone, whose data rate has been designed to match human language, which in turn is matched to the speed of perception and cognition
- capitainenemo 2y ago
  It might be though that even though our processing rate is limited to 10 bits per second, shortening the communication loop between the helper AI and the human might allow the human to switch subjects more productively by getting faster feedback. The human would be in an executive approval role like the lead character in Accelerando with their agents, assuming they trusted their agents to be delegated to.

lccerina 2y ago

This is such a bad paper. Almost all calculations and equations look like some back of envelope calculation. A decent researcher would have provided some tests to their hypotheses.

GeoAtreides 2y ago
The numbers cited and used in calculations are supported by citations. The purpose of this paper is not to test a hypothesis, or to gather new data, but to think about existing data and new directions of research. This is spelled out in the paper's abstract, which is kind of summary of the whole paper, useful to get a very quick idea about the paper's purpose -- expanded further in the paper's introduction and re-visited again in the paper's conclusion.
- lccerina 2y ago
  Thank you for explaining what an abstract is... The fact that those number come from a citation doesn't make them true. This is a badly written paper that a decent researcher wouldn't have written (and I know that the author has many papers, I am speaking about this one) and a decent reviewer would have rejected. A paragraph about Elon Musk? Guesstimates on information rates? As a blog post would have been okay-ish, as a scientific paper is quite bad.
  - GeoAtreides 2y ago
    >The fact that those number come from a citation doesn't make them true it does make them the citated paper problem, though. the guesstimates are explained as guestimates, and used as illustration for possible upper limits.
    - lccerina 2y ago
      The problem is that the PR machine of caltech then spits out articles like https://medicalxpress.com/news/2024-12-brain-paradox-quantifies-human-thought.html https://medicalxpress.com/news/2024-12-brain-paradox-quantif... or https://www.discovermagazine.com/mind/the-information-entering-our-brains-dwarfs-the-amount-coming-out-why https://www.discovermagazine.com/mind/the-information-enteri... or https://www.scientificamerican.com/article/the-human-brain-operates-at-a-stunningly-slow-pace/ https://www.scientificamerican.com/article/the-human-brain-o... with words like "measure" and "quantifies", "fill that quantitative gap". There are no measurements here, I can guess the weight of an apple based on some prior (which my brain stores as some continuous distribution, not bits), but I am not measuring it. It's incredibly tiring that bad science is sold as good science only because it comes from some fancy university. This paper is crap and should be treated as such.

nis0s 2y ago

It seems this analysis is incorrectly assuming a serial communication mode for neuronal transmission, which isn’t what happens.

GeoAtreides 2y ago
I have read the paper and your thesis, that the analysis is assuming a serial communication mode for neuronal transmission, is incorrect.
- nis0s 2y ago
  They say this directly in section 7.2, > In contrast, central processing appears to be strictly serial… and then they proceed to give misinterpretated evidence of serialization because they’re making assumptions about lower level biochemical behavior based on higher level tissue performance. In fact, that tissue-level behavior isn’t correctly described either.
  - GeoAtreides 2y ago
    Be honest. The whole paragraph is: "In contrast, central processing appears to be strictly serial: When faced with two tasks in competition, individuals consistently encounter a “psychological refractory period” before being able to perform the second task broadbent_perception_1958, pashler_dual-task_1994. Even in tasks that do not require any motor output, such as thinking, we can pursue only one strand at a time." Clearly they're not talking about "neuronal transmission", but tasks, and further more, they cite their sources.
    - nis0s 2y ago
      I wasn’t being “dishonest”, I couldn’t copy/paste the entire text on my phone. I addressed the rest of that statement in my comment by noting that you can’t make the same assumptions about biochemical reactions and emergent behaviors of tissues. Secondly, even from a neurophysiology perspective, their cited evidence is misinterpreted. Any basic dual N-back task proves their central thesis incorrect.

cabirum 2y ago

> thinker can access about 220 ≈ 1 million possible items in the few seconds allotted Huh, no? No one is able to think about million items in a few seconds. The 20q thinking process involves bringing an incomplete set of abstract categories and asking a question that divides these categories into two halves (binary search). You don't even start from scratch, using previous experience (cache) to reuse whatever worked best the last time.

aithrowawaycomm 2y ago

I think the authors are using information theory to inappropriately flatten the complexity of the problem. On one hand we have “bits” of pre-processed sensory measurement data, then on the other hand we have “bits” of post-processed symbolic data: in many cases directly so via human language, but that would also include “the Terran unit moved a short distance” as a compact summary of a bunch of pixels updating in StarCraft. This even extends to the animal examples: the 10 bits/s figure applies to higher-level cognition. The crucial difference is that the sensory bits can be interpreted via the same “algorithm” in a context-independent way, whereas the higher-level cognition bits need their algorithms chosen very carefully (perhaps being modified at runtime). So I am just not sure why 10 bits/s of symbolic data processing is especially slow in the first place. We don’t have a relevant technological comparison because none of our technology actually processes data in that fashion.

nis0s 2y ago
When compared directly to the 10^9 bits/s for sensory information, which uses the same type of information, it is slow.
aeonik 2y ago
It's more like quantum information theory isn't it? https://en.wikipedia.org/wiki/Quantum_information https://en.wikipedia.org/wiki/Quantum_information
kbelder 2y ago
I'm running a chat LLM on my local pc. It spits out text just slightly faster than I can type, but it is using much of my CPU and redlining my GPU. Is it processing at a dozen bits per second, or hundreds of millions? If the text the LLM generates is "that is true", can I consider that one bit of information? I agree, they're artificially simplifying the framing of the question to generate a lower number than is sensible.

gmuslera 2y ago

Why not? Processing an input of 10^9 bits, making sense of all of that, and contrast it against all your existing knowledge have an output speed 10 bits/s? It is not so bad. At least if we were really processing all that information in the same way. It had to be enough to let us survive, in the context of the challenges we faced through most of our evolution. We took a lot of shortcuts and trims there, that is why we have a system 1 and system 2 in place, with a lot of builtin cognitive biases because of that.

crazygringo 2y ago

Where do they get 10 bits/second? Heck, I can type way faster than 10 bits per second, even after gzipping the output. And when I consider the amount of sensory information that I consciously process (not that comes in, but that I conceptually analyze), it's got to be way higher. 10 bits/s doesn't pass the smell test.

GeoAtreides 2y ago
The response to the question of "where do they get 10 bits/second" can be found in the paper, in great detail if I might add.
- crazygringo 2y ago
  I don't have access. Nor do most of us here probably. Can you share for us then?
  - GeoAtreides 2y ago
    this thread has 20 comments at the time of writing my comment. About two of them contain a link to the full paper, please take a look.
    - ziddoap 2y ago
      It would be a lot less abrasive to say "It's linked elsewhere, but here it is: https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234" or some variation, instead of saying "it's here somewhere, go find it".
      - GeoAtreides 2y ago
        with all due respect, it was meant to be slightly abrasive. it's understandable (?) not finding something when the thread has hundreds of comments, not so much when the thread had like 15-20 comments.

GeoAtreides 2y ago

I beg you, please read the paper before commenting. It's very interesting and it answers a lot of questions that might arise from just skimming the title.

michaelt 2y ago
Buddy, I followed the link and they want $35.95 to read the paper. This is... not a recipe for a successful discussion between people who have read the paper.
- greyface- 2y ago
  https://news.ycombinator.com/item?id=42449845 https://news.ycombinator.com/item?id=42449845
  - lgas 2y ago
    Here's the link that link links to, to save people time: https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234
MrMcCall 2y ago
That might be the funniest comment I've ever seen on HN! A plea to reason, that is probably not outside the posting guidelines, but is certainly in a gray area :-)
- GeoAtreides 2y ago
  I honestly don't understand why it would be funny or in a gray area to recommend people to actual read the paper?
  - MrMcCall 2y ago
    Asking people to read the article before commenting? A commonsense suggestion that needs to be made makes me smirk inside, not the least because I am guilty of this, too, around here. (But not this time, thank you, kind Sir.) As to being in a "gray area", have you read the posting guidelines? ;-) I'm pretty sure it says we shouldn't say things like "read the article" or "you haven't read the article, have you?" in our comments. Anyway, I'm laughing at this community (myself included) and the fact that your innocent and well-intentioned comment needs to be said here. And it did and does, my friend!

newswasboring 2y ago

Found the pre print if you don't have access https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234

MrMcCall 2y ago

The only time 'bits' will ever be an appropriate measure of human processing is when we are processing or producing diginal information artifacts, e.g. a typewritten document. Our bodies' systems are biochemical wetware that will never be aptly described using a boolean basis. That is one of the primary problems of society's obsessions with classical notions of gender. No one is male OR female. We are, every single one of us, a combination of male and female hormones. The more "male" a person is is the result of that balance favoring the male hormones; and vice versa. What humanity is now struggling with is that there are plenty of folks with lots of both or little of either and all kinds of combinations. Of course, my not being a biochemist means my categorization of hormones into "male" and "female" is, itself, likely to be a poorly booleanized representation of their natures. We are much more akin to Boltzmann's statistical mechanics description of reality, than to digital logic's boolean description.

beezlebroxxxxxx 2y ago
> That is one of the primary problems of society's obsessions with classical notions of gender. What you go on to discuss is sex, and sexual dimorphism, which is a remarkably robust way of classification. The "classical" notions of gender (tbh, "classical" doesn't make much sense here) as sex based is fairly reasonable all things considered. Consider the arguments presented in this essay [0]. That, however, doesn't really mean much for how we should treaty people in public who desire to express their gender in different ways, which is, of course, respecting of their dignity and desires, in most cases. [0]: https://philosophersmag.com/unexceptional-sex/ https://philosophersmag.com/unexceptional-sex/
- MrMcCall 2y ago
  Well said. Yeah, what I mean by classical would boil down to just genitalia, which doesn't really hold up in how we must respect the person and how they feel and choose to express themselves. Yes, so long as their expressions are not harming others, then we must respect their human right to choose who they are. I've got to give a huge hat tip to Suzi (Eddie) Izzard, who -- beyond their being just a brilliant comic and generally good human being -- taught me and my fam about how the spectrum of human configuration is way more complex than just male and female. Cheers, friend.

FL33TW00D 2y ago

The title is a reference to "The unbearable lightness of being" by Milan Kundera for those unaware.

wigster 2y ago

perhaps the 10 bits/s is throttled at the simulation level ;-)

tonetheman 2y ago

[dead]

mjburgess 2y ago

> If the questions are properly designed, each will reveal 1 bit of information about the mystery thing. If the guesser wins routinely, this suggests that the thinker can access about 2^20 ≈ 1 million possible items in the few seconds allotted. So the speed of thinking – with no constraints imposed – corresponds to 20 bits of information over a few seconds: a rate of 10 bits per second or less. This is an extrinsic definition of "information" which is task relative, and has little to do with any intrinsic processing rate (if such a thing can even be defined for the imagination). The question of why does biological hardware capable of very high "intrinsic rates" deliver problem solving at "very low extrinsic rates" seems quite trivial. Its even a non-sequitur to compare them: properties of the parts are not properties of wholes. "Why does a gas move at 1 m/s, when its molecules move at 1000s m/s..." All the 'intrinsic processing' of intelligence is concerned with deploying a very large array of cognitive skills (imagination, coordination, planning, etc.) that are fully general. Any given task has requires all of those top be in operation, and so we expect a much slower rate of 'extrinsic information processing'. Consider how foolish the paper is to compare the intrinsic processing of a wifi network with the extrinsic task-specific processing of a human: it is likewise the case that if we set a computer the challenge of coordinating the solution of a task (eg., involving several LLMs) across a network, it's task-specific performance would drop off a cliff -- having a much slower 'solution rate' than 10bit/second. These 'task-specific bits' represent a vast amount of processing work to solve a problem. And are at least as much to do with the problem, than the system solving it. It seems to me all this paper does is define tasks in a highly abstract way that imposes a uniform cost to process '1 bit of task information'. Do the same for computers, and you'd likewise find tiny bitrates. The rate at which a problem is solved is 'one part of that problem per second' for a suitable definiton of 'part'

psb217 2y ago
Another relevant point is the common anecdote about, eg, some master engineer who gets paid big bucks to come fix some problem that's been blocking up a factory for weeks. The engineer walks around, listens to a few of the machines, and then walks up to one of the machines and knocks it with his elbow Fonzi style and the factory starts running again. The factory boss is happy his factory is working, but annoyed that he paid so much for such an "easy" solution. Ie, the amount of input and processing required to produce the "right" 10 bits might be far larger than 10 bits. Another obvious example is chess. The amount of bits conveyed by each move is small but, if you want to make the right move, you should probably put some deeper thought into it. Humans are essentially organisms that collect and filter information, boil it down to a condensed soup of understanding, and emit a light sprinkle of carefully chosen bits intended to reshape the future towards their desires.

cess11 2y ago

I see there are some people in the thread that doubt the low bandwidth between conscious thought and the rest of the central nervous system. Do you also doubt that you're actually living half a second in the past, with the brain compensating for this lag between initial perception and conscious reception of the indirect effects of it?

m3kw9 2y ago

Our awareness of time seem arbitrary. If feels time went by fast/slow, how does that work anyways?

bennettnate5 2y ago

> Why can we only think about one thing at a time? Maybe this is just a perception thing. Sure, you can only really keep up one stream of thought, visualization or inner dialogue (whatever you want to call it) at a time, but perhaps that's because we learn all our lives that direct communication is a one-channel, linear thing--speaking and listening focused on one topic at a time. Our brain does plenty of thinking in the background that leads to "a-ha!" moments even when the direct focus of our thoughts isn't on that topic. What if the mind could maintain multiple threads of thoughts at once, but our language coerces our thought patterns into being linear and non-concurrent?

thmsths 2y ago
I am not qualified to judge whether you're right or wrong but I love that concept!
jdbxhdd 2y ago
Also I do not agree with the premise that we can only think about one thing at a time. We routinely communicate with multiple people at once and also communicate with the same persons in multiple threads of conversations. Of cause this means that we switch between those tasks and do not really do them in parallel. At most we listen to one person, answer a second via speech, a third via text while thinking about what to respond to a fourth We just switch our focus of attention quite fast
- imzadi 2y ago
  This is the part that bothers me. I can definitely think of multiple things at a time. It really just depends on the complexity of the tasks. I can listen to and process and audiobook while driving to work every morning, for instance. I definitely can have multiple thoughts in parallel. I remember when I used to recite prayers, I would be reciting the memorized prayer while thinking about other things. Both things were happening at the same time. The memorized task takes less processing power, but it still requires some thought to execute.
Enginerrrd 2y ago
As someone without an inner monologue, and someone that's spent a LOT of time meditating, it's not the language. It's the attention mechanisms themselves. Buddhist scholars insist that while we can have multiple threads of attention in our awareness, like strings with pearls of experience/thoughts we can only actually hold one little pearl of information from that stream in our attention at a time, and that we flit between them quite rapidly. Personally, I sort of agree, but I notice that there seems to be a time-compression thing happening where the pearl delivered to attention can contain a compressed summary of continuous perception. This seems to work for 2 things at once in awareness. When you start monitoring 3+ streams, there are gaps. And even maintaining the 2 streams continuously is exhausting so the mind tends to relax a little and leave gaps on a normal basis, but it seems like it can monitor dual feeds when its particularly important. My understanding is that neuroscience largely seems to agree with the above. (Actually, I'll note that the USUAL mode of being doesn't even monitor one stream continuously. A lot of the weird effects (and deeply interesting ones!) they talk about in meditative arts seem to pop up when you progress to being able to hold truly continuous attention.)

linuxdude314 2y ago

FWIW the title of this article is a play on the title of Milan Kundera's famous book "The Unbearable Lightness of Being".

PittleyDunkin 2y ago

> Why can we only think about one thing at a time? This sort of reasoning seems to be a symptom of inadequate communication/jargon/diction describing mental faculties. Many times during serious thought there's no discrete "number of thoughts" occuring at all: there's just a hazy mental process that resolves to some result and often many results. This reminds me of the "80% of people have no inner monologue!!!" bullshit that went around recently.

VoodooJuJu 2y ago

Bits are a unit of measurement we use in relation to computers. Humans are not computers. Do not use bits to measure anything in relation to humans. Stop thinking of humans as computers. It's dehumanizing, unhealthy, and a very narrow way to think of people. Using a computer model for humans is useless at best and misleading at worst.

skibz 2y ago
This paper isn't proposing that humans can be understood using a "computer model".

tsimionescu 2y ago

It seems very odd that the article seems to be measuring the information content of specific tasks that the brain is doing or specific objects that it is perceiving. But the brain is a general-purpose computer, not a speed-card computer, or English text computer, or binary digit computer, or Rubik's cube computer. When you look at a Rubik's cube, you don't just pick out specific positions of colored squares relative to each other. You also pick up the fact that it's a Rubik's cube and not a bird or a series of binary digits or English text. If an orange cat lunged at your Rubik's cube while you were studying it, you wouldn't process it as "face 3 has 4 red squares on the first row, then an orange diagonal with sharp claws", you'd process it as "fast moving sharp clawed orange cat attacking cube". Which implies that every time you loom at the cube you also notice that it's still a cube and not any of the millions of other objects you can recognize, adding many more bits of information. Similarly, when you're typing English text, you're not just encoding information from your brain into English text, you're also deciding that this is the most relevant activity to keep doing at the moment, instead of doing math or going out for a walk. Not to mention the precise mechanical control of your muscles to achieve the requisite movements, which we're having significant trouble programming into a robot.

gtirloni 2y ago
My thoughts exactly. It makes no sense to me that what I'm thinking and perceiving in real-time is the equivalent of 10 bit/s of data.
- sigmoid10 2y ago
  Has anyone here even read more than the title? The article literally explains that you perceive at a rate of 10^9 bits per second. But after filtering and preprocessing in the outer brain you are only left with about 10 bits per second for conscious processing for things like motor functions. Yes, you can see a Rubic's cube and perceive all sorts of facts about it and the environment at the same time. But try solving it with your hands while someone shows you a bunch of objects and asks you visual comprehension questions at the same time. You might still perceive those other objects, but consciously classifying them verbally is gonna be really difficult. It's no surprise that the feature space that your deeper brain can actively work on is quite limited.

flerchin 2y ago

On average, people can read at least 200 WPM, but much higher at the top end. This is orders of magnitude higher than 10 bps.

linearlayer 2y ago
No it's not

l33tman 2y ago

The optical nerve has an information density of around 10 MBit/s (ref https://pmc.ncbi.nlm.nih.gov/articles/PMC1564115/ https://pmc.ncbi.nlm.nih.gov/articles/PMC1564115/) Concentrating on only the symbolic thinking speed seems to be unnecessarily restrictive..

nabla9 2y ago
The only limit is the max size of Hacker News title allows. :)
a_wild_dandan 2y ago
An additional citation substantiating that 10 MB/s figure: the 3rd sentence of TFA.

gchamonlive 2y ago

The authors of the article are smuggling in the assumption that 10bits/s is slow. It's slow when compared to general computing system that we implemented in silicon substrate. But this assumption doesn't translate linearly to the brain throughput and the perception of existence. In my opinion the hypothesis is meaningless. That is not to say the article is meaningless. Actually being able to measure brain information throughput is amazing. It's only that slowness isn't absolute.

Zondartul 2y ago

Ask stupid questions, receive stupid answers.

codedokode 2y ago

> In particular, our peripheral nervous system is capable of absorbing information from the environment at much higher rates, on the order of gigabits/s. This defines a paradox: The vast gulf between the tiny information throughput of human behavior, and the huge information inputs on which the behavior is based. This enormous ratio – about 100,000,000 – remains largely unexplained The GPU is capable of performing billions of operations per second, yet Cyberpunk barely runs at 60 fps. And there is no paradox at all. By the way, the brain seems to perform better than a GPU at tasks like image recognition. Probably because it does even more operations per second than the GPU.

bee_rider 2y ago
Is the brain better than a GPU at image recognition nowadays? Actually I’m not sure how that’s measured. Certainly a GPU could be tied to a database with a lot more things in it, like you can get some pretty impressive facial recognition demos where it’ll recognize a ton of random people. But humans can see objects they’ve never seen before and sometimes guess what they might be used for, which is sort of like object recognition but better. (Or sometimes I see an object I’m technically familiar with, like an old tool of my grandpa’s, and remembering what he used it for feels more like imagining… maybe it is).
codedokode 2y ago
There is also another comparison. Imagine if your goal is to calculate an integral over 100 dimensional space (or solve a quantum system) and answer whether it is larger or less than zero. This will take enourmous time but produces a single bit of information.

mulippy 2y ago

Seems like onsciousness is the bottleneck. It has to integrate over all the perceptions. Of course this will be slower!

kidel001 2y ago

These types of articles are so fundamentally flawed... it beggars belief. Why not ask the opposite question: if bandwidth works the way they describe, why can't H100 GPUs (3TB/s bandwidth) perform sensorimotor tasks 24 trillion times faster than a human? (Spoiler alert: they can not). <s> Could it be... there is a bit of a straw man argument here? About how much information it actually takes to input and output a complete sensorimotor task? I dare say! </s

metalman 2y ago

I tried to read the article, but celldotcom has a presumably very high bit rate robot that promptly questioned my humanity, so I did the dishes and ate lunch, but that didn't get through somehow as proof.(of humanity) And so my recourse is to then read the coments here, to try and get the gist of the argument,but even well fed, doing 11 of maybe 12 bits per second, there does not seem to be any point in quibling with reality. Maybe after a bit of shock-o-late icecream, (B and G chockolate therapy with esspresso beans added)

Helmut10001 2y ago

I read somewhere that the eye transmits 10 Million bits per second to the brain. I think all of this is a matter of perspective.

titzer 2y ago

> More generally, the information throughput of human behavior is about 10 bits/s. I'm sorry, I just can't take this article seriously. They make a fundamental mistake of encoding and assume that information is discretized into word-sized or action-sized chunks. A good example is a seemingly discrete activity such as playing a musical instrument, like a guitar. A guitar has frets and strings, a seemingly small number of finite notes it can play. So it would seem a perfect candidate for discretization along the lines of the musical scale. But any guitar player or listener knows that a guitar is not a keyboard or midi synth: 1. The attack velocity and angle of the pick intones aggression and emotion, not just along a few prescribed lines like "an angry or sad or loud or quiet". 2. Timing idiosyncracies like being slightly before or after a beat, or speeding up or slowing down, or even arhythmic; the entire expression of a piece of music is changed by subtleties in phrasing. 3. Microbends. The analog nature of strings cannot be hidden entirely behind frets. Differences in the amount of pressure, how close to the fret the fingers are, and slight bending of the strings, intentional or unintentional, static or dynamic, change the pitch of the note. 4. Non-striking sounds like the amount of palming, pick scraping, tapping, and sympathetic vibrations. Of course there are lots of other things. All of these things make the difference between a master guitar player, say Hendrix, and someone just playing the same notes. And yes of course we can consider the encoding of the audio coming out of the guitar to be information--at a much higher bitrate, but what about the facial expressions, body language, etc? There are tons of channels coming off a musician, particularly live performances. This entire article just misses these in picking a quantized encoding of information that of course has a low bitrate. In short, they are missing bazillions of channels, not the least of which is expression and timing.

constantcrying 2y ago

It seems the authors conflate a problem bring easy to state with little processing power being needed to solve it. This obviously isn't true, very complex mathematical problems can be stated in very few bits. Human interactions often can be extremely complex, even though they are relatively slow. Reading a text isn't about matching symbols to words. It is about taking these words and putting them into a social context, potentially even doubting their content or imagining the inner world of the author. Obviously that is what the "inner" brain (which existence seems very dubious to me) has to do. I see absolutely no paradox at all.

devenson 2y ago

Thinking is emulated -- therefore much slower.

zelon88 2y ago

Human's can transfer up to 39 bit/s during normal speech, so I highly doubt that it's accurate to describe human "throughput" as being only 10 bit/s. https://www.science.org/content/article/human-speech-may-have-universal-transmission-rate-39-bits-second https://www.science.org/content/article/human-speech-may-hav...

mrguyorama 2y ago
How long a time can you continuously generate that 40bps? If you can only do that 1/4 the time, that's 10bps

nis0s 2y ago

I think the discussion on serial vs parallel processing is incomplete in the linked paper, and is one reason I think the 10 bits/s bottleneck is an incomplete or misinterpreted result. Here’s a review with sources on serial processing, https://journalofcognition.org/articles/10.5334/joc.185 https://journalofcognition.org/articles/10.5334/joc.185 > Cognitive psychology has mainly focused on structural and functional limitations of cognitive processes when facing multitasking requirements. Structural limitations assume strict serial processing for at least one processing stage, while functional limitations assume flexible, parallel processing only limited by the number of available resources. Human movement science, on the other hand, emphasizes the plasticity of cognition and training possibilities. As both approaches have provided ample empirical evidence for their views but have predominantly worked in isolation, this example clearly illustrates the need for a more integrative approach to multitasking. A challenge for the contemporary research on multitasking is to bring together the issues of structure, flexibility, and plasticity in human multitasking, offering a new integrative theoretical framework that accounts for this fundamental aspect of human behaviour. From one of the papers cited by the above reference (Hommel 2020), > A closer look reveals that the questions being asked in dual-task research are not particularly interesting or realistic, and the answers being given lack mechanistic detail. In fact, present theorizing can be considered mere empirical generalization, which has led to merely labeling processing bottlenecks rather than describing how they operate and how they actually produce the bottleneck. So, while I applaud the authors on generating buzz and discussion, I think their promising work will benefit from more serious consideration of the underlying neurophysiology.

stainablesteel 2y ago

there's probably lots of local computation and i think it's a mistake to simplify it all down to just one substrate or receptor

krzat 2y ago

Related question: what's the physical/biological difference between conscious and unconscious?

vok 2y ago

There's interesting discussion with the authors here: https://www.alphaxiv.org/abs/2408.10234 https://www.alphaxiv.org/abs/2408.10234

kubasienki 2y ago

https://arxiv.org/abs/2408.10234 https://arxiv.org/abs/2408.10234

_yb2s 2y ago

What would it really even mean for our brain to be fast or slow? Relative to what? Inherent speed does not matter and is ill defined, it only matters relative to the environmental processes we have to react to. We’re already orders of magnitude faster than processes like geology and weather, so it is really only other animals where speed matters, and since we’re all using the same basic hardware, we’re on a relatively level playing field there (except for house flies it seems, lol). Time as we understand it may be as much as a cultural construct as anything else (see, for example Timeless physics). Some neurodivergent people, and people from very different cultures don’t experience anything like what most people in our culture refer to as time. As for thinking about one thing at a time- I am absolutely certain this is false. Our subconscious operates massively parallel, and we only have one conscious thought at a time. But it had an interrupt system that lets it either act instantly itself in an emergency, and to bring things to consciousness when they are important. I’m sure everyone has had the experience of reacting quickly to an emergency with no conscious thought, or suddenly knowing the solution to a problem you hadn’t even been consciously thinking about.

Barrin92 2y ago

I skimmed over the paper so sorry if I didn't see it, but isn't the most obvious answer that (at least conscious) human decision making happens at a high level of abstraction? Information output or processing in terms of bits, say the numbers of words in a sentence, isn't equivalent to the meaning or effect that has in the world, which is what matters. If a general orders an army to go to war that's a few bits of text on a piece of paper, but obviously that bears no relation to the consequence of what that puts into motion in the real world. It's not like we spend most of our days speedcubing or writing and reading the dictionary. We aren't low level data processors even though that might happen unconsciously somewhere in the sensory system, but the conscious mind is a reasoning system. Even in artificial systems is that visible, from Deepmind on Alpha Zero (https://deepmind.google/discover/blog/alphazero-shedding-new-light-on-chess-shogi-and-go/ https://deepmind.google/discover/blog/alphazero-shedding-new...) "For each move, AlphaZero searches only a small fraction of the positions considered by traditional chess engines. In Chess, for example, it searches only 60 thousand positions per second in chess, compared to roughly 60 million for Stockfish." Not too draw to many parallels between the human brain and these systems, but they do obviously share the similarity that higher order conceptual decision making compared to just data processing will result in lower rates of decision making at at least the top level. That's for me what you'd expect to happen, not a paradox.

0xbadcafebee 2y ago

We process information at the rate we do because that's what works for adaptation to our environment. That's how all biological things develop. Some things are remnants of old adaptations, but generally speaking, we are the way we are because that's what enables our survival. If it seems "slow", it's because it doesn't need to be faster. Better questions would be, why are we so weak? Why are we defenseless? Why are we hairless? Why do we move so slow? Most other mammals our size are much better equipped for survival. Obviously other beings "perform better" in many ways. Yet our fleshy, weak, slow, gangly, shivery bodies are suited for thinking, for adapting, for communicating and collaborating. However unoptimal or "slow" these authors think our brains are, they are obviously perfectly capable for what they need to do, seeing as we dominate the entire planet. In any system design, every decision tends to be a tradeoff. You can gain CPU power, but it will cost you energy, heat, and probably stability or longevity. You often do not know what a "feature" will do until you add it. So it's wiser to only add the features you need to accomplish your immediate goals. If at some point in the future, our survival is existentially threatened by our lack of "processing power", our brains will adapt. Until then, leave well enough alone.

peterfirefly 2y ago

10 baud, perhaps. Definitely not 10 bits/s!

luxuryballs 2y ago

I’m interested in the phenomenon that no matter how messed up (or absent) my sensory input becomes it doesn’t seem to impact my “inner observer”. The only thing impacting it is sleep or loss of consciousness, yet even then there can be moments during these states that this observer can become lucid. It’s like the science of the soul and one notable implication is dementia, when people become unable to function and recognize things, I wonder if that “inner observer” is still just as in-tact as it is on a huge dose of LSD?

rbanffy 2y ago

I prefer to think of the inherent slowness of my organic brain as a leisurely stroll through time. I’m taking the scenic route though life.

kirkules 2y ago

The back of the envelope computations are shockingly shallow and meaningless. (20 Questions, from the intro) Trying to think of a thing for the game is not a search over a set of known things. Just saying the possibility set has size 2^N doesn't mean that choosing something in the set consists of processing the set. But even if that were the case, and if you do consider each of 2^N options, the consideration process itself is not trivial and probably varies wildly. (English typing) Touch typists do not (only) simply convert an existing/known string to a sequence of hand actions by mapping character to action. There are whole words and sequences that become units/tokens from the standpoint of muscle memory and processing (this will be relevant to the rubik's cube topic as well). When i type, there's a sort of planning and queueing of actions, but also there's monitoring of actions that allows fast error correction with pressing delete a number of times or holding it and costly determining when the error has been reached, and resuming afterward. Of course the process likely varies from person to person, but there's such a host of other things going on that should count as part of the information processed in this simple behavior that the example and numbers used in the paper for it are utterly useless even as estimates. (Rubik's cube blind speed solving) Again we see reference to the entire possibility space (from the perspective of possible configurations of the puzzle). But solvers do not identify the configuration they encounter with reference to the space, nor do they search the space for it. They look for patterns and ignore what they cannot use for the strategy they have practiced. The cuber often does not commit to memory the whole configuration, but will often convert it to a custom and bespoke mnemonic. It's just utter nonsense to refer to the number of possible configurations, it has nothing directly to do with what the human is doing. If I memorize a 30 word passage, i have not "processed the set of possible 30 word passages".

Jensson 2y ago

The answer is that we defined second after the time period it takes for us to think about something, if our thinking was faster the second would also be faster and we would still feel like we only think about about one thing per second.

byyoung3 2y ago

"Why can we only think about one thing at a time?" - this is not true? There is a difference between thinking and conscious thinking.

tim333 2y ago

Free access link https://arxiv.org/pdf/2408.10234 https://arxiv.org/pdf/2408.10234

kykeonaut 2y ago

So the human brain is a great operating system that is desperately missing a good text editor...

zogomoox 2y ago

"It is by will alone I set my mind in motion. It is by the juice of Sapho that thoughts acquire speed, the lips acquire stains, the stains become a warning. It is by will alone I set my mind in motion."

weddpros 2y ago

You can compress 10Gbit/s down to 10bit/s but then you can't do the reverse operation... Yeah, we know. To me it's like saying "I've summarised this book to 5 words, so why would you even need a few days to read it entirely?".

deleted 2y ago
[deleted]

ckemere 2y ago

Definitely a neat article insofar as it summarizes many observations about task related “bandwidth” of human output. Summarizing other comments - calculating bandwidth in a task-specific way neglects metacognition: the Rubik’s cube solver can rapidly recover if someone shoves them when their blindfold is on; the memory champion can notice a pie being aimed at their face and duck. I think that there’s still something interesting around our inability to multitask conscious behaviors, but the overarching conclusion is flawed.

effed3 2y ago

10"very-ultra-extreme-high-level"Bit/s (maybe)

leflores 2y ago

At what speed is the brain processing when a pair of Chinese ping pong masters are playing? All of this requires seeing the ball coming from the opponent, subconsciously calculating its speed, angle, and rotation to decide how to move your body, legs, arms, hands, and even fingers to hit the ball at the desired location, with the desired angle, and at the desired speed. A standard ping pong table is 2.74 meters long, and in high-speed professional games, the ball can traverse this distance in as little as 90–140 milliseconds. This means players have less than a tenth of a second to react. It's not just the immense image-processing speed required but also the mental speed to transform that visual data into precise positioning and movement of hundreds of muscles in less than a second. Ten bits per second? No way.

deleted 2y ago

[deleted]