5 ms·

> If the questions are properly designed, each will reveal 1 bit of information about the mystery thing. If the guesser wins routinely, this suggests that the t

> If the questions are properly designed, each will reveal 1 bit of information about the mystery thing. If the guesser wins routinely, this suggests that the thinker can access about 2^20 ≈ 1 million possible items in the few seconds allotted. So the speed of thinking – with no constraints imposed – corresponds to 20 bits of information over a few seconds: a rate of 10 bits per second or less. This is an extrinsic definition of "information" which is task relative, and has little to do with any intrinsic processing rate (if such a thing can even be defined for the imagination). The question of why does biological hardware capable of very high "intrinsic rates" deliver problem solving at "very low extrinsic rates" seems quite trivial. Its even a non-sequitur to compare them: properties of the parts are not properties of wholes. "Why does a gas move at 1 m/s, when its molecules move at 1000s m/s..." All the 'intrinsic processing' of intelligence is concerned with deploying a very large array of cognitive skills (imagination, coordination, planning, etc.) that are fully general. Any given task has requires all of those top be in operation, and so we expect a much slower rate of 'extrinsic information processing'. Consider how foolish the paper is to compare the intrinsic processing of a wifi network with the extrinsic task-specific processing of a human: it is likewise the case that if we set a computer the challenge of coordinating the solution of a task (eg., involving several LLMs) across a network, it's task-specific performance would drop off a cliff -- having a much slower 'solution rate' than 10bit/second. These 'task-specific bits' represent a vast amount of processing work to solve a problem. And are at least as much to do with the problem, than the system solving it. It seems to me all this paper does is define tasks in a highly abstract way that imposes a uniform cost to process '1 bit of task information'. Do the same for computers, and you'd likewise find tiny bitrates. The rate at which a problem is solved is 'one part of that problem per second' for a suitable definiton of 'part'

psb217 2y ago
Another relevant point is the common anecdote about, eg, some master engineer who gets paid big bucks to come fix some problem that's been blocking up a factory for weeks. The engineer walks around, listens to a few of the machines, and then walks up to one of the machines and knocks it with his elbow Fonzi style and the factory starts running again. The factory boss is happy his factory is working, but annoyed that he paid so much for such an "easy" solution. Ie, the amount of input and processing required to produce the "right" 10 bits might be far larger than 10 bits. Another obvious example is chess. The amount of bits conveyed by each move is small but, if you want to make the right move, you should probably put some deeper thought into it. Humans are essentially organisms that collect and filter information, boil it down to a condensed soup of understanding, and emit a light sprinkle of carefully chosen bits intended to reshape the future towards their desires.
- pixl97 2y ago
  Humans are nature's best designed filters. Or another way of saying it is, the answer was right there all along, the hard part was filtering all the non-answer out.
  - ccozan 2y ago
    filtering and _acting_ on this condensed information. For example: lighting strucks a tree, a fire starts. Man is scared but that night is very cold and near the tree is warmer. This happens a few times, und a branch falls and it is collected , incidentally is thrown on another pile of wood , starts burning -> idea of fire is formulated and since them man keeps warm. Or: man finds shiny things in a river bed, collects them, one day the whole shack burns from lighting, and discovers that the shiny thigs are now in a different shape -> metal working is born.
pizlonator 2y ago
I was about to say this but you beat me to it. Seems like this 10 number comes out of the kind of research where the objective isn’t to find the truth, but to come up with an answer that is headline grabbing. It’s the scientific equivalent of clickbait. Too bad people fall for it.
fwip 2y ago
Exactly. English text is thought to have about 10 bits per word of information content, yet you can read much more quickly than 1 word per second. That includes not just ingesting the word, but also comprehending the meaning the author is conveying and your own reflections on those words.
Ghostt8117 2y ago
This type of comment is my least favorite on HN. "Seems quite trivial," "non-sequitur to compare them," "foolish." I am not able to read the paper as I do not have access, but the published perspective has 131 citations which seem to consider everything from task-specific human abilities, to cortical processing speeds, to perception and limb movements and eye movements, and so on. I'm glad you thought about it too, but to assume that the authors are just silly and don't understand the problem space is really not a good contribution to conversation.
- cscheid 2y ago
  (Disclosure: I’m a former academic with more than a handful of papers to my name) The parent comment is harshly criticizing (fairly, in my view) a paper, and not the authors. Smart people can write foolish things (ask me how I know). It’s good, actually, to call out foolishness, especially in a concrete way as the parent comment does. We do ourselves no favors by being unkind to each other. But we also do ourselves no favors by being unnecessarily kind to bad work. It’s important to keep perspective.
  - Ghostt8117 2y ago
    I realized that I do have institutional access and so I was able to read the paper, and I stand by my initial criticism of the above comment. "It seems to me all this paper does is define tasks in a highly abstract way that imposes a uniform cost to process '1 bit of task information'." The paper uses this number and acknowledges that it is not the only possible measure, and explains why they use this number and how it was derived. It is just the start of the paper, not "all this paper does." The paper primarily focuses on counterarguments to this number to then address the primary question of the relationship between the inner and outer brain. A few questions it poses: does the superior colliculus contribute to a bottom-up "saliency map" to ultimately direct the attentional bottleneck in cognition? Why does the brain use the same neural circuitry for both rapid/parallel sensory processing and slow/serial cognition? This is not even how other parts of the body work (e.g., type I and II muscle fibers). Perhaps the associated routing machinery between input and output accounts for the billions of neurons? Maybe, like the visual cortex, the prefrontal cortex has a fine-grained organization of thousands of small modules each dedicated to a specific microtask? We do ourselves the most favors by reading research with some skepticism, and asking questions. We do ourselves no favors by writing comments after only reading an abstract (please, tell me if I'm wrong). I only point out that discounting research so blithely does nothing for improving research. This was a perspective paper - an author asking questions to better understand a possible issue and guide research. And maybe the commenter is right, maybe this is the wrong focus, but I do not believe it was truly considered.

psb217 2y ago

Another relevant point is the common anecdote about, eg, some master engineer who gets paid big bucks to come fix some problem that's been blocking up a factory for weeks. The engineer walks around, listens to a few of the machines, and then walks up to one of the machines and knocks it with his elbow Fonzi style and the factory starts running again. The factory boss is happy his factory is working, but annoyed that he paid so much for such an "easy" solution. Ie, the amount of input and processing required to produce the "right" 10 bits might be far larger than 10 bits. Another obvious example is chess. The amount of bits conveyed by each move is small but, if you want to make the right move, you should probably put some deeper thought into it. Humans are essentially organisms that collect and filter information, boil it down to a condensed soup of understanding, and emit a light sprinkle of carefully chosen bits intended to reshape the future towards their desires.

pixl97 2y ago
Humans are nature's best designed filters. Or another way of saying it is, the answer was right there all along, the hard part was filtering all the non-answer out.
- ccozan 2y ago
  filtering and _acting_ on this condensed information. For example: lighting strucks a tree, a fire starts. Man is scared but that night is very cold and near the tree is warmer. This happens a few times, und a branch falls and it is collected , incidentally is thrown on another pile of wood , starts burning -> idea of fire is formulated and since them man keeps warm. Or: man finds shiny things in a river bed, collects them, one day the whole shack burns from lighting, and discovers that the shiny thigs are now in a different shape -> metal working is born.

pizlonator 2y ago

I was about to say this but you beat me to it. Seems like this 10 number comes out of the kind of research where the objective isn’t to find the truth, but to come up with an answer that is headline grabbing. It’s the scientific equivalent of clickbait. Too bad people fall for it.

fwip 2y ago

Exactly. English text is thought to have about 10 bits per word of information content, yet you can read much more quickly than 1 word per second. That includes not just ingesting the word, but also comprehending the meaning the author is conveying and your own reflections on those words.

Ghostt8117 2y ago

This type of comment is my least favorite on HN. "Seems quite trivial," "non-sequitur to compare them," "foolish." I am not able to read the paper as I do not have access, but the published perspective has 131 citations which seem to consider everything from task-specific human abilities, to cortical processing speeds, to perception and limb movements and eye movements, and so on. I'm glad you thought about it too, but to assume that the authors are just silly and don't understand the problem space is really not a good contribution to conversation.

cscheid 2y ago
(Disclosure: I’m a former academic with more than a handful of papers to my name) The parent comment is harshly criticizing (fairly, in my view) a paper, and not the authors. Smart people can write foolish things (ask me how I know). It’s good, actually, to call out foolishness, especially in a concrete way as the parent comment does. We do ourselves no favors by being unkind to each other. But we also do ourselves no favors by being unnecessarily kind to bad work. It’s important to keep perspective.
- Ghostt8117 2y ago
  I realized that I do have institutional access and so I was able to read the paper, and I stand by my initial criticism of the above comment. "It seems to me all this paper does is define tasks in a highly abstract way that imposes a uniform cost to process '1 bit of task information'." The paper uses this number and acknowledges that it is not the only possible measure, and explains why they use this number and how it was derived. It is just the start of the paper, not "all this paper does." The paper primarily focuses on counterarguments to this number to then address the primary question of the relationship between the inner and outer brain. A few questions it poses: does the superior colliculus contribute to a bottom-up "saliency map" to ultimately direct the attentional bottleneck in cognition? Why does the brain use the same neural circuitry for both rapid/parallel sensory processing and slow/serial cognition? This is not even how other parts of the body work (e.g., type I and II muscle fibers). Perhaps the associated routing machinery between input and output accounts for the billions of neurons? Maybe, like the visual cortex, the prefrontal cortex has a fine-grained organization of thousands of small modules each dedicated to a specific microtask? We do ourselves the most favors by reading research with some skepticism, and asking questions. We do ourselves no favors by writing comments after only reading an abstract (please, tell me if I'm wrong). I only point out that discounting research so blithely does nothing for improving research. This was a perspective paper - an author asking questions to better understand a possible issue and guide research. And maybe the commenter is right, maybe this is the wrong focus, but I do not believe it was truly considered.