4 ms·

Hearing is also well into the terabytes worth of information per year. Add in touch, taste, smell, proprioception, etc and the brain gets a deluge. The differ

by Retric 2y ago

Hearing is also well into the terabytes worth of information per year. Add in touch, taste, smell, proprioception, etc and the brain gets a deluge. The difference is we’re really focused on moving around in 3D space and more abstract work, where an LLM etc is optimized for a very narrow domain.

jdietrich 2y ago
>Hearing is also well into the terabytes worth of information per year. If we assume that the human auditory system is equivalent to uncompressed digital recording, sure. Actual neural coding is much more efficient, so the amount of data that is meaningfully processed after multiple stages of filtering and compression is plausibly on the order of tens of gigabytes per year; the amount actually retained is plausibly in the tens of megabytes. Don't get me wrong, the human brain is hugely impressive, but we're heavily reliant on very lossy sensory mechanisms. A few rounds of Kim's Game will powerfully reveal just how much of what we perceive is instantly discarded, even when we're paying close attention.
- Retric 2y ago
  The sensory information form individual hairs in the ear start off with a lot more data to process than simple digital encoding of two audio streams. Neural encoding isn’t particularly efficient from a pure data standpoint just an energy standpoint. A given neuron not firing is information and those nerve bundles contain a lot of neurons.
- PaulDavisThe1st 2y ago
  "That's a coyote" I say, based on auditory evidence. Pretty damn efficient coding.
hammock 2y ago
> Hearing is also well into the terabytes worth of information per year. Add in touch, taste, smell, proprioception, etc and the brain gets a deluge Is that supposed to be a lot? Only a small fraction of that is committed to permanent storage. A random server is today processing anywhere from tens of terabytes to hundreds of petabytes annually
- Retric 2y ago
  Low level details like that aren’t relevant to this discussion. Most human processing power is at the cellular level. The amount of processing power in a single finger literally dwarfs a modern data center, but we can’t leverage that to think only live. So it’s not a question of ‘a lot’ it’s a question of orders of magnitude vs “the quantities used to train AI” Library of congress has what 39 million books, tokenize every single one and you’re talking terabytes of training data for an LLM. We can toss blog posts etc to that pile but every word ever written by a person isn’t 20 orders of magnitude larger or anything.

by Retric 2y ago

jdietrich 2y ago

>Hearing is also well into the terabytes worth of information per year. If we assume that the human auditory system is equivalent to uncompressed digital recording, sure. Actual neural coding is much more efficient, so the amount of data that is meaningfully processed after multiple stages of filtering and compression is plausibly on the order of tens of gigabytes per year; the amount actually retained is plausibly in the tens of megabytes. Don't get me wrong, the human brain is hugely impressive, but we're heavily reliant on very lossy sensory mechanisms. A few rounds of Kim's Game will powerfully reveal just how much of what we perceive is instantly discarded, even when we're paying close attention.

Retric 2y ago
The sensory information form individual hairs in the ear start off with a lot more data to process than simple digital encoding of two audio streams. Neural encoding isn’t particularly efficient from a pure data standpoint just an energy standpoint. A given neuron not firing is information and those nerve bundles contain a lot of neurons.
PaulDavisThe1st 2y ago
"That's a coyote" I say, based on auditory evidence. Pretty damn efficient coding.

hammock 2y ago

> Hearing is also well into the terabytes worth of information per year. Add in touch, taste, smell, proprioception, etc and the brain gets a deluge Is that supposed to be a lot? Only a small fraction of that is committed to permanent storage. A random server is today processing anywhere from tens of terabytes to hundreds of petabytes annually

Retric 2y ago
Low level details like that aren’t relevant to this discussion. Most human processing power is at the cellular level. The amount of processing power in a single finger literally dwarfs a modern data center, but we can’t leverage that to think only live. So it’s not a question of ‘a lot’ it’s a question of orders of magnitude vs “the quantities used to train AI” Library of congress has what 39 million books, tokenize every single one and you’re talking terabytes of training data for an LLM. We can toss blog posts etc to that pile but every word ever written by a person isn’t 20 orders of magnitude larger or anything.