4 ms·

By no means rigorous, an estimate for the number of neurons is around 10^10 [0] with an average number of connections estimated at around 8000 [1]. Call 10^10

By no means rigorous, an estimate for the number of neurons is around 10^10 [0] with an average number of connections estimated at around 8000 [1]. Call 10^10 \approx 2^40 for convenience, and 8000 \approx 2^13, which gives us a 2^53 entropy estimate, or about a petabyte of information as an estimate of what the human brain can store (discounting more exotic theories of memory stored in DNA or some such). [0] https://en.wikipedia.org/wiki/Human_brain#Microanatomy https://en.wikipedia.org/wiki/Human_brain#Microanatomy [1] https://psychology.stackexchange.com/questions/7967/how-many-synapses-in-the-average-human-brain https://psychology.stackexchange.com/questions/7967/how-many...

edulix 4y ago
Add to that that our neurons are more complex to point neurons used in typical Artificial neural-nets. A single pyramid neuron in the neocortex might be more comparable to a multilayer neural net. https://www.biorxiv.org/content/10.1101/2021.10.25.465651v1.full.pdf https://www.biorxiv.org/content/10.1101/2021.10.25.465651v1....
- pmoriarty 4y ago
  Their communication is much more complex too, and neither they nor their connections are static. We don't understand how they work at the subatomic level simply because human understanding of the subatomic world is not complete, but even just at the atomic level a single neuron is massively more complex than anything humans have created. Going up to the molecular level, even that is staggeringly more complex than the incredibly simple abstractions that make up a neural net. Is what happens in the brain at the molecular, atomic, or subatomic levels relevant or necessary to intelligence and consciousness? We just don't know yet, but we do know all of that is far more complex and very different from the simple abstractions that are used for neural nets and LLMs. The back of a napkin calculations in this thread don't even begin to do justice to the tremendous amount of "calculation" or "storage" that happens in the human brain.
fnordpiglet 4y ago
Although I’d note this is more like a 1pb stored model.
deleted 4y ago
[deleted]

edulix 4y ago

Add to that that our neurons are more complex to point neurons used in typical Artificial neural-nets. A single pyramid neuron in the neocortex might be more comparable to a multilayer neural net. https://www.biorxiv.org/content/10.1101/2021.10.25.465651v1.full.pdf https://www.biorxiv.org/content/10.1101/2021.10.25.465651v1....

pmoriarty 4y ago
Their communication is much more complex too, and neither they nor their connections are static. We don't understand how they work at the subatomic level simply because human understanding of the subatomic world is not complete, but even just at the atomic level a single neuron is massively more complex than anything humans have created. Going up to the molecular level, even that is staggeringly more complex than the incredibly simple abstractions that make up a neural net. Is what happens in the brain at the molecular, atomic, or subatomic levels relevant or necessary to intelligence and consciousness? We just don't know yet, but we do know all of that is far more complex and very different from the simple abstractions that are used for neural nets and LLMs. The back of a napkin calculations in this thread don't even begin to do justice to the tremendous amount of "calculation" or "storage" that happens in the human brain.

fnordpiglet 4y ago

Although I’d note this is more like a 1pb stored model.

deleted 4y ago

[deleted]