3 ms·
""" Conclusions The brain is about as efficient as any conventional learning machine can be given: 1. An energy budget of 10W 2. A thermodynamic cooling con
by telotortium 5y ago
"""
Conclusions
The brain is about as efficient as any conventional learning machine can be given:
1. An energy budget of 10W
2. A thermodynamic cooling constrained surface power density similar to that of earth's surface (1kW/m^2), and thus a 10cm radius.
3. A total training dataset of about 10 billion precepts or 'steps'
If we only knew the remaining secrets of the brain today, we could train a brain-sized model consisting of a small population of about 1000 agents/sims, running on about as many GPUs, in probably about a month or less, for about $1M. This would require only about 1kW per agent or less, and so if the world really desired it, we could support a population of billions of such agents without dramatically increasing total world power production.
Nvidia - the single company producing most of the relevant flops today - produced roughly 5e21 flops of GPU compute in 2021, or the equivalent of about 5 million brains, perhaps surpassing the compute of the 3.6 million humans born in the US. With 200% growth in net flops output per year from all sources it will take about a decade for net GPU compute to exceed net world brain compute.
Eventually advances in software and neuromorphic computing should reduce the energy requirement down to brain levels of 10W or so, allowing for up to a trillion brain-scale agents at near future world power supply, with at least a concomitant 100x increase in GDP. All of this without any exotic computing.
Achieving those levels of energy efficiency will probably require brain-like neuromorphic-ish hardware, circuits, and learned software via training/education. The future of AGI is to become more like the brain, not less.
"""