3 ms·
> We are probably going to need a lot more GPUs. Or a breakthrough in algorithms etc. The human brain, heck all bio brains, are proof that you don't need a lo
by Razengan 4mo ago
> We are probably going to need a lot more GPUs.
Or a breakthrough in algorithms etc.
The human brain, heck all bio brains, are proof that you don't need a lot of power or size for intelligence.
- altcognito 4mo ago20 watts for inference AND training!
- aeyes 4mo agoFor intelligence, I expect the next breakthrough to be colocation of memory and compute in the same chip. And we'll need much more of this memory, probably a few petabytes.
- ryandvm 4mo agoThe human brain has 80 billion neurons and a 100 trillion synapses. I think you're underselling the processing power of that warm chunk of meat. The real message of the last 15 years has actually been the opposite: if you throw enough processing power at it, intelligence emerges.
- thunky 4mo agoI think you're helping GPs point: there is a lot of efficiency gains to be made to match the processing power of the brain, given it's size and power draw.
- dbspin 4mo agoMoreover we've known for quite a while now that glial cells also participate in cognition and moderate learning (e.g.: [1]). When you take those connections into account the numbers get really staggering. 85 billion glial cells with trillions of protein channels facilitating communication between the glial syncytium [2]. [1] https://www.sciencedirect.com/science/article/pii/S1934590913000076 https://www.sciencedirect.com/science/article/pii/S193459091... [2] https://pmc.ncbi.nlm.nih.gov/articles/PMC5063692/ https://pmc.ncbi.nlm.nih.gov/articles/PMC5063692/
- deleted 4mo ago[deleted]
- orbital-decay 4mo agoThe real question is not how many "weights" the human brain has (neurons+synapses may or may not translate into "weights", and brain might be also inefficient for what it is), but rather how much evolutionary and social "compute" was necessary to pack everything into that capacity.
- baq 4mo agoIt’s still 20W. We have living proof what is possible within 20W. The message has always been clear - try to get silicon computer to be as power efficient as the brain is as it is obviously possible.
- torginus 4mo agoI keep thinking about the cyborg chicken mechs in Metal Gear Solid 4. If stuff like Neuralink gets advanced enough, we might find that cutting up animals and sticking electrodes in their brains is the best way to make robots. Yeah, people might object, but it can be argued that we are already subjecting scores of animals to horrors beyond comprehension just to get a bucket of chicken wings. And even if we manage to get silicon to do what brains do, it will likely cost 1000x as much and consume 1000x the power like you said. It's hell of an economic incentive.