35 ms·

Comparing a human brain in these terms makes it incredibly obvious how inefficient the human brain actually is. A 1.7T model can answer questions about practica

Comparing a human brain in these terms makes it incredibly obvious how inefficient the human brain actually is. A 1.7T model can answer questions about practically anything. You say a human brain has 150T params. So what? It struggles to give masterful answers in even 1 domain, let alone dozens/hundreds. We need to stop comparing parameters and synapses as if they actually matter, because AFAIK, they really don't.

moffkalast 2y ago
Well once again it turns out that what is hard for people is easy for computers, and vice versa. The things we go to college for 6 years for they can (relatively) master in a week of pretraining. We are optimized to smartly kill things, eat them, and reproduce, that's what machines will beat us at last lol. Right now a human expert is still obviously better in depth, but nowhere close in breadth. Probably not for much longer though, at least on the historical time scale. And granted a lot of parts of the human brain are dedicated to specific jobs that are entirely nonexistent in a normal LLM (kinematics, vision, sound, touch, taste, smell, autonomic organ control) so the actual bit we should be comparing for just language and reasoning is way smaller. Still the brain is pretty efficient inference energy wise, it's like the ultimate mixture of experts, extreme amounts of sparse storage and most of it is not computed until needed. The router must be pretty good.
jampekka 2y ago
OTOH humans can e.g. walk on two feet and drive a car.
- fragmede 2y ago
  Boston Dynamics and Waymo might not have gotten human levels of competency with those two particular tasks, but we've already got robots that are better than drunk/tired/angry humans at it, and they're getting better at it.
Jensson 2y ago
> Comparing a human brain in these terms makes it incredibly obvious how inefficient the human brain actually is Until you have AGI you can't say this since until then we don't know how much the different parts costs to replace with AI systems.
tim333 2y ago
In terms of energy use the human brain is way more efficient than LLMs. It's a completely different hardware model - the brain may have trillions of synapses but they only fire occasionally. I agree you have to compare more on the results than number of synapses etc. It's something that gives me pause on the idea that we have to build many GW of power stations. It may be possible to get much more energy efficient AI via better algorithms.
onemoresoop 2y ago
> Comparing a human brain in these terms makes it incredibly obvious how inefficient the human brain actually is. A 1.7T model can answer questions about practically anything. You say a human brain has 150T params. So what? It struggles to give masterful answers in even 1 domain, let alone dozens/hundreds. We need to stop comparing parameters and synapses as if they actually matter, because AFAIK, they really don't. Meanwhile the best class AI is trained on the output of human intelligence. When AI can learn by itself in the same way humans do (or even more efficient ways) that's when we can say human intelligence has been surpassed. Until then it's just tools for humans to use to augment their intelligence.

moffkalast 2y ago

Well once again it turns out that what is hard for people is easy for computers, and vice versa. The things we go to college for 6 years for they can (relatively) master in a week of pretraining. We are optimized to smartly kill things, eat them, and reproduce, that's what machines will beat us at last lol. Right now a human expert is still obviously better in depth, but nowhere close in breadth. Probably not for much longer though, at least on the historical time scale. And granted a lot of parts of the human brain are dedicated to specific jobs that are entirely nonexistent in a normal LLM (kinematics, vision, sound, touch, taste, smell, autonomic organ control) so the actual bit we should be comparing for just language and reasoning is way smaller. Still the brain is pretty efficient inference energy wise, it's like the ultimate mixture of experts, extreme amounts of sparse storage and most of it is not computed until needed. The router must be pretty good.

jampekka 2y ago

OTOH humans can e.g. walk on two feet and drive a car.

fragmede 2y ago
Boston Dynamics and Waymo might not have gotten human levels of competency with those two particular tasks, but we've already got robots that are better than drunk/tired/angry humans at it, and they're getting better at it.

Jensson 2y ago

> Comparing a human brain in these terms makes it incredibly obvious how inefficient the human brain actually is Until you have AGI you can't say this since until then we don't know how much the different parts costs to replace with AI systems.

tim333 2y ago

In terms of energy use the human brain is way more efficient than LLMs. It's a completely different hardware model - the brain may have trillions of synapses but they only fire occasionally. I agree you have to compare more on the results than number of synapses etc. It's something that gives me pause on the idea that we have to build many GW of power stations. It may be possible to get much more energy efficient AI via better algorithms.

onemoresoop 2y ago

> Comparing a human brain in these terms makes it incredibly obvious how inefficient the human brain actually is. A 1.7T model can answer questions about practically anything. You say a human brain has 150T params. So what? It struggles to give masterful answers in even 1 domain, let alone dozens/hundreds. We need to stop comparing parameters and synapses as if they actually matter, because AFAIK, they really don't. Meanwhile the best class AI is trained on the output of human intelligence. When AI can learn by itself in the same way humans do (or even more efficient ways) that's when we can say human intelligence has been surpassed. Until then it's just tools for humans to use to augment their intelligence.