2 ms·
> Something in there must be running fast, right? Nah. It just needs to be really wide. This is a very fuzzy comparison, but a human brain has ~100 trillion sy
by Tuna-Fish 2y ago
> Something in there must be running fast, right?
Nah. It just needs to be really wide. This is a very fuzzy comparison, but a human brain has ~100 trillion synaptic connections, which are the closest match we have to "parameters" in AI models. The largest such models currently have on the order of ~2 trillion parameters. (edit to add: and this is a low end estimate of the differences between them. There might be more stuff in neurons that effectively acts as parameters, and should be counted as such in a comparison.)
So AI models are still at least two orders of magnitude off from humans in pure width. In contrast, they run much, much faster.