3 ms·
Aren't we there yet? George Hotz has a nice series of posts on this: https://geohot.github.io/blog/jekyll/update/2020/08/20/a-silicon-person.html https://geoh
by vrnvu 2y ago
Aren't we there yet?
George Hotz has a nice series of posts on this:
https://geohot.github.io/blog/jekyll/update/2020/08/20/a-silicon-person.html https://geohot.github.io/blog/jekyll/update/2020/08/20/a-sil...
https://geohot.github.io/blog/jekyll/update/2022/02/17/brain-flops.html https://geohot.github.io/blog/jekyll/update/2022/02/17/brain...
https://geohot.github.io/blog/jekyll/update/2023/04/26/a-person-of-compute.html https://geohot.github.io/blog/jekyll/update/2023/04/26/a-per...
I sorted them by date.
- nicklecompte 2y ago[flagged]
- deleted 2y ago[deleted]
- abeppu 2y agoHmm, this reasoning is making a lot of really questionable assumptions: > We’ll use the estimates of 100 billion neurons and 100 trillion synapses for this post. That’s 100 teraweights. ... or maybe actual synapses cannot be described with a single weight. > The max firing rate seems to be 200hz [4]. I really want an estimate of “neuron lag” here, but let’s upper bound it at 5ms. ... but ANNs output float activations per pass. Biological neurons encode values with sequences of spikes which vary in timing, so the firing rate doesn't on its own tell you the rate at which neurons can communicate new values. > Remember also, that the brain is always learning, so it needs to be doing forward and backward passes. I’m not exactly sure why they are different, but [6] and [7] point to the backward pass taking 2x more compute than the forward pass. ... but the brain probably isn't doing backprop, in part because it doesn't get to observe the 'correct' output, compute a loss, etc, and because the brain isn't a DAG.
- vrnvu 2y agoYeah, fore sure. I just share it as a fun read. I think they have been discussed in HN before.