4 ms·
Lot's of assumptions here, like for example this: > Assuming each neuron has a fan-out of 1000, aka 1000 MACs, that’s 2000 FLOPS per neuron Human neurons are
by tener 4y ago
Lot's of assumptions here, like for example this:
> Assuming each neuron has a fan-out of 1000, aka 1000 MACs, that’s 2000 FLOPS per neuron
Human neurons are not built like we build the artificial neural networks. They may have complex time based behavior ("memory") and dendrites are quite complex, very far away from 2 FLOP per one.
- andai 4y agoThere was an article a while back that showed that a human neuron and artificial neural network neuron are not equivalent: it takes an entire network to simulate the behavior of a single human neuron.
- murbard2 4y agoBut it takes an entire network of human neurons to simulate the behavior of an artificial one. There's overhead in translation, but that doesn't by itself mean one is superior to the other as a basic unit.
- water-your-self 4y agoNeural network is an atrocious misnomer. The only similarity ive seen is between CNNs and vision. Thanks alex
- d_tr 4y agoA single biological brain neuron is a rich dynamical system with numerous patterns and transitions, and then you have different types of neurons where these patterns differ to some degree, and this is all asynchronous and modulated by various chemicals. Trying to translate this to FLOPS might be a fun game but I cannot see a real use for any such figure. As for the 3090 that the post mentions, its TFLOPS will drop like a stone once you start deviating from matrix multiplications and im2col towards some more complex structure...
- Veedrac 4y agoThe number you quoted was 2 FLOPs per synapse, not per neuron. A neuron is complicated and hierarchical, but it is unreasonable to say it's much more complex than an operation per synapse.