4 ms·

The human neocortex has 20B neurons, averaging 10K connections each, which is about 200T connections total. This model is only a few orders of magnitude away fr

The human neocortex has 20B neurons, averaging 10K connections each, which is about 200T connections total. This model is only a few orders of magnitude away from that, and it's already performing really well in its narrow category. Equating model 'parameters' to interneuron connections in naïve at best (and a horrible measure in general). All I'm trying to say is I find it crazy how dang big these models are getting.

tralarpa 5y ago
> This model is only a few orders of magnitude away from that I think you can add several orders of magnitude to that since nerve cells are more like microcontrollers (with memory, adaptation etc.) than simple nodes. I remember a scientific article that made a big impression on me: when a dragon fly sees a prey, only 8 neurons (connected to the eyes and the wings) are responsible for keeping it oriented toward the target.
- azeirah 5y ago
  I'm not educated well enough to really agree or disagree with your idea that we should be adding several orders of magnitude to the estimation. But I did encounter this article a while ago here on HN. Only two neurons are necessary to ride a bicycle. http://paradise.caltech.edu/cook/papers/TwoNeurons.pdf http://paradise.caltech.edu/cook/papers/TwoNeurons.pdf
- lostmsu 5y ago
  For all we know these microcontrollers may be that complicated only because they try to emulate discrete logic, and you actually need hundreds of them to make a single unit reliable enough for that purpose.
YeGoblynQueenne 5y ago
>> This model is only a few orders of magnitude away from that, and it's already performing really well in its narrow category. A few orders of magnitude and an entire category away. Artificial "neurons" only have the name "neuron" in common with biological neurons. Consequently you can stack as many layers of artificial neurons on top of each other as you may want and you won't get anywhere near the abilities of the simplest systems of biological neurons. For example, spiders have ~100 thousand neurons and there's no artificial neural networks that could shake a stick at a spider's cognitive abilities. Which are downright scary, btw. Estimated number of neurons of spiders from wikipedia: https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...
- melony 5y ago
  Spiking ones need to solve differential equations and our current hardware designs are too discrete for it to be an efficient strategy (barring an algorithmic breakthrough).
- freemint 5y ago
  I think AI is actuator constrained compared to a spider which might be the larger problem.

tralarpa 5y ago

> This model is only a few orders of magnitude away from that I think you can add several orders of magnitude to that since nerve cells are more like microcontrollers (with memory, adaptation etc.) than simple nodes. I remember a scientific article that made a big impression on me: when a dragon fly sees a prey, only 8 neurons (connected to the eyes and the wings) are responsible for keeping it oriented toward the target.

azeirah 5y ago
I'm not educated well enough to really agree or disagree with your idea that we should be adding several orders of magnitude to the estimation. But I did encounter this article a while ago here on HN. Only two neurons are necessary to ride a bicycle. http://paradise.caltech.edu/cook/papers/TwoNeurons.pdf http://paradise.caltech.edu/cook/papers/TwoNeurons.pdf
lostmsu 5y ago
For all we know these microcontrollers may be that complicated only because they try to emulate discrete logic, and you actually need hundreds of them to make a single unit reliable enough for that purpose.

YeGoblynQueenne 5y ago

>> This model is only a few orders of magnitude away from that, and it's already performing really well in its narrow category. A few orders of magnitude and an entire category away. Artificial "neurons" only have the name "neuron" in common with biological neurons. Consequently you can stack as many layers of artificial neurons on top of each other as you may want and you won't get anywhere near the abilities of the simplest systems of biological neurons. For example, spiders have ~100 thousand neurons and there's no artificial neural networks that could shake a stick at a spider's cognitive abilities. Which are downright scary, btw. Estimated number of neurons of spiders from wikipedia: https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...

melony 5y ago
Spiking ones need to solve differential equations and our current hardware designs are too discrete for it to be an efficient strategy (barring an algorithmic breakthrough).
freemint 5y ago
I think AI is actuator constrained compared to a spider which might be the larger problem.