3 ms·
Yeah, IMO if you want a rough idea about how much compute the human brain performs, and how efficiently you could do that in silicon, it's interesting to look a
by snek_case 4y ago
Yeah, IMO if you want a rough idea about how much compute the human brain performs, and how efficiently you could do that in silicon, it's interesting to look at area V1 which is one of the lowest level areas involved in vision. This brain area comprises 140 million neurons per hemisphere (so roughly 280 million in total). So that's about 1/300 of the human brain.
The interesting thing here is, what V1 does is it computes directional receptive fields based on raw data from the retinas. We can implement that in a tiny silicon chip, maybe even just a DSP chip, not a full CPU. We know how to implement this kind of computation super efficiently, and without using neural networks.
We don't fully understand what goes on in many areas of the brain, but if we can find efficient ways to implement equivalent computations, we might technically already have sufficient manufacturing technology to implement equivalent functionality in a pretty compact and power-efficient form factor, or we might not be that far from there.
A classic quote from Dijkstra: "The question of whether machines can think is about as relevant as the question of whether submarines can swim."
- skohan 4y agoI have my doubts about whether just scaling a model of V1 would be sufficient to reproduce consciousness. Structure and function are extremely closely linked in the nervous system, and neural circuitry is highly specialized in terms of structure to support a given function. E.g., motor cortex is going to be vastly different from auditory cortex and so on. Assuming that because we have 1st level feature detection in V1 fairly well understood and reproduced, we have solved neuroscience in general strikes me as a massive and unjustified leap of faith. If there were a brain region that might be emulated to reproduce "general information processing", the best candidate would probably be neocortex. As far as I am aware, that does seem to be a very wide region of the brain which has fairly consistent structure, and largely serves to integrate all other systems.
- snek_case 4y agoNo, scaling a model of V1 would not yield consciousness. The point is that V1 is a large part of the visual cortex. It's something the brain dedicates a lot of resources to, and we could implement something similar in silicon almost trivially. Your smartphone GPU may have enough compute to simulate an equivalent computation at 200 frames per second. It's possible that in the future, we'll be able to simulate consciousness very efficiently once we have a better handle on the kinds of computations that are involved, much more efficiently than what the brain does.
- skohan 4y agoAt what level of fidelity? 1 synapse == 1 16-bit floating point value? It may be enough (emphasis on may), for a specific brain region which is very directional in nature but I'm not sure if that calculation will generalize.
- Thiez 4y agoThe brain is a pretty messy place. While our bodies try hard to preserve homeostasis, conditions will always vary slightly. Hormones affect our brains and can vary in concentration from moment to moment. Type 1 diabetics can easily have their blood sugar increase by a factor of 4, without losing consciousness. There are many other chemicals and conditions (temperature?) that similarly influence the condinions under which our neurons must operate. But even in this noisy environment, our brains can usually operate just fine. Given this level of background noise, I would be very surprised if the average synapse would require more than a 16-bit floating point value for its outputs.
- skohan 4y agoCounter argument: Some synapses contain multiple receptor sub-types. Dopamine, for example, has some receptors which are inhibitory, and others which are excitatory. Each receptor is sensitive to different concentrations of dopamine. This gives the effect that when the dopamine concentrations in the synapse are low, this synapse acts to inhibit the activity of the efferent neuron, while when the concentration reaches a certain threshold, the output of the synapse flips to an excitatory signal. How would you model that system with a single F16? And that's just one example: for instance there are specific signals which trigger cascading effects in the neuron which up-regulate certain chemical pathways, causing an amplifying effect on other signals. Or you have shunting effects, where an inhibitory synapse placed at a certain point in the dendritic arbor serves to selectively cancel signals from farther down that one specific branch of the dendrite. That's not "noise" - the brain has incredibly refined and detailed mechanisms for information processing which go very far beyond `weight * sigmoid(activation) = output`