4 ms·
John von Neumann wrote an essay on the topic titled The computer and the brain, which is quite a good comparison between the two types of systems, even though k
by kaba0 5y ago
John von Neumann wrote an essay on the topic titled The computer and the brain, which is quite a good comparison between the two types of systems, even though knowledge of the two was pretty primitive at the time. The basic idea is that computers are multiple orders of magnitude faster at serial calculations, but brains offset this difference by the sheer number of “dumb” processing units, with insane number of interconnections. Also, I don’t think that comparing the training of a neural network to the brain is fair from an energy usage point of view - compare the usage of the final NN with it.
As for how optimized is the human brain, well good question. I think not even a single biological cell is close to efficient, at most it is at a local minima. The reason is perhaps that “worse is better” in terms of novel functionality. But I don’t think there was a big evolutionary pressure on sufficient intelligence once it emerged - it is sort of a first past the post wins all.
- skohan 5y ago> John von Neumann wrote an essay on the topic titled The computer and the brain I have to be honest, I would take any such comparison from the 1950's with a huge pinch of salt. I think perceptions about how "dumb" an individual neuron is as a processing unit have shifted quite a bit since then. > Also, I don’t think that comparing the training of a neural network to the brain is fair from an energy usage point of view - compare the usage of the final NN with it. I'm not considering this in terms of the training efficiency, I'm looking at it in terms of the ratio between operational utility and energy used. There's no trained ANN with anything remotely close to the overall utility of the human brain at any scale, let alone one that weighs 3lbs, fits into a human skull and runs on 20 Watts of power.
- kaba0 5y agoI only mentioned that essay because I think the fundamental vision of it is still correct — in serial computations silicone beats “meat” hands down. And that is in both power efficiency and performance. The fundamental difference between our current approach and biological brains is just as much a hardware one as it is theoretical. CPUs and GPUs are simply not best fit for this sort of usage — a “core” of them is way too powerful for what a single neuron can do (even with the more correct belief that they are not as dumb as we first thought), even if they can calculate multiple ones simultaneously. I’m not sure of specifics but couldn’t we print a pre-trained NN to a circuit that could match/beat a simple biological neural network in both speed and power efficiency? Cells are inefficient.
- skohan 5y ago> in serial computations silicone beats “meat” hands down. And that is in both power efficiency and performance. I just don't think this is a meaningful comparison, and I'm not convinced it's evidence of the "limitations" of biological computation. Silicone beats biology in doing binary computation because they're a single-purpose machine built for this task. But a brain is capable of serving as a control system to operate millions of muscle fibers in parallel to navigate the body smoothly in unpredictable 3D space, while at the same time modulating communication to find the right way to express thoughts and advance interests in complex and uncertain social hierarchies, while at the same forming opinions about pop-culture, composing sonnets, falling in love and contemplating death. For me to buy the argument that ANN's can be more efficient than biology, you'd have to show me a computer which can do all of that using less resources than the human brain. Currently we have an assembly line for math problems. > a “core” of them is way too powerful for what a single neuron can do I just think you're vastly under-counting the complexity of what happens inside a single neuron. At every synapse, there's a complex interplay of chemistry, physics and biology which constitutes the processing of the neurotransmitter signal from the presynaptic neuron. To simulate a single neuron accurately, we actually need all the resources of a very powerful computer. So it may be the case that we can boil down intelligence to some kind of process which can be printed in silicon. But I think it's also entirely likely that the extreme parallelism (vast orders of magnitude greater than the widest GPU) of the brain is required for the kind of general intelligence that humans express, and the "slowness" of biological computation is a necessary trade-off for the flexibility we enjoy. If that's the case, it's going to be very hard for a serial computer to emulate intelligence.
- kaba0 5y agoI by no means say that our brain is not impressive - even a fly’s is marvelously complex and capable. But all of them are made up from cells that were created through evolution, not intelligent design. The same way the giraffe has a recurrent nerve going all the way down and up inside its neck for absolutely no reason other than evolution modifying only one factor (neck length) without restructuring, cells have many similar sorts of “hacks”. So I think it is naive to think that biological systems are efficient. They do tend to optimize for a local minima, but there are inherent hard limits there. Also, while indeed we can’t simulate the whole of neuron, why would we want to do that? I think that is backwards. We only have to model the actually important function of a neuron. If we were to have a water computer, would it make sense to simulate fluid dynamics instead of just the logical gates? Due to the messiness of biology, indeed some hard to model factors will effect things (in the analogy, water will be spilt/evaporated) but we should rather overlook the ones that have a minimal influence on the results.