5 ms·

One of the problems is we still do not have the hardware to begin doing the needed computation to solve the problem. The brain in reality is quite slow. We kno

by 1971genocide 11y ago

One of the problems is we still do not have the hardware to begin doing the needed computation to solve the problem. The brain in reality is quite slow. We know neurons require at least 5 ms to do any computation. But the real power lies in the shear parallelism. Essentially biology set a limit of 5 ms but evolution worked around it by creating billions of neurons. Even If they are slower, because there are so many of them, they can do more computation in that 5 ms gap than all computers in the world combined ! Its truly marvellous when you sit back and think about that. When you think about how that gets done just using that information, the modern computer architecture works completely differently , What that means is we do not even have the adequate hardware to start working on this problem. We can push the current limit of computation to solve sub-problems and it seems under that constrain we have done very well. But its a slow evolution. GPUs gave us a clue, now we have FPGAs and soon we will have better hardware to create "better" intelligent machines. The machinery that is the brain is vastly complex and beautiful, but not well understood. Its a slow and incremental process but we will get there sometime this or the next century.

danieltillett 11y ago
One of the most important thing to remember is the efficiency of the human brain is layered. The ancient parts (vision for example) have been honed to an amazing level of efficiency so even the most powerful computers can't come close, while the other modules (say simple arithmetic) are so inefficient that a $2 calculator needs to uses only 1/100,000th the energy to solve the same problem. The general rule is anything we find easy as humans to accomplish is amazingly hard and ancient, while anything we find hard is relatively easy and recent (evolutionary).
- 1971genocide 11y ago
  One thing to point out that even if the "module" that does arithmetic is slow, the brain understands the context of that calculation. 2 + 2 can be viewed as a computation that happens in nature and our brain all the time. But the real magic doesn't lie in the pure computation. Our brain understands what that 2 means in the context of all the knowledge of the entire Universe. 2 cows, 2 sheeps, 2 planets ? Understanding context is what we are really good at, as said in the article "prior" knowledge. So its not surprising that a cheap calculator can outsmart me in computation when it doesn't actually compute the context of its computation, which is sort if cheating.
  - danieltillett 11y ago
    I was trying to stay away from "meaning" since it gets messy, but if you just compare the energy used by sub-conscious human arithmetic with that expended by a simple cpu then the cpu is massively more efficient. We find mathematics far more mentally taxing than visual observation purely because we are so bad at mathematics, not because visual processing is easy.
- DaFranker 11y ago
  Arithmetic savants and trained arithmeticians can come a lot closer to the calculator, showing that the usual human brain layout and/or architecture is a really poor fit for performing this type of calculations. There's also some evidence that the "efficiency" of the more powerful parts of animal brains (including humans) simply comes from a sort of n-dimensional conditional railroading, which would work like this if my understanding is correct: if you map (B/W) visual inputs to XY axes of a slice of neurons, then when a line is formed, this fires up a line of neurons, which allows a direct connection between the Z±1 neurons of the first and last neurons on the mapped line, so this connection becomes the line. These Z±1 neurons have pathways to various other nearby other potential Z±1 neurons; a connection lighting up between Z±1 neurons that don't have a straightline connection over the XY map could detect different shapes. Then the Z±1 layers and their connections could be creating highway maps between both Z±2 neurons and diagonally-offset neurons that don't "fit in the grid" we're imagining here (remember, we're in 3D and neurons don't form a homogeneous grid, so this is all abstraction anyway). These second-level connections, arguably identifying something like shapes, have connections in such a way that the next level up detects something else, etc. (insert magic we don't understand), until the detection results in recognizing a specific object, while the side-channel connections we haven't mentioned yet but kept occurring at every level connect to a different side-channel ZY map for, say, the position of the object (with its own layers of processing that eventually connect to the object concept neuron so that the "position" is associated with the recognized object), and/or perhaps other side-channel maps or parallel XY mappings for various other things. And of course, a lot of these things connect to completely different parts that we ignore here, and a lot of the information indirectly makes its way to the conscious mind in this way; the brain doesn't compile a complete vision report and submit it to the conscious mind in one download, it's all using the same hardware.
  - kefka 11y ago
    In essence, in order to figure out the brain, we would need to figure out how a single neuron actually works. I think that will take us 500 years, give or take a few.
    - LesZedCB 11y ago
- johngalt 11y ago
  I can pick up a rock and throw it at the water so it skips across the surface. It takes less than a second. How many calculations did I just do? We are 'slow' only when we formalize the problem.
  - DaFranker 11y ago
    > How many calculations did I just do? Probably not that many. Let's say you create a water-skipping robot with two different AI programs in it and a shaky, not very precise arm (but still as good as a human arm in terms of specs). One is custom-made to the physics of the problem, and will calculate the angles and the forces and the pressures and the candy tasty physekz all over, and then decide on a particular motion of the robot arm, and the rock will skip. It'll take a lot of computing power, but it'll work. The other just has a goal, to see the rock skip, and tries things at semi-random (though it has the general knowledge that it can be done and that it has to throw the rock towards the water in a particular way for it to happen and that it can control the outcome), and tries to figure out patterns between what it did and what happened as a result. Eventually, it has a particular tactic, a certain set of instructions, which could be "down down down the left thing and up up right down down the right thing and the up thing does down down up push push down push twist-force-2 at the same time", an entirely not at all complicated set of instructions with very little computation. This signal is sent through other nodes that might not have the perfect signal number, and eventually this outputs to the arm's motors... which nevertheless manage to make the rock skip, because the motion here is almost the same as the motion of the first AI, yet this one was obtained by eliminating the ones that didn't result in the rock skipping. TL;DR: You don't do that many calculations on the spot. The "maths" in most situations like this is done by elimination throughout all the attempts you've made in your life to control a throw. Subsequent throws once you're already a practiced rock-skipper just involve firing "the same neurons as usual" which send "the same signals as usual" to your muscles, which result in the same skipping as usual, with very little math. If you already know that 2x^5 + 20 = 110 implies x = 45 from doing the same calculation twenty times in the past hour, your brain isn't performing "calculations" anymore, it's just repeating a pattern that's already there in your brain like a table lookup.
LesZedCB 11y ago
I tend to disagree with the idea that we don't have the computational power equivalent to the brain. The reasoning is that, yes, all the things you say are true about having billions of slow neurons, but that makes the assumption that their only fault is that they are slow. I don't necessarily agree with that assumption. The evolutionary process doesn't find true maxima, it finds local maxima. The key is that the neuron is good enough for the task of intelligence and planning and pattern detection. However, the architecture is most likely highly inefficient. The reason why they are so good comparatively is because we just kept adding neurons, either by increasing the size of the frontal cortex and by the folding, so as to jam more in there. That is not an architectural design per se, and I highly doubt that it is optimal. Rather, it is good enough. Now, the Von Neumann computer architecture is of course lacking. Parallel computing as we know it works, but it's a total nightmare, and hardly competes with the incredible parallelism in nature. But I do think that there is sufficient power now to really feasibly compete with the brain, intelligence wise. All this with a grain of salt. I have been reading stuff in the weird corners of AI research. I think this article is fundamentally a straw man. (I believe that...) Deep learning and probabilistic machine learning are fundamentally flawed for strong AI. Jeff Hawkins is a well respected AI researcher that I think seems to agree. Another problem is of course that these are all trained in a supervised fashion. Google does well because they have huge tagged data sets. However, the brain doesn't work by training on tagged data sets, it can learn on it's own, unsupervised. So long as we keep pointing mainstream AI down the supervised statistical machine learning path, we will always be far away from strong AI.
- SomeStupidPoint 11y ago
  I don't agree that brains can learn on their own, unsupervised. Firstly, the brain has a basic feedback mechanism, which can be thought of as basic supervision: it receives quick feedback on certain kinds of hardware issues its actions caused. Most computers don't even get a similar level of feedback (eg, that they tied up the cluster on junk computations). Secondarily, modern humans are booted through this mechanism by already running humans, in what is direct supervised learning: from the time we're infants -- and for the next two decades -- our actions are supervised, commented on, corrected and our exposure to materials is metered and selected to create a (theoretically) optimum consumption plan. That we get any results out of computers with a couple months of training when comparing them to humans with a couple decades of training is testament to the fact that computer learning is orders of magnitude more effective than human learning. I find it very strange that people leave out the 2 decades of hardware tuning and supervised knowledge building that humans get when discussing about how awful it is we having to train our machines if we want them to be smart.
bglazer 11y ago
I agree that the current Von Neumann architecture is probably not adequate. However, I think there is some active research in this area. Here are a couple links: From today actually. Memristors are an exciting development if they can scale production: http://www.technologyreview.com/news/537211/a-better-way-to-build-brain-inspired-chips/ http://www.technologyreview.com/news/537211/a-better-way-to-... A startup working on this: http://brainchipinc.com/ http://brainchipinc.com/ IBM's TrueNorth Architecture: http://www.research.ibm.com/articles/brain-chip.shtml http://www.research.ibm.com/articles/brain-chip.shtml
- 1971genocide 11y ago
  All these development are really excellent. The more people who are interested in solving these problems the better. Neural Networks are awesome - but modern AI tells us that there are also various other techniques, The problem with Neural Networks is they are essentially a black box and many mathematicians and engineers are not happy working with black boxes. Right now the models for synapses and Neurons are very simple compared to what actually is happening in them. But regardless the approach is not in vain. (My current research involves autonomous motor control with tactile feedback, which is very far away from AGI)
  - bglazer 11y ago
    Can you tell me more about your research? I became really fascinated with motor control after reading an introductory neuroscience book a while back. I feel like this is not an area that receives as much attention as it should.
entee 11y ago
True. However we know shockingly little of how neurons actually do computation. Everything a neuron does it does so with a combination of electrical and chemical signals. This is why it's slow, but it also introduces considerable noise in the process. Also, because they're usually in some skull-like object, and because neurons are incredibly hard to culture, we simply lack the tools to observe them in their natural environment to the resolution you might want. Simple questions like how often a neural signal fails to cross a synapse in a brain are remarkable difficult to answer. Do we need to know how every neuron is connected to every other neuron in the brain to understand it? Even if we had such a map, would it be missing something so crucial as to be useless (for example how glia interact with the neuron)? Nobody can say for sure right now.
joe_the_user 11y ago
The human brain may indeed be beyond all computers. On the other hand, the brain and eyes of the fruit fly do amazing things. I think it's reasonable to say that vision is lacking both hardware and algorithms at this point.

danieltillett 11y ago

One of the most important thing to remember is the efficiency of the human brain is layered. The ancient parts (vision for example) have been honed to an amazing level of efficiency so even the most powerful computers can't come close, while the other modules (say simple arithmetic) are so inefficient that a $2 calculator needs to uses only 1/100,000th the energy to solve the same problem. The general rule is anything we find easy as humans to accomplish is amazingly hard and ancient, while anything we find hard is relatively easy and recent (evolutionary).

1971genocide 11y ago
One thing to point out that even if the "module" that does arithmetic is slow, the brain understands the context of that calculation. 2 + 2 can be viewed as a computation that happens in nature and our brain all the time. But the real magic doesn't lie in the pure computation. Our brain understands what that 2 means in the context of all the knowledge of the entire Universe. 2 cows, 2 sheeps, 2 planets ? Understanding context is what we are really good at, as said in the article "prior" knowledge. So its not surprising that a cheap calculator can outsmart me in computation when it doesn't actually compute the context of its computation, which is sort if cheating.
- danieltillett 11y ago
  I was trying to stay away from "meaning" since it gets messy, but if you just compare the energy used by sub-conscious human arithmetic with that expended by a simple cpu then the cpu is massively more efficient. We find mathematics far more mentally taxing than visual observation purely because we are so bad at mathematics, not because visual processing is easy.
DaFranker 11y ago
Arithmetic savants and trained arithmeticians can come a lot closer to the calculator, showing that the usual human brain layout and/or architecture is a really poor fit for performing this type of calculations. There's also some evidence that the "efficiency" of the more powerful parts of animal brains (including humans) simply comes from a sort of n-dimensional conditional railroading, which would work like this if my understanding is correct: if you map (B/W) visual inputs to XY axes of a slice of neurons, then when a line is formed, this fires up a line of neurons, which allows a direct connection between the Z±1 neurons of the first and last neurons on the mapped line, so this connection becomes the line. These Z±1 neurons have pathways to various other nearby other potential Z±1 neurons; a connection lighting up between Z±1 neurons that don't have a straightline connection over the XY map could detect different shapes. Then the Z±1 layers and their connections could be creating highway maps between both Z±2 neurons and diagonally-offset neurons that don't "fit in the grid" we're imagining here (remember, we're in 3D and neurons don't form a homogeneous grid, so this is all abstraction anyway). These second-level connections, arguably identifying something like shapes, have connections in such a way that the next level up detects something else, etc. (insert magic we don't understand), until the detection results in recognizing a specific object, while the side-channel connections we haven't mentioned yet but kept occurring at every level connect to a different side-channel ZY map for, say, the position of the object (with its own layers of processing that eventually connect to the object concept neuron so that the "position" is associated with the recognized object), and/or perhaps other side-channel maps or parallel XY mappings for various other things. And of course, a lot of these things connect to completely different parts that we ignore here, and a lot of the information indirectly makes its way to the conscious mind in this way; the brain doesn't compile a complete vision report and submit it to the conscious mind in one download, it's all using the same hardware.
- kefka 11y ago
  In essence, in order to figure out the brain, we would need to figure out how a single neuron actually works. I think that will take us 500 years, give or take a few.
  - LesZedCB 11y ago
johngalt 11y ago
I can pick up a rock and throw it at the water so it skips across the surface. It takes less than a second. How many calculations did I just do? We are 'slow' only when we formalize the problem.
- DaFranker 11y ago
  > How many calculations did I just do? Probably not that many. Let's say you create a water-skipping robot with two different AI programs in it and a shaky, not very precise arm (but still as good as a human arm in terms of specs). One is custom-made to the physics of the problem, and will calculate the angles and the forces and the pressures and the candy tasty physekz all over, and then decide on a particular motion of the robot arm, and the rock will skip. It'll take a lot of computing power, but it'll work. The other just has a goal, to see the rock skip, and tries things at semi-random (though it has the general knowledge that it can be done and that it has to throw the rock towards the water in a particular way for it to happen and that it can control the outcome), and tries to figure out patterns between what it did and what happened as a result. Eventually, it has a particular tactic, a certain set of instructions, which could be "down down down the left thing and up up right down down the right thing and the up thing does down down up push push down push twist-force-2 at the same time", an entirely not at all complicated set of instructions with very little computation. This signal is sent through other nodes that might not have the perfect signal number, and eventually this outputs to the arm's motors... which nevertheless manage to make the rock skip, because the motion here is almost the same as the motion of the first AI, yet this one was obtained by eliminating the ones that didn't result in the rock skipping. TL;DR: You don't do that many calculations on the spot. The "maths" in most situations like this is done by elimination throughout all the attempts you've made in your life to control a throw. Subsequent throws once you're already a practiced rock-skipper just involve firing "the same neurons as usual" which send "the same signals as usual" to your muscles, which result in the same skipping as usual, with very little math. If you already know that 2x^5 + 20 = 110 implies x = 45 from doing the same calculation twenty times in the past hour, your brain isn't performing "calculations" anymore, it's just repeating a pattern that's already there in your brain like a table lookup.

LesZedCB 11y ago

I tend to disagree with the idea that we don't have the computational power equivalent to the brain. The reasoning is that, yes, all the things you say are true about having billions of slow neurons, but that makes the assumption that their only fault is that they are slow. I don't necessarily agree with that assumption. The evolutionary process doesn't find true maxima, it finds local maxima. The key is that the neuron is good enough for the task of intelligence and planning and pattern detection. However, the architecture is most likely highly inefficient. The reason why they are so good comparatively is because we just kept adding neurons, either by increasing the size of the frontal cortex and by the folding, so as to jam more in there. That is not an architectural design per se, and I highly doubt that it is optimal. Rather, it is good enough. Now, the Von Neumann computer architecture is of course lacking. Parallel computing as we know it works, but it's a total nightmare, and hardly competes with the incredible parallelism in nature. But I do think that there is sufficient power now to really feasibly compete with the brain, intelligence wise. All this with a grain of salt. I have been reading stuff in the weird corners of AI research. I think this article is fundamentally a straw man. (I believe that...) Deep learning and probabilistic machine learning are fundamentally flawed for strong AI. Jeff Hawkins is a well respected AI researcher that I think seems to agree. Another problem is of course that these are all trained in a supervised fashion. Google does well because they have huge tagged data sets. However, the brain doesn't work by training on tagged data sets, it can learn on it's own, unsupervised. So long as we keep pointing mainstream AI down the supervised statistical machine learning path, we will always be far away from strong AI.

SomeStupidPoint 11y ago
I don't agree that brains can learn on their own, unsupervised. Firstly, the brain has a basic feedback mechanism, which can be thought of as basic supervision: it receives quick feedback on certain kinds of hardware issues its actions caused. Most computers don't even get a similar level of feedback (eg, that they tied up the cluster on junk computations). Secondarily, modern humans are booted through this mechanism by already running humans, in what is direct supervised learning: from the time we're infants -- and for the next two decades -- our actions are supervised, commented on, corrected and our exposure to materials is metered and selected to create a (theoretically) optimum consumption plan. That we get any results out of computers with a couple months of training when comparing them to humans with a couple decades of training is testament to the fact that computer learning is orders of magnitude more effective than human learning. I find it very strange that people leave out the 2 decades of hardware tuning and supervised knowledge building that humans get when discussing about how awful it is we having to train our machines if we want them to be smart.

bglazer 11y ago

I agree that the current Von Neumann architecture is probably not adequate. However, I think there is some active research in this area. Here are a couple links: From today actually. Memristors are an exciting development if they can scale production: http://www.technologyreview.com/news/537211/a-better-way-to-build-brain-inspired-chips/ http://www.technologyreview.com/news/537211/a-better-way-to-... A startup working on this: http://brainchipinc.com/ http://brainchipinc.com/ IBM's TrueNorth Architecture: http://www.research.ibm.com/articles/brain-chip.shtml http://www.research.ibm.com/articles/brain-chip.shtml

1971genocide 11y ago
All these development are really excellent. The more people who are interested in solving these problems the better. Neural Networks are awesome - but modern AI tells us that there are also various other techniques, The problem with Neural Networks is they are essentially a black box and many mathematicians and engineers are not happy working with black boxes. Right now the models for synapses and Neurons are very simple compared to what actually is happening in them. But regardless the approach is not in vain. (My current research involves autonomous motor control with tactile feedback, which is very far away from AGI)
- bglazer 11y ago
  Can you tell me more about your research? I became really fascinated with motor control after reading an introductory neuroscience book a while back. I feel like this is not an area that receives as much attention as it should.

entee 11y ago

True. However we know shockingly little of how neurons actually do computation. Everything a neuron does it does so with a combination of electrical and chemical signals. This is why it's slow, but it also introduces considerable noise in the process. Also, because they're usually in some skull-like object, and because neurons are incredibly hard to culture, we simply lack the tools to observe them in their natural environment to the resolution you might want. Simple questions like how often a neural signal fails to cross a synapse in a brain are remarkable difficult to answer. Do we need to know how every neuron is connected to every other neuron in the brain to understand it? Even if we had such a map, would it be missing something so crucial as to be useless (for example how glia interact with the neuron)? Nobody can say for sure right now.

joe_the_user 11y ago

The human brain may indeed be beyond all computers. On the other hand, the brain and eyes of the fruit fly do amazing things. I think it's reasonable to say that vision is lacking both hardware and algorithms at this point.