15 ms·

A neuroscience experiment failed to build a connectome for a 6502 chip

113 points by artsandsci 10y ago

nacc 10y ago
One question to ask is then: what method is useful to figure out the function of an unknown microprocessor? The engineers sometimes reverse engineer chips with electron microscope, with a perfect understanding of electronic principle, a rudimentary idea of how the microprocessor works, what the high-level function of the chip is, etc. And even equipped with this knowledge cracking a chip still takes a lot of effort, and may not succeed at all. The tools of a neuroscientist would be laughable in comparison, yet neuroscientists have cracked the auditory code (and you can have an artificial cochlea now), the lower visual system is close to be cracked, we have a pretty good understanding of hippocampus and how space is represented in the brain, and many other accomplishment. These are all results from the laughable tools we have for investigating brain. Having a full connectome of the brain, and running the brain in a simulator will be a huge step of understanding it. We don't know how to use the data now simply because we don't have the data yet, and therefore no effort is putting on interpreting the data. However the usefulness can be glimpsed from neural structures where the connections are clear: the peripheral system, spinal cord, midbrain, and lower sensory and motor regions in the brain. We understand them far better than regions we don't even know where it connects: claustrum comes into mind. A simulator of the brain, I imagine, will be similar to the human genome project: nobody will understand the whole thing quickly, but it will hugely prop neuroscience forward, sometimes in ways we cannot imagine now.
medymed 10y ago
There's a big input/output issue here. Which they address, but nonethless which makes the project failure not surprising. The inputs and outputs for a brain/neural network, e.g. pictures as inputs as recognized object as outputs, are clear. They are not clear for a chip-restricted view where your input binary stream is a post-transformed signal from an Atari joystick, and the output is a pretransformed signal to a display where the pong paddle moves. These complex transformed inputs and outputs at the chip-restricted perspective tremendously reduce the likelihood of making sense of anything. If we know the adaptor functions for the signal to the first layer of neurons in a simple convolutional neural network, for example, and the output layer is indentifiable, it might be easier to track which neurons track which features, etc. But there might be no clear intermediate output layers to a brain, so you might have a look at inputs and outputs wrt various subnetworks to see which is most likely to be the true functional subset corresponding to some definitely-occuring function (another obstacle to define). Or assume a function is occuring and see which neurons classify it under the most circumstances. But with so many connections that sounds like a big rough game to play too. I can see how this project if successful would be another step to creation of the sentient robot overlords but I'm curious as to what people see as the more immediate medical applications would be. Some of the biggest breakthroughs in brain health in the last 20 years have been about getting rid of clots (current best is arguably manual labor) and immune modulation and new biochemical drugs whose effects would require excruciatingly sophisticated and accurate biomolecular connectomic models to model and may be a long time away.
supergarfield 10y ago
I don't think the IC/brain comparison is particularly revealing. One of the key differences I see is that an IC will stop working (or at least very severely malfunction) if you disable a transistor. A brain on the other hand will probably keep working even if a sizable amount of neurons die. So little to no useful info is gained by suppressing a transistor, while lesioning a part of the brain could still produce insight if you observe behavioral changes. That being said disabling neurons sounds a little crude and may not help much in gaining a finer understanding. Then again I know nothing about neuroscience :)
- glial 10y ago
  > disabling neurons sounds a little crude and may not help much in gaining a finer understanding Yes it's crude, but artificially creating 'lesions' and investigating brain disfunction is a time-honored tradition in neuroscience. It's usually done with rats/cats/monkeys, but humans with brain damage due to strokes or surgeries also serve as useful 'natural experiments.' HM is a classic example [0]. When working with something as complicated with the brain, taking out pieces and seeing what changes is a reasonable way to investigate its operation...depending on how you feel about utilizing animals in this way. [0] https://en.wikipedia.org/wiki/Henry_Molaison https://en.wikipedia.org/wiki/Henry_Molaison
- jfoutz 10y ago
  I think the point is kind of like using arithmetic to find the area under a curve. occasionally you'll get lucky and can work out the area of an arch, but for an arbitrary curve, what you really need is calculus. So, also not a neuroscientist. So i think the general point is, their current approach is to turn off neurons and see how things change. with careful evaluation, you should be able to pin down the role of that specific neuron. With a chip, it seems like you'd want to investigate the behavior when output when output is the same as the input - just a pass through. maybe you can deduce the effect of that specific gate. My (poor) understanding is the standard tools weren't enough to identify stuff like ripple adders (or whatever's on the chip) so maybe they're using arithmetic when they need calculus.
  - 10y ago
taneq 10y ago
I don't think the title's quite accurate. It sounds like they had no trouble building a connectome for the chip, they just couldn't then reverse engineer the chip's functionality by applying standard neuroscience techniques to said connectome while it ran in a simulator. It's basically saying that even IF we had a full connectome of the brain that we could run in a simulator (and presumably produce a simulated personality) that we'd be no closer to actually understanding how the damn thing works.
- ivraatiems 10y ago
  Unfortunately, producing a working simulation of the brain doesn't mean the simulated brain would actually... work. Humans aren't created full form with personality and behavior ready to go, they learn over years. A brain in a vacuum would have none of that input.
  - andai 10y ago
    If you wish to make a brain from scratch, you must first invent the universe.
  - ccrush 10y ago
    I think there are a few instances of medical science bringing back persons from comas that may help with identifying what would happen to a fully formed brain that was completely shut down for a decent period of time. Presumably, reproducing the physical structure of such a brain and giving it similar inputs would lead to an experimentally similar result. The question I would like to see answered is what happens if we do this to someone with a grown brain and wake them up in a simulation of a space ship that traveled to a far away planet. Again presumably, the person would need nutrition and care and education for some time, but then would they be able to carry out significant advances on behalf of humanity. Would they even want to? What if they feel betrayed by their fate and creator? Then again, who doesn't? Someone write a book about this.
    - drzaiusapelord 10y ago
      That's not the criticism he's making. He's saying to get to a functional adult brain you must have a few cells that work together, then a fetus brain, then a baby brain, then an infant brain, then a toddler brain, etc all the way to an adult brain and all the care and learning needed in those stages to make it all work. You need the morphology and structures that happen in nature to get the structures and functionality needed to properly simulate a brain. We simply don't know about enough about learning, development, brains, and consciousness to just plug into a simulated adult brain and make it all work. We can only follow the path nature has laid out. I'm sure you can skip some parts like fetal stages, but we develop language at a very specific range of ages using specific techniques. We can't skip the language acquisition stage and then expect our simulated brain to magically know language. We can't skip the angry "me" stage of a toddler and expect the mind to have self-assertiveness. We can't skip the sexual and social awakening stage of teenhood and expect the mind to understand complex social concepts and sexuality, etc, etc. The only scenario I can imagine out of this problem is a per neuron copy of a working brain on the cellular level. I imagine this isn't, or won't ever be, in the cards for practical reasons. Or if you could someone divine the algorithms that create consciousness, assuming this is even possible on a transistor based machine, then that would also be a work-around. Right now we can't do either, so following nature's approach is the probably the sanest way forward. I wouldnt be surprised if the first AI ends up being more a biological 'computer' than anything having to do with x86 instruction sets and transistors, but that's just a guess on my part.
zw123456 10y ago
I get it the point they were trying to make is that you cannot understand the software by looking at the hardware. But I also agree that the analogy is very weak.
glial 10y ago
A microprocessor is admittedly a poor metaphor for a brain. However, I think the article still makes a strong point, which is that neuroscientists use many experimental and signal-processing techniques whose utility hasn't even been demonstrated in principle, much less in practice. To me that's the largest take-away from this article. To use a set of techniques to investigate the operation of a device whose function we fully understand, and still fail to draw meaningful conclusions, raises serious questions about the usefulness of those same techniques in a much more complicated and less-well-understood domain.
- enord 10y ago
  >However, I think the article still makes a strong point, which is that neuroscientists use many experimental and signal-processing techniques whose utility hasn't even been demonstrated in principle, much less in practice. This is the point. There are multiple levels of interpretation layered on top of the physical process. There is no reason to believe that they are tidy or hierarchical either, abstractions and indirections can leak left and right (or top to bottom). We lack the language and tools to describe these structures laid bare, and words like "abstraction" and "indirection" are probably a bad fit. Even "interpretation" might prove to be conceptually misleading when cracking this nut, so to speak.
- SubiculumCode 10y ago
  and yet we know alot. Hippocampal damage causes deep amnesia, prefrontal damage leads to poor inhibition, introspection, etc. I could go on.
  - TheOtherHobbes 10y ago
    That's not a lot - that's actually very little. We have a crude 3D map of some of the top-level systems, but very little idea how or why they work as they do. Which is not a surprise. Imagine someone hands you an encapsulated CPU and you don't know what a clock is, what external memory is, or what an instruction set is. How long will it take you to understand what it does and how it works? Now let's say you know what a gate is, and you take the top off and reverse engineer the circuit. Does that help? IMO, not as much as we'd all like. For a CPU the important information isn't in the connectome, it's in the design manual. It's a fair bet the brain is rather more complicated. And there's no manual. Having a crude map of some top-level systems is a good start, but I'm surprised that there seems to have been a lot of labelling and identification, but comparatively little interest in investigating how information is passed between the subsystems and how they function together.
mrob 10y ago
Previous discussion: https://news.ycombinator.com/item?id=11780565 https://news.ycombinator.com/item?id=11780565
stochastician 10y ago
Original author of the paper here, happy to answer questions. It's a little strange refreshing HN and discovering new popular press about your paper.
- cornholio 10y ago
  Aside from professional hubris, do you think neuroscience is a good way to learn to create a thinking machine ?
- logicallee 10y ago
  Here's a [serious] question for you - today by my estimation there are over 1,000 organizations (governments, data warehouses, cloud storage solutions) that use more computing power than it would take to simulate a brain in realtime. So why does ANYBODY think it will NEVER happen? It is as though someone declared, "Powered heavier-than-air flight will never be possible." Uh, of course they will? It's totally obvious? Like how can any reasonable or even poorly-informed, uninvolved person who is in a hurry, possibly not see how completely obvious this is? Help me understand this. Never is a long time. - EDIT: case in point, for some inexplicable reason this comment, a direct question to you stochastician, in the above form (I didn't edit except to add [serious] and the last sentence) was just downvoted twice. (I take it by people who consider everything I just said to be BS.) Just to be clear none of the above is sarcastic, etc. To me - and I suppose you - it's pretty clear what will eventually be happen, and "never" is a strong claim. I completely fail to see how anyone can claim "never" when we're ahead in terms of hardware, today.
  - soVeryTired 10y ago
    I think it's partly a problem of not having a roadmap to get from where we are now to where we want to be. Contrast AI with, say, interplanetary space travel. We're a long way away from that too, but there's a more obvious path to getting there. We need better control of nuclear energy, more robust robotics designs, and more efficient environmental control. These are engineering challenges: for the most part, the science is already laid out. With AI the science isn't there. We don't know which direction to move in, let alone how to move in that direction.
dontreact 10y ago
Title needs to be changed. As mentioned by the author elsewhere in this thread, the connectome (map of connections) was so good that they managed to build a perfect simulation. The point of the article seems to be that in this simulation, typical non-connectomics techniques used today in Neuroscience (highly specific lesions which in the real brain are created by things such optogenetics) were not able to elucidate how this simulation worked.
deleted 10y ago
[deleted]

113 points by artsandsci 10y ago

nacc 10y ago

One question to ask is then: what method is useful to figure out the function of an unknown microprocessor? The engineers sometimes reverse engineer chips with electron microscope, with a perfect understanding of electronic principle, a rudimentary idea of how the microprocessor works, what the high-level function of the chip is, etc. And even equipped with this knowledge cracking a chip still takes a lot of effort, and may not succeed at all. The tools of a neuroscientist would be laughable in comparison, yet neuroscientists have cracked the auditory code (and you can have an artificial cochlea now), the lower visual system is close to be cracked, we have a pretty good understanding of hippocampus and how space is represented in the brain, and many other accomplishment. These are all results from the laughable tools we have for investigating brain. Having a full connectome of the brain, and running the brain in a simulator will be a huge step of understanding it. We don't know how to use the data now simply because we don't have the data yet, and therefore no effort is putting on interpreting the data. However the usefulness can be glimpsed from neural structures where the connections are clear: the peripheral system, spinal cord, midbrain, and lower sensory and motor regions in the brain. We understand them far better than regions we don't even know where it connects: claustrum comes into mind. A simulator of the brain, I imagine, will be similar to the human genome project: nobody will understand the whole thing quickly, but it will hugely prop neuroscience forward, sometimes in ways we cannot imagine now.

medymed 10y ago

There's a big input/output issue here. Which they address, but nonethless which makes the project failure not surprising. The inputs and outputs for a brain/neural network, e.g. pictures as inputs as recognized object as outputs, are clear. They are not clear for a chip-restricted view where your input binary stream is a post-transformed signal from an Atari joystick, and the output is a pretransformed signal to a display where the pong paddle moves. These complex transformed inputs and outputs at the chip-restricted perspective tremendously reduce the likelihood of making sense of anything. If we know the adaptor functions for the signal to the first layer of neurons in a simple convolutional neural network, for example, and the output layer is indentifiable, it might be easier to track which neurons track which features, etc. But there might be no clear intermediate output layers to a brain, so you might have a look at inputs and outputs wrt various subnetworks to see which is most likely to be the true functional subset corresponding to some definitely-occuring function (another obstacle to define). Or assume a function is occuring and see which neurons classify it under the most circumstances. But with so many connections that sounds like a big rough game to play too. I can see how this project if successful would be another step to creation of the sentient robot overlords but I'm curious as to what people see as the more immediate medical applications would be. Some of the biggest breakthroughs in brain health in the last 20 years have been about getting rid of clots (current best is arguably manual labor) and immune modulation and new biochemical drugs whose effects would require excruciatingly sophisticated and accurate biomolecular connectomic models to model and may be a long time away.

supergarfield 10y ago

I don't think the IC/brain comparison is particularly revealing. One of the key differences I see is that an IC will stop working (or at least very severely malfunction) if you disable a transistor. A brain on the other hand will probably keep working even if a sizable amount of neurons die. So little to no useful info is gained by suppressing a transistor, while lesioning a part of the brain could still produce insight if you observe behavioral changes. That being said disabling neurons sounds a little crude and may not help much in gaining a finer understanding. Then again I know nothing about neuroscience :)

glial 10y ago
> disabling neurons sounds a little crude and may not help much in gaining a finer understanding Yes it's crude, but artificially creating 'lesions' and investigating brain disfunction is a time-honored tradition in neuroscience. It's usually done with rats/cats/monkeys, but humans with brain damage due to strokes or surgeries also serve as useful 'natural experiments.' HM is a classic example [0]. When working with something as complicated with the brain, taking out pieces and seeing what changes is a reasonable way to investigate its operation...depending on how you feel about utilizing animals in this way. [0] https://en.wikipedia.org/wiki/Henry_Molaison https://en.wikipedia.org/wiki/Henry_Molaison
jfoutz 10y ago
I think the point is kind of like using arithmetic to find the area under a curve. occasionally you'll get lucky and can work out the area of an arch, but for an arbitrary curve, what you really need is calculus. So, also not a neuroscientist. So i think the general point is, their current approach is to turn off neurons and see how things change. with careful evaluation, you should be able to pin down the role of that specific neuron. With a chip, it seems like you'd want to investigate the behavior when output when output is the same as the input - just a pass through. maybe you can deduce the effect of that specific gate. My (poor) understanding is the standard tools weren't enough to identify stuff like ripple adders (or whatever's on the chip) so maybe they're using arithmetic when they need calculus.
- 10y ago

taneq 10y ago

I don't think the title's quite accurate. It sounds like they had no trouble building a connectome for the chip, they just couldn't then reverse engineer the chip's functionality by applying standard neuroscience techniques to said connectome while it ran in a simulator. It's basically saying that even IF we had a full connectome of the brain that we could run in a simulator (and presumably produce a simulated personality) that we'd be no closer to actually understanding how the damn thing works.

ivraatiems 10y ago
Unfortunately, producing a working simulation of the brain doesn't mean the simulated brain would actually... work. Humans aren't created full form with personality and behavior ready to go, they learn over years. A brain in a vacuum would have none of that input.
- andai 10y ago
  If you wish to make a brain from scratch, you must first invent the universe.
- ccrush 10y ago
  I think there are a few instances of medical science bringing back persons from comas that may help with identifying what would happen to a fully formed brain that was completely shut down for a decent period of time. Presumably, reproducing the physical structure of such a brain and giving it similar inputs would lead to an experimentally similar result. The question I would like to see answered is what happens if we do this to someone with a grown brain and wake them up in a simulation of a space ship that traveled to a far away planet. Again presumably, the person would need nutrition and care and education for some time, but then would they be able to carry out significant advances on behalf of humanity. Would they even want to? What if they feel betrayed by their fate and creator? Then again, who doesn't? Someone write a book about this.
  - drzaiusapelord 10y ago
    That's not the criticism he's making. He's saying to get to a functional adult brain you must have a few cells that work together, then a fetus brain, then a baby brain, then an infant brain, then a toddler brain, etc all the way to an adult brain and all the care and learning needed in those stages to make it all work. You need the morphology and structures that happen in nature to get the structures and functionality needed to properly simulate a brain. We simply don't know about enough about learning, development, brains, and consciousness to just plug into a simulated adult brain and make it all work. We can only follow the path nature has laid out. I'm sure you can skip some parts like fetal stages, but we develop language at a very specific range of ages using specific techniques. We can't skip the language acquisition stage and then expect our simulated brain to magically know language. We can't skip the angry "me" stage of a toddler and expect the mind to have self-assertiveness. We can't skip the sexual and social awakening stage of teenhood and expect the mind to understand complex social concepts and sexuality, etc, etc. The only scenario I can imagine out of this problem is a per neuron copy of a working brain on the cellular level. I imagine this isn't, or won't ever be, in the cards for practical reasons. Or if you could someone divine the algorithms that create consciousness, assuming this is even possible on a transistor based machine, then that would also be a work-around. Right now we can't do either, so following nature's approach is the probably the sanest way forward. I wouldnt be surprised if the first AI ends up being more a biological 'computer' than anything having to do with x86 instruction sets and transistors, but that's just a guess on my part.

zw123456 10y ago

I get it the point they were trying to make is that you cannot understand the software by looking at the hardware. But I also agree that the analogy is very weak.

glial 10y ago

A microprocessor is admittedly a poor metaphor for a brain. However, I think the article still makes a strong point, which is that neuroscientists use many experimental and signal-processing techniques whose utility hasn't even been demonstrated in principle, much less in practice. To me that's the largest take-away from this article. To use a set of techniques to investigate the operation of a device whose function we fully understand, and still fail to draw meaningful conclusions, raises serious questions about the usefulness of those same techniques in a much more complicated and less-well-understood domain.

enord 10y ago
>However, I think the article still makes a strong point, which is that neuroscientists use many experimental and signal-processing techniques whose utility hasn't even been demonstrated in principle, much less in practice. This is the point. There are multiple levels of interpretation layered on top of the physical process. There is no reason to believe that they are tidy or hierarchical either, abstractions and indirections can leak left and right (or top to bottom). We lack the language and tools to describe these structures laid bare, and words like "abstraction" and "indirection" are probably a bad fit. Even "interpretation" might prove to be conceptually misleading when cracking this nut, so to speak.
SubiculumCode 10y ago
and yet we know alot. Hippocampal damage causes deep amnesia, prefrontal damage leads to poor inhibition, introspection, etc. I could go on.
- TheOtherHobbes 10y ago
  That's not a lot - that's actually very little. We have a crude 3D map of some of the top-level systems, but very little idea how or why they work as they do. Which is not a surprise. Imagine someone hands you an encapsulated CPU and you don't know what a clock is, what external memory is, or what an instruction set is. How long will it take you to understand what it does and how it works? Now let's say you know what a gate is, and you take the top off and reverse engineer the circuit. Does that help? IMO, not as much as we'd all like. For a CPU the important information isn't in the connectome, it's in the design manual. It's a fair bet the brain is rather more complicated. And there's no manual. Having a crude map of some top-level systems is a good start, but I'm surprised that there seems to have been a lot of labelling and identification, but comparatively little interest in investigating how information is passed between the subsystems and how they function together.

mrob 10y ago

Previous discussion: https://news.ycombinator.com/item?id=11780565 https://news.ycombinator.com/item?id=11780565

stochastician 10y ago

Original author of the paper here, happy to answer questions. It's a little strange refreshing HN and discovering new popular press about your paper.

cornholio 10y ago
Aside from professional hubris, do you think neuroscience is a good way to learn to create a thinking machine ?
logicallee 10y ago
Here's a [serious] question for you - today by my estimation there are over 1,000 organizations (governments, data warehouses, cloud storage solutions) that use more computing power than it would take to simulate a brain in realtime. So why does ANYBODY think it will NEVER happen? It is as though someone declared, "Powered heavier-than-air flight will never be possible." Uh, of course they will? It's totally obvious? Like how can any reasonable or even poorly-informed, uninvolved person who is in a hurry, possibly not see how completely obvious this is? Help me understand this. Never is a long time. - EDIT: case in point, for some inexplicable reason this comment, a direct question to you stochastician, in the above form (I didn't edit except to add [serious] and the last sentence) was just downvoted twice. (I take it by people who consider everything I just said to be BS.) Just to be clear none of the above is sarcastic, etc. To me - and I suppose you - it's pretty clear what will eventually be happen, and "never" is a strong claim. I completely fail to see how anyone can claim "never" when we're ahead in terms of hardware, today.
- soVeryTired 10y ago
  I think it's partly a problem of not having a roadmap to get from where we are now to where we want to be. Contrast AI with, say, interplanetary space travel. We're a long way away from that too, but there's a more obvious path to getting there. We need better control of nuclear energy, more robust robotics designs, and more efficient environmental control. These are engineering challenges: for the most part, the science is already laid out. With AI the science isn't there. We don't know which direction to move in, let alone how to move in that direction.

dontreact 10y ago

Title needs to be changed. As mentioned by the author elsewhere in this thread, the connectome (map of connections) was so good that they managed to build a perfect simulation. The point of the article seems to be that in this simulation, typical non-connectomics techniques used today in Neuroscience (highly specific lesions which in the real brain are created by things such optogenetics) were not able to elucidate how this simulation worked.

deleted 10y ago

[deleted]