6 ms·

Could a Neuroscientist Understand a Microprocessor? (2017)

146 points by KenanSulayman 7y ago

jugg1es 7y ago
I would be interested to see any followup studies that looked into the things they discuss at the end about isolation of computational processes.
oceliker 7y ago
Previous discussion: https://news.ycombinator.com/item?id=16817832 https://news.ycombinator.com/item?id=16817832 This is one of my favorite papers of all time and I'm happy to see it on the front page again.
tbenst 7y ago
I wrote a lay summary of this paper that compares and contrasts with the whole brain imaging and stimulation literature for those interested in a lighter read! http://www.neuwritewest.org/blog/can-we-reverse-engineer-the-brain-like-a-computer http://www.neuwritewest.org/blog/can-we-reverse-engineer-the...
- darsnack 7y ago
  Great summary! At the end you mention the difficulty extrapolating beyond an HH neuron model. I think curious readers will find the work of Jim Smith (https://fcrc.acm.org/plenary-speakers/james-e-smith-plenary https://fcrc.acm.org/plenary-speakers/james-e-smith-plenary) interesting in this regard. His work starts with the possible information representation scheme (temporal coding <=> binary coding) and a compute unit (SRM0 neuron <=> transistor) and builds up the equivalent of Boolean logic/algebra from there. As opposed to a neuroscientist understanding a processor, Jim is a computer architect using his techniques to understand the brain.
6gvONxR4sf7o 7y ago
Lately I've been wondering an analogous question in NLP. Could modern NLP techniques learn what software source code does?
- The_rationalist 7y ago
  A neural network doesn't understand anything but sure it could predict the program output sequence based on the input sequence. Would there be any useful use case to such a technology?
  - gdy 7y ago
    Could it?
    - SeanAppleby 7y ago
      GPT-2 doesn't do a _great_ job at this, but it does to some extent seem to have learned some relationships in structures of text in different programming languages. For example while toying around with it, I gave it a javascript function that referenced an undefined callback, and GPT-2 gave me back a function, with almost correct syntax and mostly random logic, but with the name and call signature of the callback.
      - 6gvONxR4sf7o 7y ago
        These sorts of models do well at generating text, whether it's code or language. That's what you're talking about. They can perform the "execution" to some degree too for natural language (e.g. the squad tasks). But Q&A and text generation are very different tasks. I haven't seen anyone apply a transformer to source code based Q&A tasks.
  - 6gvONxR4sf7o 7y ago
    There's no technological use for neuroscience analysis tools to model a microchip, nor for NLP modeling source code, but that isn't the point. It's a potentially useful target to hone the methodology itself on. I actually don't think something like BERT would do so well at learning to essentially execute code. The counterexample is for a transformer with context width of N tokens, to try to model "def f(x) { ... }; <at least N more tokens here>; def g(x) { y = f(x) ...};". In this case, in trying to evaluate "g(4)", f appears as the literal token "f" and nothing more. To be able to learn to evaluate g, a model would need to be able to do (at least) distant coreference resolution.
AbrahamParangi 7y ago
"What I cannot create, I do not understand"
- java-man 7y ago
  This quote was seen on Richard Feynman's blackboard at the time of his death.
- brmgb 7y ago
  Pretty much everyone on this planet could at some point create a brain (with assistance). I don't think anyone will pretend to understand it.
  - brianhorakh 7y ago
    You mean create a brain by convetion i.E. having intercouse of course. Given raw cellular material, gtca biological code samples, design an organic chemical engine "brain" capable of self replication, repair and inference? Its sorting through the 4.5 billion years of leftover legacy dna code that makes it tedious to understand.
    - bitwiser 7y ago
      So what you're saying is....reproductive organs are a layer of abstraction. brain explodes
      - bronzeage 7y ago
        literally mind fuck.
jonnycomputer 7y ago
In contrast, I was underwhelmed. Neuroscientists typically try to develop tasks that create specific contrasts that highlight aspects of specific sorts of computations. They didn't do anything like that. For example, in the field I work in, we very very reliably get signal that track reward prediction errors in the striatum (e.g. BOLD response in fmri) during reward learning tasks.
- eli_gottlieb 7y ago
  >For example, in the field I work in, we very very reliably get signal that track reward prediction errors in the striatum (e.g. BOLD response in fmri) during reward learning tasks. Yeah, but that's a property of how the BOLD response correlates with the task structure, not a map of what computations the brain actually does. Yeah, ok, I'm shortening things by a lot, but neuroimaging really is quite fraught in terms of what sorts of computational-level inferences we can draw from task-based experiments.
  - jonnycomputer 7y ago
    In these experiments, the BOLD signal reliably scales with reward prediction error, which is a computation, the difference between the reward expected and received. In short, we fit reinforcement learning models to in-task behavior, then correlate parameters of that model to neuroimaging acquired during that behavior. fMRI does have quite a few limitations; BOLD signal is believed to reflect the amount of work being done, loosely, speaking, but it is not a computational signal in of itself; for reinforcement learning models we'd like to measure dopamine itself. Two decades ago, the pioneering work of Peter Dayan and Read Montague established that dopamine neurons report a prediction error signal (see: http://static.vtc.vt.edu/media/documents/SchultzDayanMontague1997Science.pdf http://static.vtc.vt.edu/media/documents/SchultzDayanMontagu...), but recording of neurotransmitter release in humans is hard, and frankly, long thought impossible (given ethical constraints). However, recent work by Read Montague has done just that. See: https://www.pnas.org/content/113/1/200 https://www.pnas.org/content/113/1/200 Keep tuned.
    - 7y ago
BadThink6655321 7y ago
Of course a neuroscientist can understand a microprocessor. That’s easy. Let me know when a microprocessor can understand a neuroscientist.
- microcolonel 7y ago
  Indeed, the title leads you in an unsatisfying direction.
  - QuesnayJr 7y ago
    I think the title is extremely witty, since you expect to see the sentence in a different order.
- scott_s 7y ago
  In case anyone thinks the above joke is actually what the paper is doing: it is not. The work is trying to use the same techniques that neuroscientists use on neural systems on a processor. The reason is simple: we already understand processors backwards-and-forwards. The authors' hypothesis is that if the techniques we use on neural systems do not yield useful insights that match the already known-correct understanding of a processor, then we may have a problem with our current neural techniques.
- knolax 7y ago
  It seems the paper is about using neuroscience techniques (designed for the brain which is hard to measure nondestructively) to examine microprocessors (which we already know all the physical measurements of) in order to evaluate how effective these techniques would be if we weren't so limited in our measurement of the brain. It's not as vapid as the title suggests.
SubiculumCode 7y ago
This article keeps coming up over and over; I suspect that it appeals to the HN crowd's background, but really, the article, and others like it, underestimate the sophisticated behavioral methods we cognitive neuroscientists employ, as well as the value of convergent lines of evidence taken from molecular to systems neuroscience.
mywittyname 7y ago
I think their lesion approach is a little awkward and probably doesn't map that well between the fields. Removing a transistor, IMHO, is more akin to removing an eye or some other body part. Yeah, it allows one to determine if that transitor was important, but its impact on behavior is due to damaged hardware. Plus, the software probably can't be written to compensate for missing transistors. Instead of removing transistors from the CPU to determine if it was important to the "behavior" of the game, I think it would be more analogous to delete portions of memory and see if that alters game behavior. Even then, I think it's much more likely to cause a crash if done without some a priori information.
adenadel 7y ago
This is sort of a spiritual sequel to Can a biologist fix a radio?--Or, what I learned while studying apoptosis. https://www.ncbi.nlm.nih.gov/pubmed/12242150 https://www.ncbi.nlm.nih.gov/pubmed/12242150
mantoto 7y ago
A computer chip works fundamentally different. This paper doesn't make sense at all.

146 points by KenanSulayman 7y ago

I would be interested to see any followup studies that looked into the things they discuss at the end about isolation of computational processes.

oceliker 7y ago

Previous discussion: https://news.ycombinator.com/item?id=16817832 https://news.ycombinator.com/item?id=16817832 This is one of my favorite papers of all time and I'm happy to see it on the front page again.

tbenst 7y ago

I wrote a lay summary of this paper that compares and contrasts with the whole brain imaging and stimulation literature for those interested in a lighter read! http://www.neuwritewest.org/blog/can-we-reverse-engineer-the-brain-like-a-computer http://www.neuwritewest.org/blog/can-we-reverse-engineer-the...

darsnack 7y ago
Great summary! At the end you mention the difficulty extrapolating beyond an HH neuron model. I think curious readers will find the work of Jim Smith (https://fcrc.acm.org/plenary-speakers/james-e-smith-plenary https://fcrc.acm.org/plenary-speakers/james-e-smith-plenary) interesting in this regard. His work starts with the possible information representation scheme (temporal coding <=> binary coding) and a compute unit (SRM0 neuron <=> transistor) and builds up the equivalent of Boolean logic/algebra from there. As opposed to a neuroscientist understanding a processor, Jim is a computer architect using his techniques to understand the brain.

6gvONxR4sf7o 7y ago

Lately I've been wondering an analogous question in NLP. Could modern NLP techniques learn what software source code does?

The_rationalist 7y ago
A neural network doesn't understand anything but sure it could predict the program output sequence based on the input sequence. Would there be any useful use case to such a technology?
- gdy 7y ago
  Could it?
  - SeanAppleby 7y ago
    GPT-2 doesn't do a _great_ job at this, but it does to some extent seem to have learned some relationships in structures of text in different programming languages. For example while toying around with it, I gave it a javascript function that referenced an undefined callback, and GPT-2 gave me back a function, with almost correct syntax and mostly random logic, but with the name and call signature of the callback.
    - 6gvONxR4sf7o 7y ago
      These sorts of models do well at generating text, whether it's code or language. That's what you're talking about. They can perform the "execution" to some degree too for natural language (e.g. the squad tasks). But Q&A and text generation are very different tasks. I haven't seen anyone apply a transformer to source code based Q&A tasks.
- 6gvONxR4sf7o 7y ago
  There's no technological use for neuroscience analysis tools to model a microchip, nor for NLP modeling source code, but that isn't the point. It's a potentially useful target to hone the methodology itself on. I actually don't think something like BERT would do so well at learning to essentially execute code. The counterexample is for a transformer with context width of N tokens, to try to model "def f(x) { ... }; <at least N more tokens here>; def g(x) { y = f(x) ...};". In this case, in trying to evaluate "g(4)", f appears as the literal token "f" and nothing more. To be able to learn to evaluate g, a model would need to be able to do (at least) distant coreference resolution.

AbrahamParangi 7y ago

"What I cannot create, I do not understand"

java-man 7y ago
This quote was seen on Richard Feynman's blackboard at the time of his death.
brmgb 7y ago
Pretty much everyone on this planet could at some point create a brain (with assistance). I don't think anyone will pretend to understand it.
- brianhorakh 7y ago
  You mean create a brain by convetion i.E. having intercouse of course. Given raw cellular material, gtca biological code samples, design an organic chemical engine "brain" capable of self replication, repair and inference? Its sorting through the 4.5 billion years of leftover legacy dna code that makes it tedious to understand.
  - bitwiser 7y ago
    So what you're saying is....reproductive organs are a layer of abstraction. brain explodes
    - bronzeage 7y ago
      literally mind fuck.

jonnycomputer 7y ago

In contrast, I was underwhelmed. Neuroscientists typically try to develop tasks that create specific contrasts that highlight aspects of specific sorts of computations. They didn't do anything like that. For example, in the field I work in, we very very reliably get signal that track reward prediction errors in the striatum (e.g. BOLD response in fmri) during reward learning tasks.

eli_gottlieb 7y ago
>For example, in the field I work in, we very very reliably get signal that track reward prediction errors in the striatum (e.g. BOLD response in fmri) during reward learning tasks. Yeah, but that's a property of how the BOLD response correlates with the task structure, not a map of what computations the brain actually does. Yeah, ok, I'm shortening things by a lot, but neuroimaging really is quite fraught in terms of what sorts of computational-level inferences we can draw from task-based experiments.
- jonnycomputer 7y ago
  In these experiments, the BOLD signal reliably scales with reward prediction error, which is a computation, the difference between the reward expected and received. In short, we fit reinforcement learning models to in-task behavior, then correlate parameters of that model to neuroimaging acquired during that behavior. fMRI does have quite a few limitations; BOLD signal is believed to reflect the amount of work being done, loosely, speaking, but it is not a computational signal in of itself; for reinforcement learning models we'd like to measure dopamine itself. Two decades ago, the pioneering work of Peter Dayan and Read Montague established that dopamine neurons report a prediction error signal (see: http://static.vtc.vt.edu/media/documents/SchultzDayanMontague1997Science.pdf http://static.vtc.vt.edu/media/documents/SchultzDayanMontagu...), but recording of neurotransmitter release in humans is hard, and frankly, long thought impossible (given ethical constraints). However, recent work by Read Montague has done just that. See: https://www.pnas.org/content/113/1/200 https://www.pnas.org/content/113/1/200 Keep tuned.
  - 7y ago

BadThink6655321 7y ago

Of course a neuroscientist can understand a microprocessor. That’s easy. Let me know when a microprocessor can understand a neuroscientist.

microcolonel 7y ago
Indeed, the title leads you in an unsatisfying direction.
- QuesnayJr 7y ago
  I think the title is extremely witty, since you expect to see the sentence in a different order.
scott_s 7y ago
In case anyone thinks the above joke is actually what the paper is doing: it is not. The work is trying to use the same techniques that neuroscientists use on neural systems on a processor. The reason is simple: we already understand processors backwards-and-forwards. The authors' hypothesis is that if the techniques we use on neural systems do not yield useful insights that match the already known-correct understanding of a processor, then we may have a problem with our current neural techniques.
knolax 7y ago
It seems the paper is about using neuroscience techniques (designed for the brain which is hard to measure nondestructively) to examine microprocessors (which we already know all the physical measurements of) in order to evaluate how effective these techniques would be if we weren't so limited in our measurement of the brain. It's not as vapid as the title suggests.

SubiculumCode 7y ago

This article keeps coming up over and over; I suspect that it appeals to the HN crowd's background, but really, the article, and others like it, underestimate the sophisticated behavioral methods we cognitive neuroscientists employ, as well as the value of convergent lines of evidence taken from molecular to systems neuroscience.

mywittyname 7y ago

I think their lesion approach is a little awkward and probably doesn't map that well between the fields. Removing a transistor, IMHO, is more akin to removing an eye or some other body part. Yeah, it allows one to determine if that transitor was important, but its impact on behavior is due to damaged hardware. Plus, the software probably can't be written to compensate for missing transistors. Instead of removing transistors from the CPU to determine if it was important to the "behavior" of the game, I think it would be more analogous to delete portions of memory and see if that alters game behavior. Even then, I think it's much more likely to cause a crash if done without some a priori information.

adenadel 7y ago

This is sort of a spiritual sequel to Can a biologist fix a radio?--Or, what I learned while studying apoptosis. https://www.ncbi.nlm.nih.gov/pubmed/12242150 https://www.ncbi.nlm.nih.gov/pubmed/12242150

mantoto 7y ago

A computer chip works fundamentally different. This paper doesn't make sense at all.