5 ms·

I don't think the IC/brain comparison is particularly revealing. One of the key differences I see is that an IC will stop working (or at least very severely mal

I don't think the IC/brain comparison is particularly revealing. One of the key differences I see is that an IC will stop working (or at least very severely malfunction) if you disable a transistor. A brain on the other hand will probably keep working even if a sizable amount of neurons die. So little to no useful info is gained by suppressing a transistor, while lesioning a part of the brain could still produce insight if you observe behavioral changes. That being said disabling neurons sounds a little crude and may not help much in gaining a finer understanding. Then again I know nothing about neuroscience :)

glial 10y ago
> disabling neurons sounds a little crude and may not help much in gaining a finer understanding Yes it's crude, but artificially creating 'lesions' and investigating brain disfunction is a time-honored tradition in neuroscience. It's usually done with rats/cats/monkeys, but humans with brain damage due to strokes or surgeries also serve as useful 'natural experiments.' HM is a classic example [0]. When working with something as complicated with the brain, taking out pieces and seeing what changes is a reasonable way to investigate its operation...depending on how you feel about utilizing animals in this way. [0] https://en.wikipedia.org/wiki/Henry_Molaison https://en.wikipedia.org/wiki/Henry_Molaison
jfoutz 10y ago
I think the point is kind of like using arithmetic to find the area under a curve. occasionally you'll get lucky and can work out the area of an arch, but for an arbitrary curve, what you really need is calculus. So, also not a neuroscientist. So i think the general point is, their current approach is to turn off neurons and see how things change. with careful evaluation, you should be able to pin down the role of that specific neuron. With a chip, it seems like you'd want to investigate the behavior when output when output is the same as the input - just a pass through. maybe you can deduce the effect of that specific gate. My (poor) understanding is the standard tools weren't enough to identify stuff like ripple adders (or whatever's on the chip) so maybe they're using arithmetic when they need calculus.
- AstralStorm 10y ago
  Sorry, but integral as done by a neural network is not generally reversible by taking out terms, unless it's a strictly linear no threshold neural net. This one is not. Same problem with the binary functions like most kinds of digital electronics - they're typically combined in highly nonlinear ways. However, knocking out a whole IC and seeing what it did is more reasonable and about the level we are at when understanding functions of subsystems in the brain.
  - jfoutz 10y ago
    Exactly, that's a much more elegant way of putting it. Simply removing components isn't a powerful enough technique to understand the meaning of the component.
    - irascible 10y ago
      I don't really understand what "meaning" means in this context, and indeed for the brain itself. I could see some sort of classifI cation scheme.. like "this group of neurons maps many to few outputs".. these "amplify signals from the inputs", as something valuable.. and indeed it seems that training an ANN to answere questions about functional units would be a good Avenue to explore. But asking for what it means is pointless, and seems more like an ontological question, for which the only answers are things like "it means that vision works" or "it means that slight imbalances in dopamine don't kill the brain". I think trying to understand how it works isn't as important as duplicating these functions in hardware in order to produce an oracle.
comex 10y ago
A processor also operates with a high level of indirection. The 6502 doesn't have transistors dedicated to jumping or shooting; it has transistors dedicated to 'add value to accumulator', an instruction that might appear hundreds of times in the game code even on a 6502, with a different purpose each time. If they instead tried the lesioning approach on software, i.e. nopping out one instruction at a time, I think they could relatively easily identify the function of various sections of the code. Alternatively, if they analyzed a video game implemented directly in hardware rather than using a CPU (say, the original Breakout), that would probably work better too. Of course, nobody really knows how much indirection is used by the brain, but it's probably closer to a direct hardware implementation than a CPU. One respect in which it differs from both, of course, is plasticity; neither hardware nor software created by humans is given to constant self-modification.
- posterboy 10y ago
  > neither hardware nor software created by humans is given to constant self-modification. Of course it is. A VM running on a CPU is basically just constantly self modifying data with the help of modular hardware. I'd hypothesize, the brain has inert structures at it's core, too (which is why we are all alike), and the rest is the data of a state machine (an IC being part of the rest, from a phenomenological point of view). Of course the complexity might be beyond a state machine or linear bounded automata, i.e. deterministic (think quantum).
dottedmag 10y ago
Please read the original paper: the authors have discovered a bunch of transistors which, disabled, only prevent one of several games they ran from functioning, so, not knowing better, they could be dubbed "Donkey Kong transistor" / "Space Invaders transistor".

glial 10y ago

> disabling neurons sounds a little crude and may not help much in gaining a finer understanding Yes it's crude, but artificially creating 'lesions' and investigating brain disfunction is a time-honored tradition in neuroscience. It's usually done with rats/cats/monkeys, but humans with brain damage due to strokes or surgeries also serve as useful 'natural experiments.' HM is a classic example [0]. When working with something as complicated with the brain, taking out pieces and seeing what changes is a reasonable way to investigate its operation...depending on how you feel about utilizing animals in this way. [0] https://en.wikipedia.org/wiki/Henry_Molaison https://en.wikipedia.org/wiki/Henry_Molaison

jfoutz 10y ago

I think the point is kind of like using arithmetic to find the area under a curve. occasionally you'll get lucky and can work out the area of an arch, but for an arbitrary curve, what you really need is calculus. So, also not a neuroscientist. So i think the general point is, their current approach is to turn off neurons and see how things change. with careful evaluation, you should be able to pin down the role of that specific neuron. With a chip, it seems like you'd want to investigate the behavior when output when output is the same as the input - just a pass through. maybe you can deduce the effect of that specific gate. My (poor) understanding is the standard tools weren't enough to identify stuff like ripple adders (or whatever's on the chip) so maybe they're using arithmetic when they need calculus.

AstralStorm 10y ago
Sorry, but integral as done by a neural network is not generally reversible by taking out terms, unless it's a strictly linear no threshold neural net. This one is not. Same problem with the binary functions like most kinds of digital electronics - they're typically combined in highly nonlinear ways. However, knocking out a whole IC and seeing what it did is more reasonable and about the level we are at when understanding functions of subsystems in the brain.
- jfoutz 10y ago
  Exactly, that's a much more elegant way of putting it. Simply removing components isn't a powerful enough technique to understand the meaning of the component.
  - irascible 10y ago
    I don't really understand what "meaning" means in this context, and indeed for the brain itself. I could see some sort of classifI cation scheme.. like "this group of neurons maps many to few outputs".. these "amplify signals from the inputs", as something valuable.. and indeed it seems that training an ANN to answere questions about functional units would be a good Avenue to explore. But asking for what it means is pointless, and seems more like an ontological question, for which the only answers are things like "it means that vision works" or "it means that slight imbalances in dopamine don't kill the brain". I think trying to understand how it works isn't as important as duplicating these functions in hardware in order to produce an oracle.

comex 10y ago

A processor also operates with a high level of indirection. The 6502 doesn't have transistors dedicated to jumping or shooting; it has transistors dedicated to 'add value to accumulator', an instruction that might appear hundreds of times in the game code even on a 6502, with a different purpose each time. If they instead tried the lesioning approach on software, i.e. nopping out one instruction at a time, I think they could relatively easily identify the function of various sections of the code. Alternatively, if they analyzed a video game implemented directly in hardware rather than using a CPU (say, the original Breakout), that would probably work better too. Of course, nobody really knows how much indirection is used by the brain, but it's probably closer to a direct hardware implementation than a CPU. One respect in which it differs from both, of course, is plasticity; neither hardware nor software created by humans is given to constant self-modification.

posterboy 10y ago
> neither hardware nor software created by humans is given to constant self-modification. Of course it is. A VM running on a CPU is basically just constantly self modifying data with the help of modular hardware. I'd hypothesize, the brain has inert structures at it's core, too (which is why we are all alike), and the rest is the data of a state machine (an IC being part of the rest, from a phenomenological point of view). Of course the complexity might be beyond a state machine or linear bounded automata, i.e. deterministic (think quantum).

dottedmag 10y ago

Please read the original paper: the authors have discovered a bunch of transistors which, disabled, only prevent one of several games they ran from functioning, so, not knowing better, they could be dubbed "Donkey Kong transistor" / "Space Invaders transistor".