4 ms·

Silicon is actually closer to the brain than you might think. Neurons transfer charge by diffusion, so do transistors operating in sub-threshold. The problem is

Silicon is actually closer to the brain than you might think. Neurons transfer charge by diffusion, so do transistors operating in sub-threshold. The problem is that we almost exclusively use transistors operating above threshold because it is required for digital logic. Analog CMOS circuits can approach the energy efficiency of real neurons.

hatsunearu 9y ago
Uh oh, not this again. There is so much woo about using subthreshold FETs to simulate neurons when we don't even know how neurons work. I've seen the work of J. Hasler in school and she seemed to be fond about simulating a type of neuron (winner-take-all) that is hard to train with backpropagation (vanishing gradients just by inspection) and has limited grounds on physical simulation of neurons. Do you have any other resources about serious attempts at using subthreshold FETs to simulate neurons?
- p1esk 9y ago
  I think what people (including Hassler) care about is not so much simulating neurons, but the energy efficiency. Once we figure out AGI algorithms, we will want to build them in hardware. This justifies the continuing research in subthreshold FETs, because they could allow very efficient computation (not necessarily biologically realistic).
  - hatsunearu 9y ago
    Of course, subthreshold FET circuits have incredibly low energy usage. And I do think that subthreshold analog neural nets have very high utility, but I am skeptical that they can be used to faithfully reproduce real neurons. I am however very confident they can be used to create real computing hardware that can efficiently do inference etc.
- skummetmaelk 9y ago
  Well that's kind of a non-starter attitude isn't it? We don't know how neurons work so we shouldn't try to figure it out by emulating their behaviour with electronics? Analog neuromorphic approaches to not attempt to simulate neurons, they emulate them in silicon. Partly because of a belief that research in this area is required to produce ultra low power computational devices and partly to explore the real time dynamics of spiking neural networks. There are very few research groups working on this, but you can look up Kwabena Boahen's group at Stanford. They do large scale real time emulation and are currently building a sub threshold neuron accelerator for the neuro engineering framework from Chris Eliasmith at Waterloo which is famously used to create SPAUN. There is also the Karlheinz Meier group at Heidelberg university which does wafer-scale networks of neurons in accelerated time. Giacomo Indiveri at ETH Zurich has silicon neurons with on-chip learning circuits that the others are missing.

hatsunearu 9y ago

Uh oh, not this again. There is so much woo about using subthreshold FETs to simulate neurons when we don't even know how neurons work. I've seen the work of J. Hasler in school and she seemed to be fond about simulating a type of neuron (winner-take-all) that is hard to train with backpropagation (vanishing gradients just by inspection) and has limited grounds on physical simulation of neurons. Do you have any other resources about serious attempts at using subthreshold FETs to simulate neurons?

p1esk 9y ago
I think what people (including Hassler) care about is not so much simulating neurons, but the energy efficiency. Once we figure out AGI algorithms, we will want to build them in hardware. This justifies the continuing research in subthreshold FETs, because they could allow very efficient computation (not necessarily biologically realistic).
- hatsunearu 9y ago
  Of course, subthreshold FET circuits have incredibly low energy usage. And I do think that subthreshold analog neural nets have very high utility, but I am skeptical that they can be used to faithfully reproduce real neurons. I am however very confident they can be used to create real computing hardware that can efficiently do inference etc.
skummetmaelk 9y ago
Well that's kind of a non-starter attitude isn't it? We don't know how neurons work so we shouldn't try to figure it out by emulating their behaviour with electronics? Analog neuromorphic approaches to not attempt to simulate neurons, they emulate them in silicon. Partly because of a belief that research in this area is required to produce ultra low power computational devices and partly to explore the real time dynamics of spiking neural networks. There are very few research groups working on this, but you can look up Kwabena Boahen's group at Stanford. They do large scale real time emulation and are currently building a sub threshold neuron accelerator for the neuro engineering framework from Chris Eliasmith at Waterloo which is famously used to create SPAUN. There is also the Karlheinz Meier group at Heidelberg university which does wafer-scale networks of neurons in accelerated time. Giacomo Indiveri at ETH Zurich has silicon neurons with on-chip learning circuits that the others are missing.