4 ms·

Perhaps someone can clear up my confusion, but I classify neural networks as just one category of algorithms. It doesn't strike me as something that will replac

Perhaps someone can clear up my confusion, but I classify neural networks as just one category of algorithms. It doesn't strike me as something that will replace our binary systems anytime soon because, well, we haven't shown any systems being replaced. And I'm talking about things like an Operating System, programming language, audio driver, web browser, and so on. So I'm confused why "neuromorphic" chips are a big deal, apart from just being able to run our computer vision algorithms faster. Additionally while inspired by the brain, that's all they are, inspired. There is still hundreds of differences between our neural nets in our head and the ones we build on the computer. The article talks about "brain-inspired architectures that route for efficiency and performance.", but aren't our own brains efficient because they literally grow new pathways on demand? I feel like this sort of flexibility is something we will never see in silicon, and I don't think we even have a computational understanding of it at all at the moment.

skummetmaelk 10y ago
Brains do not really grow new pathways, they are more or less set in stone after development finishes. The goal of neuromorphic chips is also not to replace operating systems for the same reason that quantum computers won't replace conventional ones. They have strong sides and weak sides. When you say that all the networks are is inspired by biology, it is true, but varies hugely between implementations. Boahen is of the "true" neuromorphic school, people who use actual analog silicon neurons. These people are exclusively found in academia as all of industry only uses digital techonology. This includes the coveted IBM TrueNorth "neuromorphic" processor. Thus the goal of these projects is not just to create supercomputing architectures, but to create systems that mimic the way the brain works and thereby learn how it manages to perform all these complex operations.
- hacker_9 10y ago
  > Brains do not really grow new pathways, they are more or less set in stone after development finishes. Neuroplasticity and Neurogenesis has been observed well into adulthood though [1]. > Thus the goal of these projects is not just to create supercomputing architectures, but to create systems that mimic the way the brain works and thereby learn how it manages to perform all these complex operations. Sounds like a lot of expensive guesswork to me! Good luck to them I guess. [1] https://web.stanford.edu/group/hopes/cgi-bin/hopes_test/neuroplasticity/ https://web.stanford.edu/group/hopes/cgi-bin/hopes_test/neur...
  - skummetmaelk 10y ago
    Very true, but the neuroplasticity observed is markedly different from forming entirely new pathways. It is more like changing the weights of a network already in place. The neurogenesis is also limited to the hippocampus and olfactory bulb. Your cortex does not have a steady supply of new cells. Making these chips is really not all that expensive. Most groups produce only test chips in multi-project wafer runs, with a cost of 1-5k$ per chip. It is just like normal research, you have to make up experiments and then actually do them. So far there are a substantial number of successes in the domain of low-power neural networks.
  - emcq 10y ago
    There have been successes of this line of research. For example, touchpads and Synaptics came from thinking about how to build analog capacitor networks [0]. There are spiking cameras that can produce significantly higher range and temporal resolution than conventional approaches [1]. Like neural networks, the field sort of quieted down in the 90s but perhaps now is a new time for the field. If you view the purpose of academics to train the next generation rather than simply advancing the field, the quality of reasearchers and engineers that come out of Kwabenas lab is superb. They easily work at companies like Intel, SpaceX, NIH, etc. [1] https://inilabs.com/ https://inilabs.com/ [0] https://en.m.wikipedia.org/wiki/Carver_Mead https://en.m.wikipedia.org/wiki/Carver_Mead
sp332 10y ago
The neural networks that have exploded in popularity lately are nothing like actual neurons. They're usually just elements of a matrix, and the "layers" are just matrix multiplications. "Neuromorphic" ones are still pretty simple compared to the real thing but they are much more complex than the other neural networks. They try to physically model interactions between neurons, so each neuron is constantly sending and receiving and self-tuning. It's nothing like a matrix multiplication. Edit: It also runs efficiently on very different hardware. https://www.nextplatform.com/2017/02/15/memristor-research-highlights-neuromorphic-device-future/ https://www.nextplatform.com/2017/02/15/memristor-research-h...
fragapanagos 10y ago
Indeed we do not have a good, computational understanding of what makes brains efficient. It's plausible that growing new pathways could contribute to the efficiency of the brain, but is that the only reason the brain is efficient? What about the brain's spiking communication? What about the synaptic and dendritic dynamics? Are any of these "features" useful or just vestigial byproducts of evolution? Without that computational understanding, it's hard to make satisfying arguments one way or the other about what exactly makes the brain efficient. You can cast neuromorphic chip development as a path to realizing that understanding. Recall how convolutional neural networks only realized their heyday once the hardware resources to implement them (i.e. GPUs) was readily available. Once GPUs made running CNNs practical, people found all sorts of new ways to understand and use neural networks. In other works, the computational model and the implementing hardware must be co-developed before something actually useful is possible.
Seanny123 10y ago
As I mention in the blog post I link to in my comment [1], the idea isn't to replace. As for whether spiking neural nets are true representations of the human brain, I think it comes down to choosing the right level of abstraction for the problem. Should your model include neurogenesis? Should you stop at the quantum level? I think the answer is, that it depends. [2] [1] https://news.ycombinator.com/item?id=14078487 https://news.ycombinator.com/item?id=14078487 [2] https://www.sciencedirect.com/science/article/pii/S095943881300189X https://www.sciencedirect.com/science/article/pii/S095943881...

skummetmaelk 10y ago

Brains do not really grow new pathways, they are more or less set in stone after development finishes. The goal of neuromorphic chips is also not to replace operating systems for the same reason that quantum computers won't replace conventional ones. They have strong sides and weak sides. When you say that all the networks are is inspired by biology, it is true, but varies hugely between implementations. Boahen is of the "true" neuromorphic school, people who use actual analog silicon neurons. These people are exclusively found in academia as all of industry only uses digital techonology. This includes the coveted IBM TrueNorth "neuromorphic" processor. Thus the goal of these projects is not just to create supercomputing architectures, but to create systems that mimic the way the brain works and thereby learn how it manages to perform all these complex operations.

hacker_9 10y ago
> Brains do not really grow new pathways, they are more or less set in stone after development finishes. Neuroplasticity and Neurogenesis has been observed well into adulthood though [1]. > Thus the goal of these projects is not just to create supercomputing architectures, but to create systems that mimic the way the brain works and thereby learn how it manages to perform all these complex operations. Sounds like a lot of expensive guesswork to me! Good luck to them I guess. [1] https://web.stanford.edu/group/hopes/cgi-bin/hopes_test/neuroplasticity/ https://web.stanford.edu/group/hopes/cgi-bin/hopes_test/neur...
- skummetmaelk 10y ago
  Very true, but the neuroplasticity observed is markedly different from forming entirely new pathways. It is more like changing the weights of a network already in place. The neurogenesis is also limited to the hippocampus and olfactory bulb. Your cortex does not have a steady supply of new cells. Making these chips is really not all that expensive. Most groups produce only test chips in multi-project wafer runs, with a cost of 1-5k$ per chip. It is just like normal research, you have to make up experiments and then actually do them. So far there are a substantial number of successes in the domain of low-power neural networks.
- emcq 10y ago
  There have been successes of this line of research. For example, touchpads and Synaptics came from thinking about how to build analog capacitor networks [0]. There are spiking cameras that can produce significantly higher range and temporal resolution than conventional approaches [1]. Like neural networks, the field sort of quieted down in the 90s but perhaps now is a new time for the field. If you view the purpose of academics to train the next generation rather than simply advancing the field, the quality of reasearchers and engineers that come out of Kwabenas lab is superb. They easily work at companies like Intel, SpaceX, NIH, etc. [1] https://inilabs.com/ https://inilabs.com/ [0] https://en.m.wikipedia.org/wiki/Carver_Mead https://en.m.wikipedia.org/wiki/Carver_Mead

sp332 10y ago

The neural networks that have exploded in popularity lately are nothing like actual neurons. They're usually just elements of a matrix, and the "layers" are just matrix multiplications. "Neuromorphic" ones are still pretty simple compared to the real thing but they are much more complex than the other neural networks. They try to physically model interactions between neurons, so each neuron is constantly sending and receiving and self-tuning. It's nothing like a matrix multiplication. Edit: It also runs efficiently on very different hardware. https://www.nextplatform.com/2017/02/15/memristor-research-highlights-neuromorphic-device-future/ https://www.nextplatform.com/2017/02/15/memristor-research-h...

fragapanagos 10y ago

Indeed we do not have a good, computational understanding of what makes brains efficient. It's plausible that growing new pathways could contribute to the efficiency of the brain, but is that the only reason the brain is efficient? What about the brain's spiking communication? What about the synaptic and dendritic dynamics? Are any of these "features" useful or just vestigial byproducts of evolution? Without that computational understanding, it's hard to make satisfying arguments one way or the other about what exactly makes the brain efficient. You can cast neuromorphic chip development as a path to realizing that understanding. Recall how convolutional neural networks only realized their heyday once the hardware resources to implement them (i.e. GPUs) was readily available. Once GPUs made running CNNs practical, people found all sorts of new ways to understand and use neural networks. In other works, the computational model and the implementing hardware must be co-developed before something actually useful is possible.

Seanny123 10y ago

As I mention in the blog post I link to in my comment [1], the idea isn't to replace. As for whether spiking neural nets are true representations of the human brain, I think it comes down to choosing the right level of abstraction for the problem. Should your model include neurogenesis? Should you stop at the quantum level? I think the answer is, that it depends. [2] [1] https://news.ycombinator.com/item?id=14078487 https://news.ycombinator.com/item?id=14078487 [2] https://www.sciencedirect.com/science/article/pii/S095943881300189X https://www.sciencedirect.com/science/article/pii/S095943881...