5 ms·

> Does not need to be trained? The writer maybe wanted to say it does not need to be programmed in the traditional way. There are two ways to build spiking neu

> Does not need to be trained? The writer maybe wanted to say it does not need to be programmed in the traditional way. There are two ways to build spiking neural net architectures without training. The [Neural Engineering Framework](https://medium.com/@seanaubin/a-way-around-the-coming-performance-walls-neuromorphic-hardware-with-spiking-neurons-facd4291b201 https://medium.com/@seanaubin/a-way-around-the-coming-perfor...) and [Deneve's approach which I'm more uncertain about](https://forum.nengo.ai/t/robustness-of-deneves-networks/320?u=seanny123 https://forum.nengo.ai/t/robustness-of-deneves-networks/320?...). To be clear, when I say "training" in this context, I mean they don't require iterative rounds of stochastic gradient descent. They just need to be given the function that you want to approximate and then they're done.

visarga 9y ago
Yes, maybe it uses something other than gradient descent, but why? Nothing works nearly as well by comparison. In a chip we can do things we can't do in the brain, such as sending gradients back on the same connections.
- Seanny123 9y ago
  > Nothing works nearly as well by comparison. Depends on what you mean by "well". It would be foolish of me to deny the advances of deep learning. However, there are limitations and the NEF can act as a bridge in those cases. For example, if you want: 1. To approximate a dynamic system 2. To use symbolic computation 3. Use the neural networks as an initial guess at a solution to a problem and then want to use gradient descent to figure out the actual solution I talk about using NEF as a bridge in [this blog post](https://medium.com/@seanaubin/deep-learning-is-almost-the-brain-3aaecd924f3d https://medium.com/@seanaubin/deep-learning-is-almost-the-br...), if you're curious. Unfortunately, I don't know of any examples using the "initial guess" paradigm that I describe, but that's more of a "not enough people using Nengo" problem.
hvis 9y ago
What do you mean by "give the function"? If you can do that, either by writing a deterministic algorithm, or even defining it analytically, there's no need for machine learning, neural networks or etc.
- Seanny123 9y ago
  > there's no need for machine learning, neural networks or etc. There is if: 1. You want an energy-efficient distributed representation of the function. 2. You want to use the function as an initial guess and then learn online afterwards.

visarga 9y ago

Yes, maybe it uses something other than gradient descent, but why? Nothing works nearly as well by comparison. In a chip we can do things we can't do in the brain, such as sending gradients back on the same connections.

Seanny123 9y ago
> Nothing works nearly as well by comparison. Depends on what you mean by "well". It would be foolish of me to deny the advances of deep learning. However, there are limitations and the NEF can act as a bridge in those cases. For example, if you want: 1. To approximate a dynamic system 2. To use symbolic computation 3. Use the neural networks as an initial guess at a solution to a problem and then want to use gradient descent to figure out the actual solution I talk about using NEF as a bridge in [this blog post](https://medium.com/@seanaubin/deep-learning-is-almost-the-brain-3aaecd924f3d https://medium.com/@seanaubin/deep-learning-is-almost-the-br...), if you're curious. Unfortunately, I don't know of any examples using the "initial guess" paradigm that I describe, but that's more of a "not enough people using Nengo" problem.

hvis 9y ago

What do you mean by "give the function"? If you can do that, either by writing a deterministic algorithm, or even defining it analytically, there's no need for machine learning, neural networks or etc.

Seanny123 9y ago
> there's no need for machine learning, neural networks or etc. There is if: 1. You want an energy-efficient distributed representation of the function. 2. You want to use the function as an initial guess and then learn online afterwards.