4 ms·
> The gradient computation is only ill-defined at places where a spike would get added or deleted. I feel like I'm missing something here. Like if you do it na
by im3w1l 3y ago
> The gradient computation is only ill-defined at places where a spike would get added or deleted.
I feel like I'm missing something here. Like if you do it naively, the gradient is zero when a spike isn't added or deleted. And infinite when it is. Which is completely unhelpful.
Now the "natural" solution is to invent some differentiable approximation of the spiking network, and compute derivates of that, and hope that the approximation is close enough that optimizing it leads to the spiking network learning something useful.
A more principled version might be to inject some noise into the network. This would mean that you have a probability of spiking in a certain pattern (or better, a class of patterns that all have the same semantic). You could differentiate the probability of correct output with respect to the weights and try to drive it towards 1.
Is your approach in either of these classes?
- jackblemming 3y agoPeople have tried using probabilistic SNNs because the gradient is well defined there. The issue is it’s computationally intractable to calculate so you still have to use an approximation and you’re thus not better off than the people using a surrogate gradient of the non-probabilistic SNN.
- deleted 3y ago[deleted]
- im3w1l 3y agoI see now that this comment was based on a misunderstanding of what an SNN is and how it works, so feel free to disregard.