3 ms·
Firstly, I'm coming at this from the pint of view of standard feed forward NNs, rather than spiking neural networks, so apologies in advance if I am way off-bas
by taylorius 2y ago
Firstly, I'm coming at this from the pint of view of standard feed forward NNs, rather than spiking neural networks, so apologies in advance if I am way off-base.
By global activation limit, I assume you mean the total sum of neuron activation during one forward pass? So halving it would strongly encourage sparsity? How do you enforce the reduced limit? Just put it in the loss function? Or does the spiking nature allow you to "sparsify" the spiking by raising a threshold?
Very very interesting stuff. I'm working with differential evolution, and incorporate correlation measures between candidates to (try to) allow for multiple divergent solutions, and it also feels fairly off the beaten track, compared to mainstream AI. But who cares, let's enjoy the dark forest! :-)
- bob1029 2y ago> By global activation limit, I assume you mean the total sum of neuron activation during one forward pass? Yes. For each candidate network I keep a counter of the total # of neuron activations. This figure is constrained by a global limit (which varies based on prior performance). We simply break out of the simulation loop once the allotted resources are exhauted. This means that more efficient networks are more likely to score higher.