Respect to the OP as spiking networks are a different domain, but the field of Graceful Degradation and Perturbations itself is almost older than the field of N

Respect to the OP as spiking networks are a different domain, but the field of Graceful Degradation and Perturbations itself is almost older than the field of Neural Networks itself. Generally, the bitter lesson occurs - these techniques show some kind of useful robustness at parameter M, but fail to show as much of a useful robustness relatively as parameter M*100. For example, before attention, using Drop out (in essence a form of forcing robustness) was one of the most common techniques. The quality of robustness or its resulting sub parts (e.g. the tradeoff between precision and recall) is incorporated as a measure in some of the fields basic scores, such as harmonic F1 score. Still it, like alternate activation functions, data beyond matrices and vectors and non-linear networks, remain very unstudied.