>These are placed on the outputs of a neuron and are one of those weird mathematical hacks where I can't really explain why it's needed, Because when you compo

>These are placed on the outputs of a neuron and are one of those weird mathematical hacks where I can't really explain why it's needed, Because when you compose linear functions you get linear functions. So having linear everything is a waste of all layers but one. In order for this not to happen, you need nonlinearity.