3 ms·
Do we even understand how a neuron works well enough to say that either thing is correct. I thought a relu was just the activation function that decides what we
by andy_ppp 6y ago
Do we even understand how a neuron works well enough to say that either thing is correct. I thought a relu was just the activation function that decides what weight to triggers the next part of the network with?
- freeone3000 6y agoWe understand enough about how a biological neuron works to know that single signals do not provoke a diminished response after the hysterisis period, which is not modeled by either a memristor or a sigmoid. Sigmoids don't have hysterisis, and you have to backflow a memristor for it to recover which is not true of biological nuerons. (The activation function is the neuron('s mathematical model) - we elide everything that happens internal to the neuron and model it as only its inputs and outputs.)