3 ms·
The family of sigmoid functions has nice gradient properties with theoretical backing. Good starting read: https://stats.stackexchange.com/questions/162988/why-
by sbszllr 3y ago
The family of sigmoid functions has nice gradient properties with theoretical backing. Good starting read: https://stats.stackexchange.com/questions/162988/why-sigmoid-function-instead-of-anything-else/318209#318209 https://stats.stackexchange.com/questions/162988/why-sigmoid...
- lostmsu 3y agoDoesn't really explain why not more typical ReLU + layer norm or some alternative.