5 ms·
I assume that splicing together two decaying exponential functions at the origin would work just as well in practice. Also known as tanh: https://en.wikipedia.
by MauranKilom 5y ago
I assume that splicing together two decaying exponential functions at the origin would work just as well in practice.
Also known as tanh: https://en.wikipedia.org/wiki/Hyperbolic_functions https://en.wikipedia.org/wiki/Hyperbolic_functions
One "disadvantage" is that it doesn't saturate to [-1.0, 1.0] like appropriately scaled tanh.
- CodesInChaos 5y agoBy splicing together I mean a piecewise function which is `exp(x) - 1` on the left and `1 - exp(-x)` on the right. Which should be similar enough to tanh for most purposes.
- stephencanon 5y agoSure, it even has continuous derivatives of all orders and the right slope at the origin. It just doesn’t saturate to +/-1 as fast, which probably doesn’t matter.