4 ms·
Sigmoid Colon: The biologically inspired activation function
- version_five 4y agodef sigmoidcolon(x): return -0.00019838297921413554 * x ** 3 + 0.13378547907766572 * x ** 2 + -29.00155803868406 * x ** 1 + 2345.683462308047
- tantalor 4y agoLittle early, SIGBOVIK isn't for a few weeks.
- KRAKRISMOTT 4y agoWhat's the paired loss function? Is cross entropy still computationally/mathematically efficient with this function?
- poopingface 4y agoThis activation function is meant to substitute non-linearities in the middle of the neural network (e.g. in the place of ReLU and similar). Since the output is unbounded, you wouldn't want to use it in place of the sigmoid function for binary cross entropy. Furthermore it operates on each output individually, making it a poor substitution for softmax prior to a cross entropy loss.
- deleted 4y ago[deleted]
- KRAKRISMOTT 4y agoSo what's the recommended residue function?
- redox99 4y agoAs long as it's differentiable, and the derivative isn't too crazy to calculate it's fine. In this case Sigmoid Colon is a third degree polynomial. Computationally it should be fine, but the cube term probably means that you will have numerical stability issues if your data is not very normalized.
- mjhay 4y agoThis post stinks.
- stevefan1999 4y agoThis is going to be the shit post of the year
- csense 4y agoA possible contender for the Ig Nobel prize.