3 ms·
What's the paired loss function? Is cross entropy still computationally/mathematically efficient with this function?
by KRAKRISMOTT 4y ago
What's the paired loss function? Is cross entropy still computationally/mathematically efficient with this function?
- poopingface 4y agoThis activation function is meant to substitute non-linearities in the middle of the neural network (e.g. in the place of ReLU and similar). Since the output is unbounded, you wouldn't want to use it in place of the sigmoid function for binary cross entropy. Furthermore it operates on each output individually, making it a poor substitution for softmax prior to a cross entropy loss.
- deleted 4y ago[deleted]
- KRAKRISMOTT 4y agoSo what's the recommended residue function?
- redox99 4y agoAs long as it's differentiable, and the derivative isn't too crazy to calculate it's fine. In this case Sigmoid Colon is a third degree polynomial. Computationally it should be fine, but the cube term probably means that you will have numerical stability issues if your data is not very normalized.