This activation function is meant to substitute non-linearities in the middle of the neural network (e.g. in the place of ReLU and similar). Since the output is

This activation function is meant to substitute non-linearities in the middle of the neural network (e.g. in the place of ReLU and similar). Since the output is unbounded, you wouldn't want to use it in place of the sigmoid function for binary cross entropy. Furthermore it operates on each output individually, making it a poor substitution for softmax prior to a cross entropy loss.