Why would you want to approximate tanh for the use in neural networks? Every smoothed step function will do, so if your concern is speed, why not design somethi

Why would you want to approximate tanh for the use in neural networks? Every smoothed step function will do, so if your concern is speed, why not design something for speed, who cares if it is an established mathematical function? Because you might also need the derivative and tanh(x) has a quite nice one with 1 - tanh²(x) that is cheap to compute if you already have tanh(x)?