Not really, no. That's motivated by not getting impractically small gradients on the plateaus and spoiling the optimization properties when used for deep ANNs.

Not really, no. That's motivated by not getting impractically small gradients on the plateaus and spoiling the optimization properties when used for deep ANNs. The sigmoids it replaced had a bit more neuroscience inspiration, but so oversimplified it's just barely.