4 ms·In my totally unscientific experiments, tanh worked better on shallow networks.by walrus 10y agoIn my totally unscientific experiments, tanh worked better on shallow networks.