> all it does is clamp negative values to zero. How could a network use that for learning? This "simple" effect is actually huge because it allows non linear m

> all it does is clamp negative values to zero. How could a network use that for learning? This "simple" effect is actually huge because it allows non linear mapping between input and output. This, is completely changing the size of what's learnable.