This is fascinating because one of the first things they teach you in ML class is why nonlinear activations are necessary -- because otherwise, your entire netw

This is fascinating because one of the first things they teach you in ML class is why nonlinear activations are necessary -- because otherwise, your entire network is mathematically equivalent to a single linear transformation since linear transformations are composable! I'm going to have to read through this paper because I'm really curious if they posit any theories as to why removing nonlinear transforms in their model had such a positive effect in this particular scenario.