I have always thought of it the other way around, neural networks are a kind of transform similar to the Fourier transform. But you know, instead of a sum of si

I have always thought of it the other way around, neural networks are a kind of transform similar to the Fourier transform. But you know, instead of a sum of sinusoid components you get a sum of weighted <insert your activation function>.