When we don’t have a model of the problem it takes enormous amounts of power to synthesize one out of neurons or another general function approximating primitiv

When we don’t have a model of the problem it takes enormous amounts of power to synthesize one out of neurons or another general function approximating primitive. But once we understand a little bit about the problem we can model 80-90% of its behavior with a handful of parameters. Add in some bias and noise parameters and you have an accurate trainable machine learning model that’s orders of magnitude more efficient. Take for example a spring which can be modeled by 1 or 2 parameters. But its impulse response looks like a sin curve multiple by exponential decay. If you just train neurons to match input/outputs from a spring you need a ridiculous number of model parameters to describe that shape. CNNS have seen an enormous amount of success due to this fact: a lot of processes can be modeled by convolution.