> The other is that current neural networks are probably very inefficient algorithms. They are fundamentally based on linear algebra but there is some evidence

> The other is that current neural networks are probably very inefficient algorithms. They are fundamentally based on linear algebra but there is some evidence that the non-linearity is the key. Deep neural networks, though they use linear algebra, are definitely non linear, as a result of both their architecture and their use of non linear activation functions. Their non-linearity is part of the reason they can classify phenomena that linear techniques like simple regression cannot.