4 ms·

Neural networks are really just a fancy statistical regression. Just the same as you could run a high-degree polynomial regression on some data and "not really

Neural networks are really just a fancy statistical regression. Just the same as you could run a high-degree polynomial regression on some data and "not really know" why it chose the coefficients it did or why it yields a particular output for another input, neural networks typically involve a lot of mysterious coefficients, and it's hard to qualitatively explain why the neural network gives a particular output for another input. But in both cases, the underlying principles and mechanism of action are very well understood.

argonaut 11y ago
That's not true. Depending on the method for polynomial regression, you do know why it chose the coefficients. Furthermore, you know exactly what your model looks like, because you're dictating what the model is explicitly! (... a polynomial). In contrast to a convolutional neural net where you might have no idea what the filters in the third (not first) convolutional layer are doing. This is also a pretty bad example because polynomial regression is not giving amazing groundbreaking results. Polynomial regression has been known for centuries and gives pretty meh results in the real world.
- nl 11y ago
  As an aside to argonaut's response (which is correct) it is important to note that there is a lot of ongoing work in understanding what the hidden layers in a CNN are doing. For image-classification type problems, it's pretty easy to pass an image in to the network and output the features each layer extracts. Indeed, many recent systems rely on CNN-extracted features being passed to a SVN classifier to classify images that the CNN wasn't trained on (because the SVN is easier to train will less data, and using the CNN frees one from doing feature engineering).
  - argonaut 11y ago
    We have a good understanding of the first CNN layer (they turn out to be edge/blob detectors). But later layers are not well understood.

argonaut 11y ago

That's not true. Depending on the method for polynomial regression, you do know why it chose the coefficients. Furthermore, you know exactly what your model looks like, because you're dictating what the model is explicitly! (... a polynomial). In contrast to a convolutional neural net where you might have no idea what the filters in the third (not first) convolutional layer are doing. This is also a pretty bad example because polynomial regression is not giving amazing groundbreaking results. Polynomial regression has been known for centuries and gives pretty meh results in the real world.

nl 11y ago
As an aside to argonaut's response (which is correct) it is important to note that there is a lot of ongoing work in understanding what the hidden layers in a CNN are doing. For image-classification type problems, it's pretty easy to pass an image in to the network and output the features each layer extracts. Indeed, many recent systems rely on CNN-extracted features being passed to a SVN classifier to classify images that the CNN wasn't trained on (because the SVN is easier to train will less data, and using the CNN frees one from doing feature engineering).
- argonaut 11y ago
  We have a good understanding of the first CNN layer (they turn out to be edge/blob detectors). But later layers are not well understood.