5 ms·

Very well written, and I applaud the effort. But personally I don't care for the "magical" aura that writers tend to give ANNs - to me, they are simply (non-lin

Very well written, and I applaud the effort. But personally I don't care for the "magical" aura that writers tend to give ANNs - to me, they are simply (non-linear) function approximators that have a nice fitting algorithm. They work well for some problems and poorly for others. Also, beware of over-fitting - ANNs tend to be parameter-heavy, although there are approaches to prune the connections.

thearn4 13y ago
The mysticism around ANN seems to come and go at least once per developer generation since the 50s
- davorak 13y ago
  I think this problem would be significantly less if there was no"neural" in "neural network." That said that one word has probably brought a significant number of eyes to the subject that would have looked elsewhere with out it.
Houshalter 13y ago
>they are simply (non-linear) function approximators that have a nice fitting algorithm. Couldn't "function approximator" describe most machine learning approaches? And a nice fitting algorithm is of course the goal.
- fpgaminer 13y ago
  I believe svantana was merely trying to differentiate ANNs from magic, not ANNs from other ML algorithms. Indeed you are correct that machine learning as a whole could be seen as function fitting.
- shoo 13y ago
  Yes. I'd go further - the "nice fitting algorithm" is to minimise the error on the training set as a function of the weight parameters, and one obvious way to (locally) minimise that is gradient descent + the chain rule. The math / applied math machinery is all incredibly general, and a useful way to think about many machine learning algorithms. http://en.wikipedia.org/wiki/Gradient_descent http://en.wikipedia.org/wiki/Gradient_descent http://en.wikipedia.org/wiki/Chain_rule#Higher_dimensions http://en.wikipedia.org/wiki/Chain_rule#Higher_dimensions Not to say that applied math is the only valuable perspective to think about, there's clearly also statistical and computational views as well. E.g. we're trying to approximate a function that we can never directly evaluate (error on the samples we havent seen yet)

thearn4 13y ago

The mysticism around ANN seems to come and go at least once per developer generation since the 50s

davorak 13y ago
I think this problem would be significantly less if there was no"neural" in "neural network." That said that one word has probably brought a significant number of eyes to the subject that would have looked elsewhere with out it.

Houshalter 13y ago

>they are simply (non-linear) function approximators that have a nice fitting algorithm. Couldn't "function approximator" describe most machine learning approaches? And a nice fitting algorithm is of course the goal.

fpgaminer 13y ago
I believe svantana was merely trying to differentiate ANNs from magic, not ANNs from other ML algorithms. Indeed you are correct that machine learning as a whole could be seen as function fitting.
shoo 13y ago
Yes. I'd go further - the "nice fitting algorithm" is to minimise the error on the training set as a function of the weight parameters, and one obvious way to (locally) minimise that is gradient descent + the chain rule. The math / applied math machinery is all incredibly general, and a useful way to think about many machine learning algorithms. http://en.wikipedia.org/wiki/Gradient_descent http://en.wikipedia.org/wiki/Gradient_descent http://en.wikipedia.org/wiki/Chain_rule#Higher_dimensions http://en.wikipedia.org/wiki/Chain_rule#Higher_dimensions Not to say that applied math is the only valuable perspective to think about, there's clearly also statistical and computational views as well. E.g. we're trying to approximate a function that we can never directly evaluate (error on the samples we havent seen yet)