3 ms·
I don't understand all of the criticism here. This is Ch. 4 in a basic intro to neural networks. The author provides a well written, intuitive, and concise demo
by nautilius 5y ago
I don't understand all of the criticism here. This is Ch. 4 in a basic intro to neural networks. The author provides a well written, intuitive, and concise demonstration of how sigmoids can be pieced together to approximate functions, in the layman's sense of the word. It helped me build intuition when I came across this maybe 5 (?) years ago. There are other good 'Chapters' about vanishing gradient or backprop.
The criticism here is mostly about things that aren't even the topic of the article: 'it cannot compute non-computable functions', 'it cannot predict tomorrow's stock price', 'splines are more efficient', 'it cannot predict how a stick looks bent in water'.
It's like saying 'I read "Zen and the art of motorcycle maintenance, twice, and still don't know how to adjust spark plugs. Stupid book"'.
- GTP 5y agoThe criticism is that is over-representing neural networks' capabilities: saying that they can compute any function is a very big and incorrect claim, IMO it should at least mention that non-computable functions are a thing and of course a neural network can't have better capabilities than the machine that is running it. This is especially because there are people out there thinking that machine learning is able to solve any possible problem, if one of those people comes across this proof they'll think: "Neural netwoks really can solve any conceivable problem! Look, there's even a mathematical proof of this!"
- renewiltord 5y agoIt’s funny, but this is how I felt about my son’s kindergarten. They taught him about “numbers” but instead they only used Natural numbers. The least they could have done is mention at least the one-point compactification of that space. Fortunately, I was able to intervene before he was permanently damaged by the class. He’s still struggling with showing that it is homeomorphic to the subspace of R formed by {1/n for all n in R that are in N} union {0} and I blame today’s pedagogy for that.
- GTP 5y agoThe context is different, the problem here is the hype that goes in a very specific direction (i.e. you can do anything with ML) as I briefly explained in another comment
- nautilius 5y agoI feel "Numbers" are pretty hyped-up!
- GTP 5y agoGoing back to your original comment, if I say "let's talk about linear functions" and then all I say also applies to a more general context, eg. polynomials, then it is fine. But if I say "let's talk about polynomials" and then all I say only applies to linear functions then it is wrong. Your sarcastic comment falls under the first case, the article discussed here falls under the second.
- nautilius 5y agoI wish you had read my original comment, where, let me repeat it a second time for you, I state that the author writes about how to approximate a 'function in the laymen sense of the word'. And that's what he does. Nowhere does he bring hype into it; no one reading about how to approximate a simple y=f(x) will misread this. My guess is that you read the website like you read my comment, if at all, otherwise we wouldn't be having this ridiculous conversation. renewiltord's comment is spot on and much better phrased than I could have done it.
- GTP 5y agoI got that you say that the author is talking about a 'function in the laymen sense of the world', what I'm saying is that this might not be clear from reading that webpage without some prior knowledge of the cases when this will fail.
- deleted 5y ago
- nautilius 5y agoAs I wrote, 'function in the laymen sense of the word'. And no one actually reading this can, without malice, misunderstand this: the author uses a nice and smooth y=f(x) as an example, and shows that it can be approximated with sigmoids. Nothing more, nothing less. And he does a good job showing this.
- GTP 5y agoYou say no one can misunderstand what the author wrote without malice, but there is a saying that reads "you should not attribute to malice what can be attributed to ignorance". Especially among people outside of CS, there is currently a lot of hype regarding ML and AI in general, if one of those people looks around to check if a neural network can really do anything and founds this, what would they understand?
- nautilius 5y agoYou're telling me you were confused by what the author could possible mean with his nice and smooth y=f(x) and that you misunderstood him that really he was implying that GAI is just around the corner? The whole discussion here is one long list of "Gotcha!", missing the point and the context of the article completely. You think these non-CS people you evoke will be too dense to understand that the author is just approximating a simple function, and then will leverage their non-CS background and immediately conclude that ANN can 'solve' Cantor-dust, non-computability and the halting problem? That's a very specific set of background your non-CS people will have to have to fall for that. [Edit:] What do you think they'll make of "Multilayer feedforward networks are universal approximators"? ("This paper rigorously establishes that standard multilayer feedforward networks with as few as one hidden layer using arbitrary squashing functions are capable of approximating any Borel measurable function from one finite dimensional space to another to any desired degree of accuracy, provided sufficiently many hidden units are available. In this sense, multilayer feedforward networks are a class of universal approximators.")
- GTP 5y agoThe people I'm referring to just don't know that there exist non-computabke function. So they wouln't draw specific conclusion about problems like the halting problem and silmilar because they don't know them, they would just conclude that any problem they might face can be solved by a NN. That's all.