61 ms·
So my impression of the way that Face recognition works is that the neural-net part of the system recognizes the faces in a photo, recognizes the features on th
by joe_the_user 8y ago
So my impression of the way that Face recognition works is that the neural-net part of the system recognizes the faces in a photo, recognizes the features on the faces and provides a map of the features to the system. The system takes these parameters and stores and recovers a given face with them. The ability to recognize individual faces comes because conveniently these parameters don't change much (but clearly it hinges on this). Of course, there are a lot of small enhancements to this process and these evolve but this is still basic outline.
Which is to say that convolutional neural networks, the most common and successful neural networks, fundamentally only operate by recognizing a broad category (face, nose-on-face, mouth-on-face). This limitation seems as inherent in NNs operating with a black-box/training-set/testing-set cycle as it is in any architecture consideration. If all you can "tell" the NN is "this is X, this is not X", then it's plausible that's all it will give back. Moreover, "X/not-X" testing allows a LOT of training to happen in an unambiguous fashion, which seems necessary when what you're doing gradually pushing a giant bunch of semi-arbitrary detectors into place.
Machine learning theorists can do a lot with nothing but this category-recognition tool - like having a go program recognize "good moves" and "good positions". Note, that the more elaborate "neural net within neural net" approaches involve vast levels of computing power - you're brute-forcing brute-force. AlphaGoZero probably involved the largest level of computing power ever harnessed for a single-use app. AlphaGoZero involve megaflop levels close to those estimated for the human brain though such estimates are always debatable (perhaps why we didn't see many games by it).
The thing is there are plenty of ordinary computing algorithms do "more" than neural nets - perform logical operations or sort and search by vague similarity or classify on spectrum of multiple qualities (rather than the binary X or not-X of a neural network). It is just that these break down at an "industrial scale", whereas neural nets are shine at this scale.
However, despite these virtues, it seems likely we'll see neural nets approaching their particular limits reasonably soon, if they haven't (in at least a sense) done so already.
And it seems like any "real" extension/alternative will have to be able to use big data but use it in a more sophisticated fashion than neural nets.
Oh, this relates to the video most in that when Bengio talks about how to extend neural networks, he's mostly just mentioning extension in terms of how to add abstract qualities like "causation", where one can certainly see crude, simplistic things in computation that could be called "working with causation" but you don't have that yet when you go to the level of a large-scale neural network.
- mgurlitz 8y agoNNs are not binary -- they're fundamentally analog, so for classification problems, they produce probabilities that a test case is in each possible class. A binary "X/not-X" test often comes from applying a threshold to the NN's output. Quoting from the Universal approximation theory's Wikipedia, "neural networks can represent a wide variety of interesting functions." While they may be much better at pattern recognition, it's possible to produce almost anything with one, including Go moves, if you can devise a method to interpret the outputs.
- iandanforth 8y ago>> NNs are not binary -- they're fundamentally analog That's incorrect. https://arxiv.org/abs/1602.02830 https://arxiv.org/abs/1602.02830
- candiodari 8y agoYou could say they're "probabilistically binary", that would at least be intuitively accurate.
- joe_the_user 8y agoNNs are not binary -- they're fundamentally analog, so for classification problems, they produce probabilities that a test case is in each possible class. I think that's "true and false". Yes, NNs produce a "chance of being X" but no, NNs don't naturally produce a human-intuitive "degree of being X". If the NN is looking for a "red fire engine", it's not necessarily going to produce "degree of redness" as its output. -- As to neural nets representing many functions; neural nets, Taylor series, Fourier series and so-forth can represent/approximate "any function". True but effectively irrelevant. What matters is what function a given methodology can be trained, programmed or whatever into representing.
- glass_of_water 8y ago> Note, that the more elaborate "neural net within neural net" approaches involve vast levels of computing power... This is one thing I worry about, that even if we were to get closer to AGI, the algorithms/system might not feasibly run with our current computing paradigm.