4 ms·
"Here's what you should remember: the only real success of deep learning so far has been the ability to map space X to space Y using a continuous geometric tran
by andreyk 9y ago
"Here's what you should remember: the only real success of deep learning so far has been the ability to map space X to space Y using a continuous geometric transform, given large amounts of human-annotated data."
This statement has a few problems - there is no real reason to interpret the transforms as geometric (they are fundamentally just processing a bunch of numbers into other numbers, in what sense is this geometric), and the focus on human-annotated data is not quite right (Deep RL and other things such as representation learning have also achieved impressive results in Deep Learning). More importantly, saying " a deep learning model is "just" a chain of simple, continuous geometric transformations " is pretty misleading; things like the Neural Turing Machine have shown that enough composed simple functions can do pretty surprisingly complex stuff. It's good to point out that most of deep learning is just fancy input->output mappings, but I feel like this post somewhat overstates the limitations.
- deleted 9y ago[deleted]
- emodendroket 9y agoPresumably that's why the word "just" is in scare quotes.
- backpropaganda 9y agoJust because there's a paper on it, and the model has a name, doesn't mean it works. NTM and deep RL don't work for real problems.
- divideby0829 9y agoyeah this was my main problem, I guess he is technically right because they are geometric but many of his analogies like the paper crumpling were deeply misleading as they would imply that the transformations are linear. The fact that they are not is fundamental to neural networks working.
- tsbertalan 9y agoPaper-crumpling is nonlinear. Maybe your complaint is rather that paper-crumpling is "only" a topology-preserving diffeomorphism?