11 ms·
Obstacles on the Path to AI
- tomp 11y agoEverytime I read something like this, I get sad that I don't understand most of it. But then I get happy because at least I understand a little bit :)
- pzone 11y agoReading some random slides from a specialist talk is not exactly the easiest way to learn this stuff.
- lowglow 11y agorecs?
- draven 11y agoMost often the slides alone are useless, they are only supposed to be a support for the talk. Talks recordings (or transcripts) are more useful, you can only get a high level idea of what the talk is about from slides. I would be awesome to have a platform where you get recommendations of what to learn / read or online courses in order to understand a given talk.
- mkorfmann 11y agoYes absolutely! A platform where not only the answers are praised, but also curious questions.
- grondilu 11y agoThat's only the slides. Is there a video of the talk?(assuming there is a talk, that is)
- haddr 11y agoI'd love to see the video too... LeCun is probably most interesting person to hear in this topic...
- crazypyro 11y agoIn case you didn't see the comment originally (I didn't either), someone else posted a link. http://techtalks.tv/talks/whats-wrong-with-deep-learning/61639/ http://techtalks.tv/talks/whats-wrong-with-deep-learning/616...
- vkhuc 11y agoAlthough I'm working on deep neural nets, this material is too advanced to me. Looks like deep nets + bayesian reasoning is the next big thing.
- imh 11y agoPick up this book. It's fantastic. https://mitpress.mit.edu/books/probabilistic-graphical-models https://mitpress.mit.edu/books/probabilistic-graphical-model...
- vonnik 11y ago"There is no way in hell that you can learn billions of parameters with RL." I really love LeCun's provocative stances, but I get suspicious when people talk about impossibilities. RL is making huge strides. People adopted the same tone with neural nets years ago, and LeCun proved them wrong...
- paulsutter 11y agoHe's saying you won't learn billions is parameters with RL /alone/ because "one scalar reward per trial isn't going to cut it". I find that convincing and a key insight. RL is going to be fundamental to AGI, but he's saying curiosity / unsupervised learning will be necessary. And I say this as a big believer in the need for more work on RL.
- vonnik 11y agoYes, I should have added the context about the scalar reward, but I'm not sure why that changes anything. This may seem like a naive question, but it's sincere: What makes a scalar reward less effective at modifying a Q function than a scalar error that's used in backprop and assigned to a neural network's coefficients?
- jhartmann 11y agoThe error in backprop is a vector quantity, not a single scalar for each time step. In RL the goal is to optimize the overall sum of the reward over all time steps. Backprop attempts to minimize the magnitude of the error for a given loss function, by moving in the negative direction of the gradient of the function. Backprop just moves a lot more variables to a desired outcome, that is what LeCun is saying. The representational power of a single scalar 'score' doesn't have much ability to optimize such a large n-dimensional function in any efficient way.
- vonnik 11y agoMaybe I'm thinking about things wrong, but can't you have a scalar quantity to represent error for backprop at the end of a neural network (in a supervised regression problem for example)? That scalar error becomes a vector as it is assigned to various weights. I'm not trying to be obtuse, but it seems like the scalar reward in RL is also modifying countless variables in the Q functions of the state-action pairs that led to the final outcome/reward... Are those, by definition, smaller in number than the parameters of a neural network?
- deleted 11y ago[deleted]
- KasianFranks 11y agoI like the focus on vector space.
- ankurdhama 11y agoThe real obstacles to the path to AI is that we don't even have the right questions to ask and people are looking for answers to the wrong questions and announcing the next big thing in AI.
- thangalin 11y agoThe talk: http://techtalks.tv/talks/whats-wrong-with-deep-learning/61639/ http://techtalks.tv/talks/whats-wrong-with-deep-learning/616...
- cesarsalgado 11y agoThis is not the same talk. The "what's wrong with deep learning" talk was given in CVPR 2015. The slides linked in this HN post was presented in BayLearn: Bay Area Machine Learning Symposium.
- _0ffh 11y agoThanks for the link, but goodness gracious, 6 GB for one hour of low quality video? What are these people thinking, that internet bandwidth is bestowed on all of us from on high?