7 ms·
Predictive Learning [pdf]
- lowglow 10y agoFor anyone that doesn't know Yann LeCun, he's the head of AI over at Facebook, but surprisingly he's positively and consistently straightforward concerning the current hype driving AI and its technologies. He deserves respect because of this alone.
- hyperopia 10y agoYour Asteria AI project looks pretty sweet.
- lowglow 10y agoThanks, are you working in AI? You should join our gitter channel.
- hackinthebochs 10y agoNot to mention the father of CNNs.
- escap 10y agoit's a bit more complicated: https://en.wikipedia.org/wiki/Convolutional_neural_network#History https://en.wikipedia.org/wiki/Convolutional_neural_network#H... https://www.quora.com/Who-invented-convolution-neural-networks https://www.quora.com/Who-invented-convolution-neural-networ...
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- jamesblonde 10y agoWho are you? Joergen Schmidthueber?
- mac01021 10y ago"Straightforward" meaning what? Just bluntly dismissive of undue hype? What statements has he made to this effect?
- lowglow 10y agoPretty much this in general. Check out any of his social profiles as he's pretty outspoken about his thoughts and opinions.
- earthly10x 10y agoMemory Modules in vector space reminds me of some earlier work described as Associative Memory Modules (AMMs) http://sumve.com/biomimetic-cognition/biomimetic-api.html http://sumve.com/biomimetic-cognition/biomimetic-api.html
- ehudla 10y agoInteresting how what "common sense" is evolved since the days of symbolic AI (cf. Cyc). The history presented begins rather late; show some love to the founders of the field...
- ehudla 10y agoFor Cyc, see here: https://news.ycombinator.com/item?id=11300567 https://news.ycombinator.com/item?id=11300567
- vowelless 10y agoWill the other slides be posted? And how about videos?
- chessweb01 10y agoIf page 33 depicts the working of the brain on a very high level, the world model (or simulator) residing inside the agent must contain a model/simulator of the agent itself. Could this give rise to self perception or consiousness?
- azinman2 10y agoI wouldn't assume it must, although it would be neat if it did. There are many ways to get agents to react to situations without self-awareness. In fact, the agents themselves can be decomposed into many otherwise incompatible sub-agents. See Minsky's Society of Mind.
- sytelus 10y agothis is an excellent point. This model has to be recursive and may be depth of recursion has something to do with self-perception?
- quonn 10y agoI don't understand this argument. (It keeps coming up.) The two issues I have with this are: a) Self-perception does not seem like consciousness to me at all. In meditation, if done properly, there is very little self left. It feels more like pure awareness. It is almost the opposite of the model of the self that the brain constructs. b) I fail to see how the fact that a mechanism refers to itself should somehow give rise to the feeling of conciousness. Why would it? Nobody would predict consciousness from that if we would not already know it exists and it's easy to imagine a device that has a model of itself and is not self-aware. I understand the need to somehow fit this into our scientific framework, and the idea that "consciousness is just what it feels to have a brain" is the best thing we have, but I don't think it explains anything. There is something we are missing.
- donmatito 10y agoAnswers to "can we give consciousness to AI" is heavily dependent on how you define consciousness in the first place. Many definitions can coexist, some more actionable than others. "Being aware of the existence of oneself in the world, and being able to reflect on oneself's decision" seems relatively practical. So, Self-perception + self-reflection = consciouness (as a definition) From this starting point, it seems reasonable to derive that consciousness can arise from 1) mental representation of the world that include oneself 2) empathy for others (I can guess why this other worker has taken this decision) that, once applied to the actions of the self as if it were an external agent, gives self-reflection.
- raverbashing 10y agoVery interesting set of slides It seems the current state of prediction is only slightly better than the state of image recognition pre multiple level NNs There might be still a theoretical jump that's needed
- felippee 10y agoYes, the field has emerged out of MNIST/ImageNet and that is what those algorithms are optimised for. For modelling actual dynamics different design is necessary. It happens that the design that makes sense also seems to agree very well with the observed biology of the cortex. You can find links to our Predictive Vision Model in this thread as well as a few additional thoughts here: http://blog.piekniewski.info/2016/11/30/learning-physics-is-the-way-to-go/ http://blog.piekniewski.info/2016/11/30/learning-physics-is-...
- sushirain 10y agoMy summary of the slides: to achieve "common sense", Unsupervised Learning is the the next step. Use all information in the data for predicting everything you can including the past. Use adversarial networks to do that.
- daveytea 10y agoHe also presented at CMU Robotics a few weeks ago (and used these slides). Video here: https://youtu.be/IbjF5VjniVE https://youtu.be/IbjF5VjniVE
- rdudekul 10y agoAnother good one here: https://www.youtube.com/watch?v=Gwad1cWMcC0 https://www.youtube.com/watch?v=Gwad1cWMcC0
- edtufte 10y agoThese slides remind me of the worst PowerPoints ever.
- thomasahle 10y agoReally? They have pictures and everything.
- MawNicker 10y agoA wonderful demonstration of copyright violation/enforcement.
- pakl 10y agoLecun has identified a real problem for AI -- the need to understand the real world, the link between intelligence and prediction over time. But the tools he is using are not the right ones. Deep conv nets were not designed with prediction over time in mind. Here's one reason: Deep convolutional nets do not handle dynamical information from their lowest layers. By design, conv layers and pooling layers immediately begin discarding spatial information that could be used for building up predictions. In contrast it is possible to start with recurrent/feedback networks at the very first layers of the network. These initial layers can begin building up predictions at the pixel, color, and lighting level (example: our recent preprint[1]). My colleague, who as some of you know enjoys blogging, wrote a more thorough post in response to LeCun's recent CMU lecture on the same topic as these slides[2]. [1] https://arxiv.org/abs/1607.06854 https://arxiv.org/abs/1607.06854 [2] Blog: "A few comments on the Yann LeCun lecture at CMU, 11.2016" http://blog.piekniewski.info/2016/11/21/yann-lecun-cmu-11-2016-comments/ http://blog.piekniewski.info/2016/11/21/yann-lecun-cmu-11-20...
- felippee 10y agoHi, I'm the blogger. I just want to add a very simple statement: In order to create a model of the world, the machine learning substrate has to have the capacity to cover the observed dynamics. Hard not to agree with this, almost sounds like a tautology. Now let's try to derive conclusions: - World dynamics is full of multi-scale interactions (e.g. in vision illumination of a single pixel depends on the whole scene and the whole scene depends on many tiny details). To capture that, the machine learning substrate has to allow low level representations access high level stuff. Hence feedback all over the place. Which is exactly what is seen in the biological cortex. This is not recurrent layer made of LSTMs, this is a FULLY RECURRENT system. This will not be achieved with any franken-neocognitron deep network neither with MSE, nor adversarial nor even triple adversarial loss function. This requires a new approach and together with several colleagues after a few years of continuous and intense thinking and modelling we have proposed a solution: http://blog.piekniewski.info/2016/11/04/predictive-vision-in-a-nutshell/ http://blog.piekniewski.info/2016/11/04/predictive-vision-in... As well as a full paper https://arxiv.org/abs/1607.06854 https://arxiv.org/abs/1607.06854 Now, I'm not saying this is all done. This is just a beginning of a really exciting research adventure and many things look very promising. It will require however for the AI field to get out of a pretty "deep" local minimum it is in right now.