Yeah, not to be too dismissive or cliched, but these slides come off as if LeCun just now got around to reading some Andy Clark or reading neurosci/cogsci paper

Yeah, not to be too dismissive or cliched, but these slides come off as if LeCun just now got around to reading some Andy Clark or reading neurosci/cogsci papers by Tenenbaum or Friston. The idea that the human brain has to work via active prediction rather than passive signal processing has been well-established in cogsci and neurosci for a while now. The interesting question, then, is how we make machine-learning systems do prediction well. Probabilistic/generative models have been "wandering in the desert" for a while now because Monte Carlo methods are just so slow, especially for high-dimensional, hierarchical prediction problems like we want to solve in machine learning. On the upside, STAN now has automatic variational inference for continuous probability models, and work on things like the "concrete distribution" (https://arxiv.org/abs/1611.00712 https://arxiv.org/abs/1611.00712) can help us continuously approximate discrete probabilistic reasoning. Maybe as these techniques move into the mainstream in systems like Venture or Picture we can start to scale up predictive/generative/probabilistic modelling to match optimization-based connectionist methods?