3 ms·

Can anyone point to any reference where models depart from "closely-supervised end-to-end learning"? In the article, LeCun & Manning argue this paradigm has so

by nextos 9y ago

Can anyone point to any reference where models depart from "closely-supervised end-to-end learning"? In the article, LeCun & Manning argue this paradigm has some limitations, and I do agree. I'm thinking the field will evolve to systems becoming a combination of probabilistic logic-based engines (which represent formal causal reasoning) plus lots of deep models (which represent intuition, hypothesis generation and specialized tasks like vision).

make3 9y ago
"Unsupervised Neural Machine Translation", https://arxiv.org/abs/1710.11041 https://arxiv.org/abs/1710.11041
nl 9y ago
There are lots of examples. In the NLP space, Word2Vec and other embeddding models learn analogy relationships in an unsupervised way. Things like the OpenAI unsupervised sentiment detection model[1] show there is a lot more that can be done in this area. [1] https://blog.openai.com/unsupervised-sentiment-neuron/ https://blog.openai.com/unsupervised-sentiment-neuron/
f00_ 9y ago
Generative Adversarial Networks (GAN) and Autoencoders are the ones that come to mind. There are also all the models used in research leading up to the "deep learning" craze: Helmholtz machine, Boltzman machine, Deep belief networks. http://pyro.ai/examples/vae.html http://pyro.ai/examples/vae.html http://www.scholarpedia.org/article/Boltzmann_machine http://www.scholarpedia.org/article/Boltzmann_machine http://www.scholarpedia.org/article/Deep_belief_networks http://www.scholarpedia.org/article/Deep_belief_networks Geoff Hinton's Coursera course goes pretty deep on the unsupervised models https://www.coursera.org/learn/neural-networks https://www.coursera.org/learn/neural-networks Reinforcement learning probably counts too (Deep Q Learning, Policy Gradient, Actor-Critic networks might be equivalent to GANs?) http://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html http://pytorch.org/tutorials/intermediate/reinforcement_q_le... https://github.com/vy007vikas/PyTorch-ActorCriticRL https://github.com/vy007vikas/PyTorch-ActorCriticRL

by nextos 9y ago

make3 9y ago

"Unsupervised Neural Machine Translation", https://arxiv.org/abs/1710.11041 https://arxiv.org/abs/1710.11041

nl 9y ago

There are lots of examples. In the NLP space, Word2Vec and other embeddding models learn analogy relationships in an unsupervised way. Things like the OpenAI unsupervised sentiment detection model[1] show there is a lot more that can be done in this area. [1] https://blog.openai.com/unsupervised-sentiment-neuron/ https://blog.openai.com/unsupervised-sentiment-neuron/

f00_ 9y ago

Generative Adversarial Networks (GAN) and Autoencoders are the ones that come to mind. There are also all the models used in research leading up to the "deep learning" craze: Helmholtz machine, Boltzman machine, Deep belief networks. http://pyro.ai/examples/vae.html http://pyro.ai/examples/vae.html http://www.scholarpedia.org/article/Boltzmann_machine http://www.scholarpedia.org/article/Boltzmann_machine http://www.scholarpedia.org/article/Deep_belief_networks http://www.scholarpedia.org/article/Deep_belief_networks Geoff Hinton's Coursera course goes pretty deep on the unsupervised models https://www.coursera.org/learn/neural-networks https://www.coursera.org/learn/neural-networks Reinforcement learning probably counts too (Deep Q Learning, Policy Gradient, Actor-Critic networks might be equivalent to GANs?) http://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html http://pytorch.org/tutorials/intermediate/reinforcement_q_le... https://github.com/vy007vikas/PyTorch-ActorCriticRL https://github.com/vy007vikas/PyTorch-ActorCriticRL