(1) Adversarial learning is unsupervised and works great. Most of language modeling is unsupervised (you predict next word, but it's not real supervision becaus

(1) Adversarial learning is unsupervised and works great. Most of language modeling is unsupervised (you predict next word, but it's not real supervision because it's self-supervision). There're many works in computer vision which are unsupervised and still give more or less reasonable performance. See f.e. http://arxiv.org/pdf/1511.05045v2.pdf http://arxiv.org/pdf/1511.05045v2.pdf for unsupervised learning in action recognition, also http://arxiv.org/pdf/1511.06434v2.pdf http://arxiv.org/pdf/1511.06434v2.pdf and http://www.arxiv-sanity.com/search?q=unsupervised http://www.arxiv-sanity.com/search?q=unsupervised (2) ImageNet supervision gives you much information to solve other computer vision tasks. So perhaps we don't need to learn everything in unsupervised manner, we might learn most features relevant for most tasks using several supervision tasks. It is kind of cheating but very reasonable one. Moreover, (3) We observe now just fantastic decrease of perplexity (btw, it's all unsupervised = self-supervised). It's quite probable that in the very near future neural chat bots write reasonable stories, answer intelligibly with common sense, discuss things. All of this would be just a mere consequence of low enough perplexity. If neural net says smth inconsistent it means that it gives too much probability to some inappropriate words i.e, it's perplexity isn't optimized yet. (4) It's quite probable that it would open a finish line for human-level AI. AI would be able to learn from textbooks, scientific articles, video lectures. Btw, http://arxiv.org/pdf/1602.03218.pdf http://arxiv.org/pdf/1602.03218.pdf gives a potential to synthesize IBM Watson with deep learning. May be, the finish line to human level AI has been opened already.