As always, John Langford has one foot firmly planted in practice, and knows deeply what practitioners will care about in the next five years. Incidentally, he i

As always, John Langford has one foot firmly planted in practice, and knows deeply what practitioners will care about in the next five years. Incidentally, he is the author of Vowpal Wabbit, one of the fastest out-of-core learning implementations. He also has a profound theoretical knowledge. This article is a more open-ended, forward-looking version of his blog post "The Ideal Large Scale Learning Class" (http://hunch.net/?p=1729 http://hunch.net/?p=1729). That blog post is required reading for anyone who wants to do large-scale learning, and understand the current state-of-the-art. "Almost all the big impact algorithms operate in pseudo-linear or better time." This is one of the reasons for the resurgence in neural networks. It's useful to learn a non-linear model over 1 billion examples, which is something you can't do with an SVM. "How do we efficiently learn in settings where exploration is required?" I've been exploring this problem recently. For example, I am designing an interface where a user can interact with search results to improve the quality. Besides ML issues, this also touches on UX questions: Is it better to force the user to give feedback on five crucial search results? Or should you show 50 results and have the user cherry pick results that are particularly good/bad? "How can we learn to index efficiently?" Another important question. One particularly interesting approach is using a dense hash code to do semantic search. For example, see the approach Semantic Hashing, which I describe in this forward-looking O'Reilly Strata talk about ML: http://strataconf.com/strata2011/public/schedule/detail/16934 http://strataconf.com/strata2011/public/schedule/detail/1693... (YouTube: http://www.youtube.com/watch?v=fEUw8igr1IY http://www.youtube.com/watch?v=fEUw8igr1IY) That talk overlaps a bit with Langford's post, except my Strata talk was about new developments for people in industry, not upcoming topics of interest for academics.