3 ms·
That's a really good question and why the initial promise of deep learning was best shown in vision. It is only recently that deep learning in text has given en
by Smerity 10y ago
That's a really good question and why the initial promise of deep learning was best shown in vision. It is only recently that deep learning in text has given enough performance gains to justify the (large) computational overhead.
For many languages, and many tasks performed on these languages, features have already been discretely provided. If you take English, split on the space character, and then throw those into a standard classifier (Naive Bayes, SVM, logistic regression), you're likely to get a reasonable result. If you add n-grams, so a little bit of collocation information, you tend to do even better. For most tasks, you may even make the simplifying assumption that features don't even interact at all. Even naive use of these methods will get you 90% of the way in many NLP tasks with very little computational overhead (as is evidence in this article).
For other languages, generating those features is a more complicated matter. Chinese is written without spaces between words for example. Extracting such discrete features from there requires an extra step in the pipeline. The way in which you split the characters can also depend on context, requiring an interaction between these components. This interaction is very iffy when it's in a pipeline and you don't have gradients flowing end to end, as is the case with most deep learning systems.
Images are far more like the latter case. There are few discrete features that can be easily extracted from images. Images are also scarily high dimensional - the number of pixels, the number of colours, the interaction between pixels, etc. Using human generated methods of feature extraction on this is fraught with a lot of complicated hand tuning and a lot of misses.
tldr; Text, by virtue of being human generated, is far more structured and far more amenable to easily extracting discrete representative features. For images, we just never really found a highly effective way of producing discrete representative features by hand.
Deep learning, by virtue of providing the feature extraction in an automated manner directed by the loss on the task, has helped solve a major issue that plagued computer vision.