3 ms·
> One thing I would like to say is my qualm with the obsession with unsupervised learning as the quintessential technique. Unsupervised != no human input. I'm
by ageitgey 8y ago
> One thing I would like to say is my qualm with the obsession with unsupervised learning as the quintessential technique. Unsupervised != no human input.
I'm not sure I follow the qualm you are trying to get across. Are you saying you disagree with the term 'unsupervised' because unsupervised algorithms still bake in human assumptions (like a human-designed word embedding model) so that's essentially still supervision?
The obsession with 'unsupervised' learning as the quintessential technique is about getting better results for less money/effort. The premise is that we assume deep models tend to scale up in accuracy as training data size increases, so we always want larger datasets to increase accuracy. But creating labeled data takes a linear amount of human effort ($$$) as the dataset size increases. At a certain point, creating more labeled data to improve a model is not cost effective or maybe even impossible.
Unlabeled data can be acquired nearly for free in nearly unlimited quantities in many cases. So if we can use unlabeled data instead (even if it requires complex pre-processing like CBOW embedding models which essentially turn bits of the unlabeled data into it's own label) your final results-per-dollar-invested goes through the roof over supervised learning. That's the obsession. It's not about literally no supervision being involved in the process. It's about driving down the cost of data acquisition while driving up the percent of the world's available data you can use for training a model.
I apologize in advance if I'm missing the point you are making.
- hervature 8y agoNo need to apologize, I like HN precisely because people point out the flaws/confusing parts of my comments/opinions. I agree with everything you say and am happy you said it because this is how unsupervised learning should be viewed. I.e., a better ROI in specific cases. However, I have seen too often the "cake of AI" where the batter is unsupervised, the icing is supervised, and the cherry on top is reinforcement learning. Somehow, this image connotes that unsupervised is at the core of AI and also the most important. For what is a cake consisting of only icing and a cherry? Where I disagree with you is that the obsession is purely driven from "results-per-dollar-invested", at least in the academic world. That being said, unsupervised learning is a great tool and definitely worthy of research. To summarize, my comment was completely tangential of this paper (the authors make no such claims). It was more of a stream of consciousness comment that arose because I envisioned someone reading the paper and saying "see!, unsupervised learning leads to real understanding, no humans needed!"