3 ms·
Totally agree. Similarly for word embeddings like word2vec, GLoVE, fasttext etc in the case of NLP. I think this is fundamental - if you teach a human how to
by alexcnwy 9y ago
Totally agree.
Similarly for word embeddings like word2vec, GLoVE, fasttext etc in the case of NLP.
I think this is fundamental - if you teach a human how to recognize street signs, you don't need to show them millions of examples - just one or a few of each is enough because we build on reference experiences of past objects seen through life experience to encode the new images as memories.
- 131012 9y agoWould you care to elaborate? My main doubt comes the fact that language meaning may vary between different contexts, but I am no expert and am earnestly curious about using NLP and ML with not-that-big data.
- JacobiX 9y agoVector representation are useful for many natural language processing tasks. In word embedding like word2vec, GLoVE, fasttext for each word the algorithm learns an associated vector in dimension n (x1, x2, .., xn). A word maybe close to another in one dimension (or a subspace) but far away in another one. Moreover good representation allows meaningful vector space arithmetic: Queen - Women == King word representations are typically trained on very large unlabeled data, but once the algorithm learns the features you can use them for your small dataset. EDIT: Add more explanations.
- rjtavares 9y agoThis blog post[1] is a great example of positive unintended consequences of deep learning: > We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment. [1]: https://blog.openai.com/unsupervised-sentiment-neuron/ https://blog.openai.com/unsupervised-sentiment-neuron/