3 ms·
I have a similar classification project and I used word2vec to build embeddings from 1GB+ of text, then I just do vector similarity between the article and the
by visarga 9y ago
I have a similar classification project and I used word2vec to build embeddings from 1GB+ of text, then I just do vector similarity between the article and the topics.
The vector of an article can be obtained by summing the vectors of its words (minus stop words). For a topic you just sum up 5-10 of the topic keywords. You don't need to exhaustively list all the topic keywords because word2vec automatically maps them in close vicinity.
This system has the advantage that you don't need a training dataset. It's unsupervised learning coupled with a small amount of supervised topic pointers.