4 ms·
I tried weighted average, but in my case the weights were computed by doing dot product between all vectors in the phrase (m x m), then taking the average over
by visarga 7y ago
I tried weighted average, but in my case the weights were computed by doing dot product between all vectors in the phrase (m x m), then taking the average over rows for each word and normalising. Kind of like a poor man's Transformer. It will boost words that are supported by other similar words in the same phrase.
- Der_Einzige 7y agoCan you provide psudocode or code for this? I'd post a snippet of my implementation but I'm too dumb to figure out the format. But wow this is much better for results! Now to figure out how to make it fast!
- visarga 7y agoSomething like this, including thresholding for pairs of words that have low dot product: import numpy as np def inner_product_rank(vecs, threshold=0.5): sims_mat = np.dot(vecs, vecs.transpose()) sims_mat = sims_mat - threshold sims_mat[sims_mat<0] = 0 ranks = np.sum(sims_mat, axis=0) return ranks Then you can take a weighted sum of the vecs or use the ranks to select the most related words. It is also possible to run spectral clustering on sims_mat to get the main topics of the text, it works quite well.