3 ms·
> Entropy-weighted similarity: We adjust the similarity scores between query tokens and related documents based on the entropy of each token. Sounds a lot like
by leobg 2y ago
> Entropy-weighted similarity: We adjust the similarity scores between query tokens and related documents based on the entropy of each token.
Sounds a lot like BM25 weighted word embeddings (e.g. fastText).
- dmezzetti 2y ago> Sounds a lot like BM25 weighted word embeddings (e.g. fastText). If you're interested in this topic, I wrote an article on this method back in 2020: https://medium.com/towards-data-science/building-a-sentence-embedding-index-with-fasttext-and-bm25-f07e7148d240 https://medium.com/towards-data-science/building-a-sentence-...
- nblgbg 2y agoThanks for sharing the article. How exactly are you combining BM25 and fastText? Are you combining the TF-IDF score + embedding distance? What are the weights for each of these?