2 ms·
The vectors are not normalized. That is: from __future__ import print_function from collections import Counter def cosine_similarity_unweighted(a,
by dalke 11y ago
The vectors are not normalized. That is:
from __future__ import print_function
from collections import Counter
def cosine_similarity_unweighted(a, b):
a = Counter(a)
b = Counter(b)
dotProduct = sum((c in a) * (c in b) for c in a)
magnitudeA = sum(1 for c in a)
magnitudeB = sum(1 for c in b)
return dotProduct / (magnitudeA * magnitudeB)**0.5
def cosine_similarity_weighted(a, b):
a = Counter(a)
b = Counter(b)
dotProduct = sum(a.get(c, 0) * b.get(c, 0) for c in a)
magnitudeA = sum(value**2 for value in a.values())
magnitudeB = sum(value**2 for value in b.values())
return dotProduct / (magnitudeA * magnitudeB)**0.5
print("Scores for 'hello' and 'holl'")
print("unweighted", cosine_similarity_unweighted("hello", "holl"))
print("weighted", cosine_similarity_weighted("hello", "holl"))
gives the output:
Scores for 'hello' and 'holl'
unweighted 0.866025403784
weighted 0.925820099773
The article gives:
holl index: 0.9258200997725514
which means the article used letter weights.
That said, the scores for your example are the same, because each of the letters in 'house' is used only once (the 3 in the numerator cancels out the 3 in the denominator):
Scores for 'house' and 'hiusehiusehiuse'
unweighted 0.8
weighted 0.8
If the query had instead been 'houss', you would see a difference in the similarities:
Scores for 'houss' and 'hiusehiusehiuse'
unweighted 0.67082039325
weighted 0.676123403783