4 ms·
I disagree that your version is more readable. If you don’t know linear algebra, the code is inscrutable, and if you do, dot(x,y)/norm(x)/norm(y) is about as cl
by toxik 3y ago
I disagree that your version is more readable. If you don’t know linear algebra, the code is inscrutable, and if you do, dot(x,y)/norm(x)/norm(y) is about as close to the math as you can get.
- drcongo 3y agoMy linear algebra is terrible, but I understand what's happening in Simon's example pretty well.
- toxik 3y agoI sincerely doubt you possess any kind of geometric understanding of what is going on in that code just from reading the first principles variant of cosine similarity.
- forgotpwd16 3y ago>If you don’t know linear algebra, the code is inscrutable Whether you do or don't know linear algebra, code is self-explanatory. Dot product is just the sum of element-wise products. But what the heck are dot and norm to someone that doesn't.
- toxik 3y agoThat’s exactly my point, it ISN'T self explanatory when it’s mathematical code like this, expressed in terms of addition and multiplication. Understanding the dot product as “sum of multiplications” is an extremely shallow understanding, and in no way does it explain why taking the dot product and dividing by magnitudes (What are those? Root of sum of squares? Why?) produces anything useful. This code says nothing about the geometric nature of what is going on, only the arithmetic. Then we might as well read assembler. Abstractions help us reason.
- simonw 3y agoThe whole point of my talk here was to reassure people that you don't need to understand linear algebra in order to make use of embeddings.
- alexvitkov 3y agoYou either know the math or you don't, and if you don't, you're not gonna get any new magic intuitions no matter how it's shuffled around.
- forgotpwd16 3y ago>What are those? Root of sum of squares? Why? I see what you mean. Probably can be argued that learning and applying a specific concept doesn't necessarily imply learning the deeper mathematics behind it and understanding its nature. So can just say cosine similarity measures the similarity of two vectors (lists in python) and its implementation is that sum divided by the product of sqrts of those two other sums without having to introduce and explain dot product and norm. Of course the reverse can be argued too.
- Q6T46nT668w6i3m 3y agoI’d think you’d want to know dot and norm before embeddings. Especially since most people learn mathematical objects from the perspective of the operators that construct and manipulate them.