4 ms·

> dot_product = sum(x * y for x, y in zip(a, b)) Wait, why would you do this and not use vectorised numpy operations? > I actually got ChatGPT to write all of

> dot_product = sum(x * y for x, y in zip(a, b)) Wait, why would you do this and not use vectorised numpy operations? > I actually got ChatGPT to write all of my different versions of cosine similarity Ah...

simonw 3y ago
Two reasons. First, when I'm trying to explain stuff to people I find numpy syntax gets in the way. And second, numpy isn't the lightest dependency. I use it when I need the performance but I don't like to default to it.
- lordgrenville 3y ago
  Makes sense. And sorry for the snark - I enjoyed the post, and generally enjoy your writing as well.
- toxik 3y ago
  I disagree that your version is more readable. If you don’t know linear algebra, the code is inscrutable, and if you do, dot(x,y)/norm(x)/norm(y) is about as close to the math as you can get.
  - drcongo 3y ago
    My linear algebra is terrible, but I understand what's happening in Simon's example pretty well.
    - toxik 3y ago
      I sincerely doubt you possess any kind of geometric understanding of what is going on in that code just from reading the first principles variant of cosine similarity.
  - forgotpwd16 3y ago
    >If you don’t know linear algebra, the code is inscrutable Whether you do or don't know linear algebra, code is self-explanatory. Dot product is just the sum of element-wise products. But what the heck are dot and norm to someone that doesn't.
    - toxik 3y ago
      That’s exactly my point, it ISN'T self explanatory when it’s mathematical code like this, expressed in terms of addition and multiplication. Understanding the dot product as “sum of multiplications” is an extremely shallow understanding, and in no way does it explain why taking the dot product and dividing by magnitudes (What are those? Root of sum of squares? Why?) produces anything useful. This code says nothing about the geometric nature of what is going on, only the arithmetic. Then we might as well read assembler. Abstractions help us reason.
      - simonw 3y ago
        The whole point of my talk here was to reassure people that you don't need to understand linear algebra in order to make use of embeddings.
      - alexvitkov 3y ago
        You either know the math or you don't, and if you don't, you're not gonna get any new magic intuitions no matter how it's shuffled around.
      - forgotpwd16 3y ago
        >What are those? Root of sum of squares? Why? I see what you mean. Probably can be argued that learning and applying a specific concept doesn't necessarily imply learning the deeper mathematics behind it and understanding its nature. So can just say cosine similarity measures the similarity of two vectors (lists in python) and its implementation is that sum divided by the product of sqrts of those two other sums without having to introduce and explain dot product and norm. Of course the reverse can be argued too.
    - Q6T46nT668w6i3m 3y ago

simonw 3y ago

Two reasons. First, when I'm trying to explain stuff to people I find numpy syntax gets in the way. And second, numpy isn't the lightest dependency. I use it when I need the performance but I don't like to default to it.

lordgrenville 3y ago
Makes sense. And sorry for the snark - I enjoyed the post, and generally enjoy your writing as well.
toxik 3y ago
I disagree that your version is more readable. If you don’t know linear algebra, the code is inscrutable, and if you do, dot(x,y)/norm(x)/norm(y) is about as close to the math as you can get.
- drcongo 3y ago
  My linear algebra is terrible, but I understand what's happening in Simon's example pretty well.
  - toxik 3y ago
    I sincerely doubt you possess any kind of geometric understanding of what is going on in that code just from reading the first principles variant of cosine similarity.
- forgotpwd16 3y ago
  >If you don’t know linear algebra, the code is inscrutable Whether you do or don't know linear algebra, code is self-explanatory. Dot product is just the sum of element-wise products. But what the heck are dot and norm to someone that doesn't.
  - toxik 3y ago
    That’s exactly my point, it ISN'T self explanatory when it’s mathematical code like this, expressed in terms of addition and multiplication. Understanding the dot product as “sum of multiplications” is an extremely shallow understanding, and in no way does it explain why taking the dot product and dividing by magnitudes (What are those? Root of sum of squares? Why?) produces anything useful. This code says nothing about the geometric nature of what is going on, only the arithmetic. Then we might as well read assembler. Abstractions help us reason.
    - simonw 3y ago
      The whole point of my talk here was to reassure people that you don't need to understand linear algebra in order to make use of embeddings.
    - alexvitkov 3y ago
      You either know the math or you don't, and if you don't, you're not gonna get any new magic intuitions no matter how it's shuffled around.
    - forgotpwd16 3y ago
      >What are those? Root of sum of squares? Why? I see what you mean. Probably can be argued that learning and applying a specific concept doesn't necessarily imply learning the deeper mathematics behind it and understanding its nature. So can just say cosine similarity measures the similarity of two vectors (lists in python) and its implementation is that sum divided by the product of sqrts of those two other sums without having to introduce and explain dot product and norm. Of course the reverse can be argued too.
  - Q6T46nT668w6i3m 3y ago