5 ms·

I think the use of the term "cosine" here is needlessly confusing. It is the dot product of normalized vectors. Sure, when you do the maths, it gives out a cosi

• adw 2y ago
  > we are not doing geometry here we absolutely are doing geometry here, given we're talking about metrics in a vector space – and this is trigonometry you learned by the first year of high school.
  • jameshart 2y ago
    You might like to think of vectors in their geometric interpretation but vectors are not inherently geometric - vectors are just lists of numbers, which we sometimes interpret geometrically because it helps us comprehend them. High dimensional vectors grow increasingly ungeometric as we have to wrestle with increasingly implausible numbers of orthogonal spatial dimensions in order to render them ‘geometric’. In the end, vectors (long lists of numbers a1, a2, a3, … an) start looking more like discrete functions f(i) = ai. And you can extend the same concept all the way to continuous functions - they’re like infinite dimensional vectors. For continuous functions over a finite interval the dot product (usually called the inner product in this domain) is just the integral of the product of two functions, and the ‘magnitude’ of a function is its RMS, and that means functions have a ‘cosine similarity’ which is not remotely geometric. There isn’t any geometric sense in which there is an ‘angle between’ cos(x) and sin(x) except it turns out that they have a cosine similarity of 0 so it implies the ‘angle between’ them is 90°, which actually makes a lot of sense. But in this same sense there’s an ‘angle between’ any two functions (over an interval). But we are not doing geometry here.
    • drdeca 2y ago
      > increasingly implausible numbers of orthogonal spatial dimensions in order to render them ‘geometric’. Implausible how? “geometric” doesn’t mean “embeds nicely in 3D space”. What’s wrong with talking about the angle between two L^2 functions defined on an interval? Geometric reasoning still works? If you take a span of two functions, you have a plane. What’s the issue?
      • HPsquared 2y ago
        In this case can people just prepend "hyper-" as in hyperplane etc? Hyper-line, hyper-angle. (Speaking as someone who has heard 'hyperplane' a few times but not others)
        • klodolph 2y ago
          No, that would be incorrect. A plane is 2D. If you have two functions, and take their span, you get a 2D plane. It is a regular, flat, 2D plane. When people say “hyperplane” they are generally talking about something with more than two dimensions.
          • drdeca 2y ago
            At least when the ambient vector space is more than 3-dimensional, yeah. Specifically, a hyperplane generally refers to something with codimension 1. (So, when the ambient vector space is finite-dimensional, the dimension of a hyperplane is one less than the dimension of the ambient vector space.)
    • bowsamic 2y ago
      > You might like to think of vectors in their geometric interpretation but vectors are not inherently geometric - vectors are just lists of numbers No. They can be expressed as lists of numbers in a basis if the vector space is equipped with a scalar product but the vector itself is an object that transcends the specific numbers it is expressed in. What you’re saying here is totally wrong and I recommend you check out the Wikipedia page on vector spaces. The geometrical object “a vector” is the more fundamental thing than the list of numbers
  • yzydserd 2y ago
    Where I live, where many people live, we enter high school aged 11. We haven’t been introduced at school to geometry yet. I suspect you’re using American terminology. When talking about school years it’s often useful to talk about the year or grade of school, like “9th grade” or “year 9” as it’s more universal.
    • adw 2y ago
      I’m not and, unfortunately, those aren’t universal either, even within a country. The normal terminology where I grew up would be S1, which follows P7. I would expect most people to know about trigonometric functions by age 12, yes. (I entered high school at 11 and the first topic tackled in maths classes was elementary trigonometry.)
• renewiltord 2y ago
  I bet it's whether your primary background is programming or mathematics. From the latter, the cosine is very natural (scalar projection etc.) and it's lots of steps to get to your thing. I'd say this was intuitive for us post high-school because of that pedagogical background.
• steve_adams_86 2y ago
  Interesting, I think it’s actually far more intuitive to think of it geometrically. I’m not sure what my brain is doing in order for this mental projection to help, but this is exactly how I made dot products “click” for me. I started to think of them in multidimensional space, almost physically (though in a very limited sense since my brain came from a monkey and generally fires on a couple cylinders).
  • hinkley 2y ago
    I expect it’s like how learning to play by ear is more intuitive than sheet music. That’s great if you’re an amateur. If you’re dealing with tensors or somesuch trying to design a fusion reactor that’s probably a crutch.
    • gnulinux 2y ago
      This is a very odd statement and depicts the different ways human brain works. As a musician, I find playing (or thinking music in terms of) sheet music so much more intuitive than play by ear. It feels like the very reason people notate, write music is because anything written down is easier to think/play than anything listened.
      • hinkley 2y ago
        I can sing along to songs I never liked that haven’t been on the radio for twenty years. So I tend to sympathize with the by ear folks.
      • steve_adams_86 2y ago
        I have a feeling there's some overlap here. I can intuit a lot of things about music and even visualize some of it, but eventually I hit limitations. What I learn through these intuitions still applies as my ability to mentally visualize or model the music begins to fail, though. It's similar with vectors. Once you have the orchestral equivalent of vectors, there's no way I'm visualizing it and doing mental geometry. However, what I learned and the modelling I developed from the "casio keyboard playing jingles" equivalent of vectors is still useful and applicable. I guess this is the point where playing by ear or mentally modelling things fails, and notation is far more helpful. Yet if a lot of us approach these complex works from the notation angle first, we might feel pretty lost and uncertain about what we're doing with it and why. I can tell I'm not articulating this well, but I like the musical analogy and wanted to get that out.
• thaumasiotes 2y ago
  > Forget about angles and geometry unless you are comfortable with N-dimensional space with N>>3. Most people aren't. The whole point of measuring similarity this way is that any two vectors exist in a two-dimensional space, which is where you measure the angle between them. Why would you need to be comfortable with high-dimensional spaces?
  • RichardLake 2y ago
    By `two vectors exist in a two-dimensional space` are you talking about how two (linearly independent) n dimensional vectors will span a 2d space?
    • thaumasiotes 2y ago
      No, I'm talking about the fact that the space spanned by two vectors is sufficient to contain those vectors. All of the analysis you could ever theoretically want to do on them can be done within that space. If you only have two vectors, you never need to consider a space with higher dimensionality than 2. Each vector is a dimension of the space, and that's it.
      • Chinjut 2y ago
        That is the same thing as what is being said in the comment you are replying "No" to.
        • thaumasiotes 2y ago
          No, these are not at all the same claim: (A) Look at this space. Every point within it can be reached by combining these two vectors. (B) Look at this space. No point outside it can be reached by combining these two vectors. Saying that two vectors span a space is claim (A). Saying that the space they span contains them is... much weaker than claim (B), but it's related to claim (B) and not to claim (A).
  • adgjlsfhk1 2y ago
    for one reason, if you're just thinking about it as fancy 2d, you will miss a lot of phenomena that occur in higher dimensional spaces. for example, almost all vectors are almost completely orthogonal which isn't true at all in low dimensional spaces
• Izkata 2y ago
  > but since we are not doing geometry here, so it isn't really helpful for a beginner to know that. This article isn't talking geometry, it's trigonometry. And half the article is visual anyway.
• dilawar 2y ago
  Hmmm.. I heard in a conference that most well understood engineering principles or theories have a neat geometric interpretation. Personally I find a theory with geometric interpretation far easier to grasp. On the other hand, the higher dimensions geometry confuses me a lot: most random sparse vectors are orthogonal to each other, and most volumes of a sphere in that dimension are concentrated in a place.
• microtonal 2y ago
  It’s quite interesting that we end up using cosine similarity. Most networks are trained with a softmax layer at the end (e.g. next word prediction). Given the close relation between softmax and logistic regression, it might make more sense to use σ(u.v) as the similarity function.
• Jaxan 2y ago
  I agree on the first point. But I find the dot product much more geometrical than the cosine. So in my mind your argument is in favour of geometry!