3 ms·

> we are not doing geometry here we absolutely are doing geometry here, given we're talking about metrics in a vector space – and this is trigonometry you lear

by adw 2y ago

> we are not doing geometry here we absolutely are doing geometry here, given we're talking about metrics in a vector space – and this is trigonometry you learned by the first year of high school.

jameshart 2y ago
You might like to think of vectors in their geometric interpretation but vectors are not inherently geometric - vectors are just lists of numbers, which we sometimes interpret geometrically because it helps us comprehend them. High dimensional vectors grow increasingly ungeometric as we have to wrestle with increasingly implausible numbers of orthogonal spatial dimensions in order to render them ‘geometric’. In the end, vectors (long lists of numbers a1, a2, a3, … an) start looking more like discrete functions f(i) = ai. And you can extend the same concept all the way to continuous functions - they’re like infinite dimensional vectors. For continuous functions over a finite interval the dot product (usually called the inner product in this domain) is just the integral of the product of two functions, and the ‘magnitude’ of a function is its RMS, and that means functions have a ‘cosine similarity’ which is not remotely geometric. There isn’t any geometric sense in which there is an ‘angle between’ cos(x) and sin(x) except it turns out that they have a cosine similarity of 0 so it implies the ‘angle between’ them is 90°, which actually makes a lot of sense. But in this same sense there’s an ‘angle between’ any two functions (over an interval). But we are not doing geometry here.
- drdeca 2y ago
  > increasingly implausible numbers of orthogonal spatial dimensions in order to render them ‘geometric’. Implausible how? “geometric” doesn’t mean “embeds nicely in 3D space”. What’s wrong with talking about the angle between two L^2 functions defined on an interval? Geometric reasoning still works? If you take a span of two functions, you have a plane. What’s the issue?
  - HPsquared 2y ago
    In this case can people just prepend "hyper-" as in hyperplane etc? Hyper-line, hyper-angle. (Speaking as someone who has heard 'hyperplane' a few times but not others)
    - klodolph 2y ago
      No, that would be incorrect. A plane is 2D. If you have two functions, and take their span, you get a 2D plane. It is a regular, flat, 2D plane. When people say “hyperplane” they are generally talking about something with more than two dimensions.
      - drdeca 2y ago
        At least when the ambient vector space is more than 3-dimensional, yeah. Specifically, a hyperplane generally refers to something with codimension 1. (So, when the ambient vector space is finite-dimensional, the dimension of a hyperplane is one less than the dimension of the ambient vector space.)
- bowsamic 2y ago
  > You might like to think of vectors in their geometric interpretation but vectors are not inherently geometric - vectors are just lists of numbers No. They can be expressed as lists of numbers in a basis if the vector space is equipped with a scalar product but the vector itself is an object that transcends the specific numbers it is expressed in. What you’re saying here is totally wrong and I recommend you check out the Wikipedia page on vector spaces. The geometrical object “a vector” is the more fundamental thing than the list of numbers
  - xigoi 2y ago
    Tuples of numbers are a special case of a vector space, which even comes with a canonical basis and inner product for free. And since the article is about word embeddings, which map words to tuples of numbers, there’s no need to mention other vector spaces in this context.
  - jameshart 2y ago
    You think this comment could have been written by someone who doesn’t understand what a vector space is? Vectors are not purely geometric objects. Geometry is a lens through which we can interpret vectors. So is linear algebra. The objects behave the same and both perspectives give us insights about them. Insisting vectors are only geometric is like saying complex numbers are geometric because they can be thought of as points on the complex plane.
- klodolph 2y ago
  > that means functions have a ‘cosine similarity’ which is not remotely geometric. It obeys the normal rules you learned in geometry. For example, pick three functions a,b,c. The functions form a triangle. The triangle obeys the triangle inequality—the distances satisfy d(a,b) ≤ d(a,c) + d(c,b). The angles of the triangle sum to 180°. This sounds an awful lot like geometry to me.
yzydserd 2y ago
Where I live, where many people live, we enter high school aged 11. We haven’t been introduced at school to geometry yet. I suspect you’re using American terminology. When talking about school years it’s often useful to talk about the year or grade of school, like “9th grade” or “year 9” as it’s more universal.
- adw 2y ago
  I’m not and, unfortunately, those aren’t universal either, even within a country. The normal terminology where I grew up would be S1, which follows P7. I would expect most people to know about trigonometric functions by age 12, yes. (I entered high school at 11 and the first topic tackled in maths classes was elementary trigonometry.)

by adw 2y ago

> we are not doing geometry here we absolutely are doing geometry here, given we're talking about metrics in a vector space – and this is trigonometry you learned by the first year of high school.

jameshart 2y ago

You might like to think of vectors in their geometric interpretation but vectors are not inherently geometric - vectors are just lists of numbers, which we sometimes interpret geometrically because it helps us comprehend them. High dimensional vectors grow increasingly ungeometric as we have to wrestle with increasingly implausible numbers of orthogonal spatial dimensions in order to render them ‘geometric’. In the end, vectors (long lists of numbers a1, a2, a3, … an) start looking more like discrete functions f(i) = ai. And you can extend the same concept all the way to continuous functions - they’re like infinite dimensional vectors. For continuous functions over a finite interval the dot product (usually called the inner product in this domain) is just the integral of the product of two functions, and the ‘magnitude’ of a function is its RMS, and that means functions have a ‘cosine similarity’ which is not remotely geometric. There isn’t any geometric sense in which there is an ‘angle between’ cos(x) and sin(x) except it turns out that they have a cosine similarity of 0 so it implies the ‘angle between’ them is 90°, which actually makes a lot of sense. But in this same sense there’s an ‘angle between’ any two functions (over an interval). But we are not doing geometry here.

drdeca 2y ago
> increasingly implausible numbers of orthogonal spatial dimensions in order to render them ‘geometric’. Implausible how? “geometric” doesn’t mean “embeds nicely in 3D space”. What’s wrong with talking about the angle between two L^2 functions defined on an interval? Geometric reasoning still works? If you take a span of two functions, you have a plane. What’s the issue?
- HPsquared 2y ago
  In this case can people just prepend "hyper-" as in hyperplane etc? Hyper-line, hyper-angle. (Speaking as someone who has heard 'hyperplane' a few times but not others)
  - klodolph 2y ago
    No, that would be incorrect. A plane is 2D. If you have two functions, and take their span, you get a 2D plane. It is a regular, flat, 2D plane. When people say “hyperplane” they are generally talking about something with more than two dimensions.
    - drdeca 2y ago
      At least when the ambient vector space is more than 3-dimensional, yeah. Specifically, a hyperplane generally refers to something with codimension 1. (So, when the ambient vector space is finite-dimensional, the dimension of a hyperplane is one less than the dimension of the ambient vector space.)
bowsamic 2y ago
> You might like to think of vectors in their geometric interpretation but vectors are not inherently geometric - vectors are just lists of numbers No. They can be expressed as lists of numbers in a basis if the vector space is equipped with a scalar product but the vector itself is an object that transcends the specific numbers it is expressed in. What you’re saying here is totally wrong and I recommend you check out the Wikipedia page on vector spaces. The geometrical object “a vector” is the more fundamental thing than the list of numbers
- xigoi 2y ago
  Tuples of numbers are a special case of a vector space, which even comes with a canonical basis and inner product for free. And since the article is about word embeddings, which map words to tuples of numbers, there’s no need to mention other vector spaces in this context.
- jameshart 2y ago
  You think this comment could have been written by someone who doesn’t understand what a vector space is? Vectors are not purely geometric objects. Geometry is a lens through which we can interpret vectors. So is linear algebra. The objects behave the same and both perspectives give us insights about them. Insisting vectors are only geometric is like saying complex numbers are geometric because they can be thought of as points on the complex plane.
klodolph 2y ago
> that means functions have a ‘cosine similarity’ which is not remotely geometric. It obeys the normal rules you learned in geometry. For example, pick three functions a,b,c. The functions form a triangle. The triangle obeys the triangle inequality—the distances satisfy d(a,b) ≤ d(a,c) + d(c,b). The angles of the triangle sum to 180°. This sounds an awful lot like geometry to me.

yzydserd 2y ago

Where I live, where many people live, we enter high school aged 11. We haven’t been introduced at school to geometry yet. I suspect you’re using American terminology. When talking about school years it’s often useful to talk about the year or grade of school, like “9th grade” or “year 9” as it’s more universal.

adw 2y ago
I’m not and, unfortunately, those aren’t universal either, even within a country. The normal terminology where I grew up would be S1, which follows P7. I would expect most people to know about trigonometric functions by age 12, yes. (I entered high school at 11 and the first topic tackled in maths classes was elementary trigonometry.)