3 ms·
That's the curse of dimensionality in effect. In very high dimensions, the probability that two randomly chosen vectors are orthogonal increases, and convention
by avinium 9y ago
That's the curse of dimensionality in effect. In very high dimensions, the probability that two randomly chosen vectors are orthogonal increases, and conventional distance metrics become less useful since most vectors end up very far from each other.
In geometric terms, think of a circle embedded in a square. As the number of dimensions increases (e.g. a sphere embedded in a cube, a 4-dimensional ball embedded in a 4-dimensional cube, etc), most of the volume in the cube is outside the radius of the sphere. Any vectors you have effectively become sparse.
Basically this means that, while you need large dimensionality to model complex relations, high dimensionality makes it difficult to model these relationships (and need exponentially more data).