3 ms·

The hardest (still missing) part of efficient cosine computation distance computation is picking a good epsilon for the `sqrt` calculation and avoiding "divisio

The hardest (still missing) part of efficient cosine computation distance computation is picking a good epsilon for the `sqrt` calculation and avoiding "division by zero" problems. We have an open issue about it in USearch and a related one in SimSIMD itself, so if you have any suggestions, please share your insights - they would impact millions of devices using the library (directly on servers and mobile, and through projects like ClickHouse and some of the Google repos): https://github.com/unum-cloud/usearch/issues/320 https://github.com/unum-cloud/usearch/issues/320

kridsdale3 3y ago
I just did a skim, and it looks like you're essentially seeing the limits of IEE 754 resolution at that scale (10 ^ -130). Theoretically they can store +/- 308 base-10 exponential range, but only about 17 decimal digits of precision. Floats were designed to be mostly-reliable within a mostly-reasonable range of numbers for the kinds of systems we deal with in our scale of the universe. The scale probed here is far far beyond Plank-scale if dealing with metric measurements. We have the converse issue (even with 64bit) at large scales. I could make a geometric vector in space from here to a rock on a planet 15 billion light years away, and another vector to an adjacent rock. I bet those would have a nondeterministic cosine issue too. I guess a solution is FP 128, or beyond.
- ashvardanian 3y ago
  Kahan floats are also commonly used in such cases, but I believe there is room for improvement without hitting those extremes. First of all, we should tune the epsilon here: https://github.com/ashvardanian/SimSIMD/blob/f8ff727dcddcd142e7e8dece33c43801af96c210/include/simsimd/spatial.h#L893 https://github.com/ashvardanian/SimSIMD/blob/f8ff727dcddcd14... As for the 64-bit version, its harder, as the higher-precision `rsqrt` approximations are only available with "AVX512ER". I'm not sure which CPUs support that, but its not available on Sapphire Rapids.
- Solvency 3y ago
  Why would floats for cosine similarity need to support ranges beyond Planck scale?
gukoff 3y ago
Can you adjust vector magnitude prior to the normalization? If the max coordinate C is close enough to 0, divide all coordinates by C.

kridsdale3 3y ago

I just did a skim, and it looks like you're essentially seeing the limits of IEE 754 resolution at that scale (10 ^ -130). Theoretically they can store +/- 308 base-10 exponential range, but only about 17 decimal digits of precision. Floats were designed to be mostly-reliable within a mostly-reasonable range of numbers for the kinds of systems we deal with in our scale of the universe. The scale probed here is far far beyond Plank-scale if dealing with metric measurements. We have the converse issue (even with 64bit) at large scales. I could make a geometric vector in space from here to a rock on a planet 15 billion light years away, and another vector to an adjacent rock. I bet those would have a nondeterministic cosine issue too. I guess a solution is FP 128, or beyond.

ashvardanian 3y ago
Kahan floats are also commonly used in such cases, but I believe there is room for improvement without hitting those extremes. First of all, we should tune the epsilon here: https://github.com/ashvardanian/SimSIMD/blob/f8ff727dcddcd142e7e8dece33c43801af96c210/include/simsimd/spatial.h#L893 https://github.com/ashvardanian/SimSIMD/blob/f8ff727dcddcd14... As for the 64-bit version, its harder, as the higher-precision `rsqrt` approximations are only available with "AVX512ER". I'm not sure which CPUs support that, but its not available on Sapphire Rapids.
Solvency 3y ago
Why would floats for cosine similarity need to support ranges beyond Planck scale?

gukoff 3y ago

Can you adjust vector magnitude prior to the normalization? If the max coordinate C is close enough to 0, divide all coordinates by C.