3 ms·

I just did a skim, and it looks like you're essentially seeing the limits of IEE 754 resolution at that scale (10 ^ -130). Theoretically they can store +/- 308

I just did a skim, and it looks like you're essentially seeing the limits of IEE 754 resolution at that scale (10 ^ -130). Theoretically they can store +/- 308 base-10 exponential range, but only about 17 decimal digits of precision. Floats were designed to be mostly-reliable within a mostly-reasonable range of numbers for the kinds of systems we deal with in our scale of the universe. The scale probed here is far far beyond Plank-scale if dealing with metric measurements. We have the converse issue (even with 64bit) at large scales. I could make a geometric vector in space from here to a rock on a planet 15 billion light years away, and another vector to an adjacent rock. I bet those would have a nondeterministic cosine issue too. I guess a solution is FP 128, or beyond.

ashvardanian 3y ago
Kahan floats are also commonly used in such cases, but I believe there is room for improvement without hitting those extremes. First of all, we should tune the epsilon here: https://github.com/ashvardanian/SimSIMD/blob/f8ff727dcddcd142e7e8dece33c43801af96c210/include/simsimd/spatial.h#L893 https://github.com/ashvardanian/SimSIMD/blob/f8ff727dcddcd14... As for the 64-bit version, its harder, as the higher-precision `rsqrt` approximations are only available with "AVX512ER". I'm not sure which CPUs support that, but its not available on Sapphire Rapids.
Solvency 3y ago
Why would floats for cosine similarity need to support ranges beyond Planck scale?