3 ms·
Looking at the code briefly, it doesn't seem like it's actually an accurate conversion. The core of the conversion algorithm appears to be based on doing a seri
by jcranmer 6y ago
Looking at the code briefly, it doesn't seem like it's actually an accurate conversion. The core of the conversion algorithm appears to be based on doing a series of floating-point multiplies, which of course will tend to trash the last few bits. Using volatile to try to avoid reassociation here. But there's no comments indicating why the rounding is going to be correct in the range provided, and my gut is that it's not going to be correct. (Especially since the double-to-int conversion is truncating, and I'd naïvely expect half the rounding error to round down, which means the last digit is off by one).