4 ms·

When rewriting sin() for Xbox 360 I found that the old function gave the wrong result for denormals - off by a factor of two. Since double has an eleven bit exp

When rewriting sin() for Xbox 360 I found that the old function gave the wrong result for denormals - off by a factor of two. Since double has an eleven bit exponent that means that roughly one number out of 2,048 would hit this special failure case. For round/ceil/floor a common failure case is the number immediately preceding 0.5. If you're "lucky" then your algorithm fails for lots of numbers ending in 0.5, but if it's just the one number then a random search will never find it. And, an exhaustive search over all of the doubles from 0.0 to 1.0 turns out to cover a really big search space (one quarter of all doubles, roughly).

dummyfunny 9y ago
> "but if it's just the one number then a random search will never find it" Minor, but I think it's not never just very rare. If you're sampling from all floats then the problematic inputs should be possible. I think the distinction people aren't seeing is what are you testing. If it's the sin routine, then I think you're making valid points. But if we're talking about testing for correctness and the overall logic of the program or the abstract expression of the calculation, there are only a limited number of variables. Trying to divide them and vary them under this kind of control seems like a valuable tactic. It might not be sufficient but I've seen way more bugs in shitty if else boolean soups than I have from bits getting flipped. If the output is so sensitive to the value of a certain calculation there should probably be a secondary or checksum. Unless you're doing games where... Framerates.