2 ms·
Thanks! That's cool to know. I did my error calculations by brute forcing the whole 16-bit range, basically: for i in (i16::MIN..=i16::MAX) { let
by jamesmunns 4y ago
Thanks! That's cool to know.
I did my error calculations by brute forcing the whole 16-bit range, basically:
for i in (i16::MIN..=i16::MAX) {
let a = (i as f32).sin() as i16;
let b = lut_sin(i);
let diff = abs_diff(a, b);
}
Which found the largest difference to be 4, and I got 0.012% from (4/32768).