2 ms·
> due to using floats No they weren't using floats. The article you've linked to says this really clearly. > This calculation was performed using a 24 bit fix
by chrisseaton 7y ago
> due to using floats
No they weren't using floats. The article you've linked to says this really clearly.
> This calculation was performed using a 24 bit fixed point register
- bobdobbs666 7y agohttp://www.cs.unc.edu/~smp/COMP205/LECTURES/ERROR/lec23/node4.html http://www.cs.unc.edu/~smp/COMP205/LECTURES/ERROR/lec23/node... They meant fixed precision not fixed point math.