4 ms·

I might suggest reading https://doc.lagout.org/science/0_Computer%20Science/3_Theory/Handbook%20of%20Floating%20Point%20Arithmetic.pdf https://doc.lagout.org/sc

by ouro 8y ago

I might suggest reading https://doc.lagout.org/science/0_Computer%20Science/3_Theory/Handbook%20of%20Floating%20Point%20Arithmetic.pdf https://doc.lagout.org/science/0_Computer%20Science/3_Theory... This is in general how floating point math works.