3 ms·
My CS course at a community college went over the limitations of floats in detail, in the CS class.
by jimmaswell 8y ago
My CS course at a community college went over the limitations of floats in detail, in the CS class.
- copperx 8y agoHow detailed, if I may ask?
- jimmaswell 8y agoI remember we learned how the representation worked bit-for-bit, and how it being stored in binary meant it couldn't perfectly represent everything in decimal. 1.01b meaning 12^0 + 02^-1 + 1*2^-2 for example.