4 ms·
How do you represent 1/10 exactly in binary fixed-point representation?
by pwdisswordfishc 4y ago
How do you represent 1/10 exactly in binary fixed-point representation?
- tsukikage 4y ago...or, indeed, 1/3 as binary coded decimal.
- amelius 4y ago> How do you represent 1/10 exactly in binary fixed-point representation? The base of your number system does not need to be the same as the base associated with the fixed point position. Easiest to explain: if you have a uint64 that represents a monetary value, you can let it express the number of dollarcents instead of dollars. Then you can express 1/10 dollar as 10 dollarcents.
- jstanley 4y agoThe claim was "you never get this kind of result with fixed point math", not "you can contrive to avoid this kind of result with fixed point math if you use a different base for the fractional part".
- whitten 4y ago1/10 is exactly 0.1