4 ms·
What are the differences in implementation of the languages that cause some to show a rounding error and others not?
by jdaley 11y ago
What are the differences in implementation of the languages that cause some to show a rounding error and others not?
- morsch 11y agoI suspect in many instances it's down to the method doing the printing being "helpful". The page documents this behaviour with Python. I've never specifically used C#, but it seems to default to the "G" format[1], the alternative "R" format shows the expected result. You can play around with it here: http://ideone.com/hxD96J http://ideone.com/hxD96J Printing out is kind of a misleading check here, maybe a better idea would be to test equality with the constant 0.3. Of course the site is just a neat illustration and not a technical whitepaper. [1] https://msdn.microsoft.com/en-us/library/dwhawy9k(v=vs.110).aspx https://msdn.microsoft.com/en-us/library/dwhawy9k(v=vs.110).... and https://msdn.microsoft.com/en-us/library/3hfd35ad(v=vs.110).aspx https://msdn.microsoft.com/en-us/library/3hfd35ad(v=vs.110)....
- yellowstuff 11y agoYes, I agree this page is misleading. It makes it look like C# is using rational numbers instead of floating point. In fact the "G" format defaults to 15 digits of precision for doubles, which is few enough to avoid ugly strings in many simple situations, but not all: > Console.WriteLine((.1 + .2)-.3); > Result: 5.55111512312578E-17
- langarto 11y agoThe printing routine. Internally, none of them can store 0.3 in a floating point variable because it is impossible to do so in exponential notation with base 2 (which is what IEE 754 uses). Some languages may optionally use an exact representation (e.g., Scheme or Python), but that's not the general case.