4 ms·

I ran into a weird problem this week trying to convert a string "8.95" to an integer (8.95) that made me question if even Int is good enough and as suggested in

• sixstringtheory 7y ago
  The problem here isn’t integers at all, as 8.95 is not an integer but an integer (8) plus a fraction (0.95 or 95/100==19/20). That’s in decimal, but since computers use binary, your decimal fraction must be converted to a binary fraction. This is what floating point is used for, although it’s important to know that floating point is an approximation of the real number line, and also that fractions that are rational in decimal may be irrational in binary, namely if the decimal denominator is not a power of two. Looks like that’s what’s going on with your example. Floating point approximation of an otherwise unrepresentable irrational binary which resulted in that repeating sequence when converted back to decimal again. I don’t know how BigDecimal works but I guess just by the name it is not floating point but rationals represented by arbitrary precision integer numerators and denominators. Definitely read up on how floating point representation works, with binary mantissas and exponents, normalization, and converting decimal fractions to binary fractions.
  • dragonwriter 7y ago
    > That’s in decimal, but since computers use binary Well, if you tell them to, e.g., by using a binary floating point type instead of a decimal type.
    • sixstringtheory 7y ago
      No, we aren’t using computers with logic gates that have 10 discrete states. Even if you’re programming to decimal, the underlying representation is still binary. But there are numerical differences between using decimals represented by rationals with binary integer numerators and denominators, vs floating point with binary mantissa and exponent.
      • dragonwriter 7y ago
        > No, we aren’t using computers with logic gates that have 10 discrete states. Even if you’re programming to decimal, the underlying representation is still binary. That is not the sense of "computers use binary" that justifies the conclusions in the sentence in which the phrase was used in the post being responded to, so while true on its own, in context it is an example of the fallacy of equivocation.
        • sixstringtheory 7y ago
          Could you help me understand where I’ve gone wrong here? I think there’s some miscommunication going on.
          • dllthomas 7y ago
            Are you familiar with binary coded decimal? Decimal format floats are in somewhat the same vein. See, for instance, https://en.m.wikipedia.org/wiki/Decimal32_floating-point_format https://en.m.wikipedia.org/wiki/Decimal32_floating-point_for...
            • sixstringtheory 7y ago
              This is basically what I meant when I said "computers use binary"
              • dllthomas 7y ago
                That sense is trivially true, but it doesn't follow that you have binary fractions.
                • sixstringtheory 7y ago
                  Sorry, but I'm not parsing this sentence: > it doesn't follow that you have binary fractions Is that in reference to something I said earlier, or are we just falling deeper down a rabbit hole of hair splitting? The point I was driving at is that a given number we think about in decimal like 12.3456 can be represented in binary in more than one way, one of which are rationals backed by integers for num/den, another is floating point with mantissa and exponent. Which way is chosen ultimately affects the outcomes of computations. I was trying to explain why OP was seeing unexpected results. I don't think I've been mistaken in the explanations...
• seppel 7y ago
  > I know this isn't StackOverflow, but could someone shed some light on to what's happening here? https://ideone.com/1U7rgf https://ideone.com/1U7rgf You cannot store 8.95 in a binary float (independent of precision). It is basically the same problem that you cannot store 1/3 as a decimal number.
• arh68 7y ago
  This is pretty standard for IEEE-754 floats/doubles. Try entering 8.95 into Float Toy [0]. It's about 1 * 2^3 * 1.11875. If you look at the red part, you see there's a "repeating decimal" 00011(1100)(1100).. that gets truncated, just like I can "recognize" 0.33 as "one-third" but if you want to multiply it by 10 I'll (probably) give you 3.30, and it's no longer special. There is always a lossy "compression" when you move across co-prime bases (pretty sure) and truncate the repeating decimals. What's really hard is printing a floating-point number. Since you have to print "8.95" when all you really have is a base-2 rational very close to it, you want to be sure you don't actually need to print 8.94999998. If you miss the last couple bits in the red part in Float Toy, you'll see the print algorithm shifts away from the 8.95 answer. [0] https://evanw.github.io/float-toy/ https://evanw.github.io/float-toy/
• tzs 7y ago
  After converting the string to float and then multiplying by 100, round to the nearest integer. Assuming that the underlying implementation of Ruby uses sscanf for the conversion (I'm assuming Ruby is in C or C++?), that should work. I had some concerns about this general issue, and did some brute force testing. The following program constructs strings "0.1", "0.2", ..., "0.9", "0.01", "0.02", ..., "0.99", ..., "0.99999999", uses sscanf to turn them into doubles, multiplies by the appropriate power of 10, rounds to the nearest integer, and checks if that is right. #include <stdio.h> #include <math.h> int main(void) { int len = 1; unsigned long div = 10; char num[30]; while (len < 9) { for (unsigned long i = 1; i < div; ++i) { sprintf(num, "%.20lu", i); num[19-len] = '.'; char * dp = num + 18 - len; double d; sscanf(dp, "%lf", &d); unsigned long ci = round(d * div); if (i != ci) printf("%s, %lf, %lu\n", dp, d, ci); } ++len; div *= 10; fprintf(stderr, "LEN=%d, DIV=%ld\n", len, div); } return 0; }