5 ms·

I got hit by JavaScript rounding on a project once. Funny thing is, IE8 was okay with it and Chrome was the one that caused issues. Turns out that 1000000 * 8

I got hit by JavaScript rounding on a project once. Funny thing is, IE8 was okay with it and Chrome was the one that caused issues. Turns out that 1000000 * 8.2 = 8199999.999999999 Sometimes we learn the hard way. The bug was getting written into an XML document and pushed to an embedded device over HTTP which was expecting an int and caused the device to crash. We fixed both bugs.

deleted 14y ago
[deleted]
ricardobeat 14y ago
// Scheme (print (* 1000000 8.2)) > 8199999.999999999 // Python >>> 1000000 * 8.2 8199999.999999999 //PHP > 1000000 * 8.2 > 8199999.999999999
- ehsanu1 14y ago
  # Ruby 1.8.7 1.8.7 :022 > 1000000 * 8.2 => 8200000.0 # Ruby 1.9.2 1.9.2p320 :001 > 1000000 * 8.2 => 8199999.999999999
- CamperBob2 14y ago
  C/C++: printf("%lf\n", 1000000 * 8.2); 8200000.000000 Here, printf() is doing the necessary rounding to hide the fact that 8.2 isn't exactly representable. It's interesting that other languages don't do something similar when outputting a float to text.
  - pindi 14y ago
    They do, you just have to request it: //Python >>> repr(1000000 * 8.2) '8199999.999999999' >>> "%lf" % (1000000 * 8.2) '8200000.000000'
- jabiko 14y ago
  I can't reproduce the PHP example: $ uname -p x86_64 $ php -v PHP 5.4.6-1ubuntu1 (cli) (built: Aug 22 2012 21:13:52) Copyright (c) 1997-2012 The PHP Group Zend Engine v2.4.0, Copyright (c) 1998-2012 Zend Technologies $ php -r 'var_dump(1000000 * 8.2);' float(8200000) $
  - ricardobeat 14y ago
    My fault, I think I copied from the wrong REPL.