3 ms·
Someone else surprised that computer arithmetic is an inaccurate model of theoretical and mathematical arithmetic. Computer arithmetic is implemented to be con
by ColinWright 12y ago
Someone else surprised that computer arithmetic is an inaccurate model of theoretical and mathematical arithmetic.
Computer arithmetic is implemented to be convenient, not to be an accurate reflection of mathematical arithmetic. Thus it's the source of many, many "gotchas."
The usual reference to this sort of thing:
What Every Computer Scientist Should Know
About Floating-Point Arithmetic[0][1]
Not really limited to computer scientists - actually more relevant to programmers.
[0] https://news.ycombinator.com/item?id=4815399 https://news.ycombinator.com/item?id=4815399
[1] http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.ht...