8 ms·

Ahah, Apple does have different math

48 points by profquail 17y ago

FooBarWidget 17y ago
It has been long known that one cannot rely on floating point for precise calculations, e.g. monetary values. One should use integers instead or some kind of other precise representation. But it does surprise me that even on the same CPU architecture you cannot expect every machine to produce the same results.
- profquail 17y ago
  I concentrated in optimization and numerical analysis for my undergrad degree in Math, and I'm still surprised on a regular basis how many programmers out there don't know about the inaccuracies and errors that come out of floating-point representations.
  - pmorici 17y ago
    Are you really that surprised. Crappy programmers abound and many introductory programming classes have students write programs to calculate monetary things using floating point so it's no small wonder many people think it's ok.
  - jwr 17y ago
    My thoughts exactly. Tom Christiansen brings these things up as if they were a huge surprise, but they aren't, or at least shouldn't be. To put it another way, it isn't Apple doing something wrong. If your code depends on rounding errors in FP representations, your code is buggy. That's also why good numerical algorithms are hard to design. You need to worry about losing precision. As a simple example, if you calculate (a-b)c and a is almost equal to b, you're much better off doing ac-b*c. These tricks are important.
  - brettnak 17y ago
    As did, I. I am a professional programmer now and didn't go through the CS school ( To all sophomores out there: We do exist. ) Do they really not teach this sort of thing in a formal CS school?
    - wtallis 17y ago
      Most CS undergraduate programs in the US suck enough these days that numerical analysis isn't taught at the undergrad level. For example, at my school, undergraduates get to choose between the automata/grammars/computability class and a "numerical methods" class that is really watered down. There is a two-semester undergraduate numerical analysis class, but as I recall it can't count toward your degree except as a general elective. To get numerical analysis to count as a CS elective, you have to take the graduate level class.
- barrkel 17y ago
  For the same CPU architecture, compiler and FPU control word, you should expect the same results. For example, on x86, many C / C++ compilers interpret long double as a 64-bit float while others interpret it as 80-bit extended precision. And extended precision TBYTE (in Intel syntax) does actually take up 10 bytes. There's a speed loss for not aligning, but no correctness loss. In a possibly related note, however, Apple has bizarre and unnecessarily strict alignment policies for Mac OS on x86: http://blogs.embarcadero.com/eboling/2009/05/20/5607 http://blogs.embarcadero.com/eboling/2009/05/20/5607
  - tentonova2 17y ago
    For the same CPU architecture, compiler and FPU control word, you should expect the same results. That's a stretch. Just because it's called "GCC" doesn't mean it behaves exactly the same on every platform, even if the CPU architecture is the same. Apple makes extensive modifications, has their own ABI (see below), etc. In a possibly related note, however, Apple has bizarre and unnecessarily strict alignment policies for Mac OS on x86: It's not "bizarre and unnecessarily strict" if you want to be able to rely on SSE2+. Apple had the advantage of being able to define their ABI without regard to most legacy concerns, and so they did. The reason that it's strictly enforced everywhere is that since Apple's compilers use SSE2+ they must be able to assume that, at function entry, the stack is properly (16 byte) aligned. I understand your pain -- I've had to update a JIT implementation to deal with this, along with quite a bit of assembly that assumed 4 byte alignment, but Apple's reasoning makes sense. See also: http://stackoverflow.com/questions/612443/why-does-the-mac-abi-require-16-byte-stack-alignment-for-x86-32/898760#898760 http://stackoverflow.com/questions/612443/why-does-the-mac-a...
    - barrkel 17y ago
      The stack is not actually aligned on function entry, because the return address is on top, so more alignment will be needed to avoid SSE2 locals being misaligned. It's not so hard for the callee side of the ABI to make sure the stack is aligned if it's going to use SSE2 and friends; it's rather more onerous to require every call site to make the alignments for the benefit of the callee.
profquail 17y ago
Tell PG: When I submitted this article, HN stripped the "!" characters out of the title...but I was then able to edit the story and put them back in there. I suppose that means there's a small bug in there somewhere...
- notauser 17y ago
  You can also use Unicode in comments, which may or ʎɐɯ ʇou be a bug but did surprise me.
  - sailormoon 17y ago
    Why on earth would the ability to use unicode in comments be a bug? Seems like a basic feature of any modern site.
- Perceval 17y ago
  It's probably a feature not a bug, designed to automatically de-sensationalize sensationalist titles!!!
  - profquail 17y ago
    Good point (I don't like sensational titles either) -- but shouldn't HN disallow you to edit the title and put them back in then?
    - echaozh 17y ago
      Few people would come back and add the sensation back. If they do, they must have thought about it and been more rational.
- lg 17y ago
  i think it's because editing doesn't apply the mysterious scrubrules, might be intentional.
TallGuyShort 17y ago
Wasn't a very similar phenomenon a huge blow to Intel's PR some years back?
- profquail 17y ago
  Sort of, but that had to do with a bug in the actual hardware of the Pentium Pro chip: http://en.wikipedia.org/wiki/Pentium_FDIV_bug http://en.wikipedia.org/wiki/Pentium_FDIV_bug
- wglb 17y ago
  No, that was not the small errors that are shown here, but rather errors of very significant magnitude.
ajross 17y ago
This isn't a new problem. The 80-bit internal 8087 FPU precision has always been a mismatch for the 64 bit IEEE double representation, even before the presence of SSE registers (which don't have the 80 bit mode) complicated things. Intel 8087 code has always been able to produce different results for the same source code, depending on when/whether/which intermediate results get spilled to memory. The Motorola 68k FPU had the same issue with higher internal than external precision, IIRC. This isn't a bug. Both code paths produce results of the highest representable precision of the hardware in question. It's just that there are multiple hardware units capable of giving you the answer, and Apple's toolchain picks a different one than whatever Tom is using elsewhere. And as has been pointed out in FooBarWidget's comment -- any code that relies on bit-precise results from floating point computation is almost certainly concealing precision bugs anyway. That's not the right way to approach floating point architecture.
- ars 17y ago
  For people unfamiliar with this, "spilled to memory" means rounded to 64 bits (from 80). x87 does floating point math with 80 bits (in registers). But the variables are stored in memory locations with 64 bits. So the results are rounded. The problem comes from when to do the rounding. And that can vary depending on if a register is needed for something else, optimizations, the order of code, etc.
tentonova2 17y ago
This shouldn't surprise anyone and I can't say I understand the breathless post. For more information, I recommend Write Great Code: Understanding the Machine -- Chapter 4, Floating-Point Representation.
modeless 17y ago
What Every Computer Scientist Should Know About Floating-Point Arithmetic: http://docs.sun.com/source/806-3568/ncg_goldberg.html http://docs.sun.com/source/806-3568/ncg_goldberg.html
jcl 17y ago
This reminds me of a page on fast and accurate geometric primitives; the code requires that you to flip the Intel FPU into a reduced precision mode to make the error easier to estimate. http://www.cs.cmu.edu/~quake/robust.pc.html http://www.cs.cmu.edu/~quake/robust.pc.html
9oliYQjP 17y ago
Apple doesn't use the x87 floating point unit. They've never needed to because they can rely on the SSE2 unit since any Intel Mac ever created has one. This isn't necessarily the case on the PC even though every modern one will have it as well. I'm pretty sure gcc is set to compile floating point numbers using a SSE2 code-path by default on OS X.
- ssp 17y ago
  One pedantic correction: SSE is the 4-wide floating point instruction set. SSE2 added integer instructions using the same registers.
wendroid 17y ago
On plan9 Trying compiler constants first... f is 0.12345, rounded to 0.1235, expanded to 0.123450003564357760000000000000 d is 0.12345, rounded to 0.1235, expanded to 0.123450000000000000000000000000 ld is 0.12345, rounded to 0.1235, expanded to 0.123450000000000000000000000000 Now trying derived values... f is 0.12345, rounded to 0.1235, expanded to 0.123450003564357760000000000000 d is 0.12345, rounded to 0.1235, expanded to 0.123450000000000000000000000000 ld is 0.12345, rounded to 0.1235, expanded to 0.123450000000000000000000000000 So don't blame the architecture

48 points by profquail 17y ago

It has been long known that one cannot rely on floating point for precise calculations, e.g. monetary values. One should use integers instead or some kind of other precise representation. But it does surprise me that even on the same CPU architecture you cannot expect every machine to produce the same results.

profquail 17y ago
I concentrated in optimization and numerical analysis for my undergrad degree in Math, and I'm still surprised on a regular basis how many programmers out there don't know about the inaccuracies and errors that come out of floating-point representations.
- pmorici 17y ago
  Are you really that surprised. Crappy programmers abound and many introductory programming classes have students write programs to calculate monetary things using floating point so it's no small wonder many people think it's ok.
- jwr 17y ago
  My thoughts exactly. Tom Christiansen brings these things up as if they were a huge surprise, but they aren't, or at least shouldn't be. To put it another way, it isn't Apple doing something wrong. If your code depends on rounding errors in FP representations, your code is buggy. That's also why good numerical algorithms are hard to design. You need to worry about losing precision. As a simple example, if you calculate (a-b)c and a is almost equal to b, you're much better off doing ac-b*c. These tricks are important.
- brettnak 17y ago
  As did, I. I am a professional programmer now and didn't go through the CS school ( To all sophomores out there: We do exist. ) Do they really not teach this sort of thing in a formal CS school?
  - wtallis 17y ago
    Most CS undergraduate programs in the US suck enough these days that numerical analysis isn't taught at the undergrad level. For example, at my school, undergraduates get to choose between the automata/grammars/computability class and a "numerical methods" class that is really watered down. There is a two-semester undergraduate numerical analysis class, but as I recall it can't count toward your degree except as a general elective. To get numerical analysis to count as a CS elective, you have to take the graduate level class.
barrkel 17y ago
For the same CPU architecture, compiler and FPU control word, you should expect the same results. For example, on x86, many C / C++ compilers interpret long double as a 64-bit float while others interpret it as 80-bit extended precision. And extended precision TBYTE (in Intel syntax) does actually take up 10 bytes. There's a speed loss for not aligning, but no correctness loss. In a possibly related note, however, Apple has bizarre and unnecessarily strict alignment policies for Mac OS on x86: http://blogs.embarcadero.com/eboling/2009/05/20/5607 http://blogs.embarcadero.com/eboling/2009/05/20/5607
- tentonova2 17y ago
  For the same CPU architecture, compiler and FPU control word, you should expect the same results. That's a stretch. Just because it's called "GCC" doesn't mean it behaves exactly the same on every platform, even if the CPU architecture is the same. Apple makes extensive modifications, has their own ABI (see below), etc. In a possibly related note, however, Apple has bizarre and unnecessarily strict alignment policies for Mac OS on x86: It's not "bizarre and unnecessarily strict" if you want to be able to rely on SSE2+. Apple had the advantage of being able to define their ABI without regard to most legacy concerns, and so they did. The reason that it's strictly enforced everywhere is that since Apple's compilers use SSE2+ they must be able to assume that, at function entry, the stack is properly (16 byte) aligned. I understand your pain -- I've had to update a JIT implementation to deal with this, along with quite a bit of assembly that assumed 4 byte alignment, but Apple's reasoning makes sense. See also: http://stackoverflow.com/questions/612443/why-does-the-mac-abi-require-16-byte-stack-alignment-for-x86-32/898760#898760 http://stackoverflow.com/questions/612443/why-does-the-mac-a...
  - barrkel 17y ago
    The stack is not actually aligned on function entry, because the return address is on top, so more alignment will be needed to avoid SSE2 locals being misaligned. It's not so hard for the callee side of the ABI to make sure the stack is aligned if it's going to use SSE2 and friends; it's rather more onerous to require every call site to make the alignments for the benefit of the callee.

profquail 17y ago

Tell PG: When I submitted this article, HN stripped the "!" characters out of the title...but I was then able to edit the story and put them back in there. I suppose that means there's a small bug in there somewhere...

notauser 17y ago
You can also use Unicode in comments, which may or ʎɐɯ ʇou be a bug but did surprise me.
- sailormoon 17y ago
  Why on earth would the ability to use unicode in comments be a bug? Seems like a basic feature of any modern site.
Perceval 17y ago
It's probably a feature not a bug, designed to automatically de-sensationalize sensationalist titles!!!
- profquail 17y ago
  Good point (I don't like sensational titles either) -- but shouldn't HN disallow you to edit the title and put them back in then?
  - echaozh 17y ago
    Few people would come back and add the sensation back. If they do, they must have thought about it and been more rational.
lg 17y ago
i think it's because editing doesn't apply the mysterious scrubrules, might be intentional.

TallGuyShort 17y ago

Wasn't a very similar phenomenon a huge blow to Intel's PR some years back?

profquail 17y ago
Sort of, but that had to do with a bug in the actual hardware of the Pentium Pro chip: http://en.wikipedia.org/wiki/Pentium_FDIV_bug http://en.wikipedia.org/wiki/Pentium_FDIV_bug
wglb 17y ago
No, that was not the small errors that are shown here, but rather errors of very significant magnitude.

ajross 17y ago

This isn't a new problem. The 80-bit internal 8087 FPU precision has always been a mismatch for the 64 bit IEEE double representation, even before the presence of SSE registers (which don't have the 80 bit mode) complicated things. Intel 8087 code has always been able to produce different results for the same source code, depending on when/whether/which intermediate results get spilled to memory. The Motorola 68k FPU had the same issue with higher internal than external precision, IIRC. This isn't a bug. Both code paths produce results of the highest representable precision of the hardware in question. It's just that there are multiple hardware units capable of giving you the answer, and Apple's toolchain picks a different one than whatever Tom is using elsewhere. And as has been pointed out in FooBarWidget's comment -- any code that relies on bit-precise results from floating point computation is almost certainly concealing precision bugs anyway. That's not the right way to approach floating point architecture.

ars 17y ago
For people unfamiliar with this, "spilled to memory" means rounded to 64 bits (from 80). x87 does floating point math with 80 bits (in registers). But the variables are stored in memory locations with 64 bits. So the results are rounded. The problem comes from when to do the rounding. And that can vary depending on if a register is needed for something else, optimizations, the order of code, etc.

tentonova2 17y ago

This shouldn't surprise anyone and I can't say I understand the breathless post. For more information, I recommend Write Great Code: Understanding the Machine -- Chapter 4, Floating-Point Representation.

modeless 17y ago

What Every Computer Scientist Should Know About Floating-Point Arithmetic: http://docs.sun.com/source/806-3568/ncg_goldberg.html http://docs.sun.com/source/806-3568/ncg_goldberg.html

jcl 17y ago

This reminds me of a page on fast and accurate geometric primitives; the code requires that you to flip the Intel FPU into a reduced precision mode to make the error easier to estimate. http://www.cs.cmu.edu/~quake/robust.pc.html http://www.cs.cmu.edu/~quake/robust.pc.html

9oliYQjP 17y ago

Apple doesn't use the x87 floating point unit. They've never needed to because they can rely on the SSE2 unit since any Intel Mac ever created has one. This isn't necessarily the case on the PC even though every modern one will have it as well. I'm pretty sure gcc is set to compile floating point numbers using a SSE2 code-path by default on OS X.

ssp 17y ago
One pedantic correction: SSE is the 4-wide floating point instruction set. SSE2 added integer instructions using the same registers.

wendroid 17y ago

On plan9 Trying compiler constants first... f is 0.12345, rounded to 0.1235, expanded to 0.123450003564357760000000000000 d is 0.12345, rounded to 0.1235, expanded to 0.123450000000000000000000000000 ld is 0.12345, rounded to 0.1235, expanded to 0.123450000000000000000000000000 Now trying derived values... f is 0.12345, rounded to 0.1235, expanded to 0.123450003564357760000000000000 d is 0.12345, rounded to 0.1235, expanded to 0.123450000000000000000000000000 ld is 0.12345, rounded to 0.1235, expanded to 0.123450000000000000000000000000 So don't blame the architecture