4 ms·

A lesson I learned as an undergraduate: Sometimes changing optimization settings can yield different answers. It's been ~17 years since then, so I can't recall

by ISL 6y ago

A lesson I learned as an undergraduate: Sometimes changing optimization settings can yield different answers. It's been ~17 years since then, so I can't recall the options used, but seeing different results emerge when I used a higher optimization option caused an unanticipated puff of smoke from my little brain. A possible moral of the story -- test every little change.

gumby 6y ago
I used to be a compiler developer and this was no big deal (except when it was an actual code generator bug!). The compiler switches tell the compiler to do different things, so of course some of them change the output semantics. It’s up to the user to decide what she wants — it’s not like the differences are obscure or surprising. Then I went on to other things and realized that developers are experts in their domains, and those “obvious” things were only clear to someone steeped in the code — not obvious at all. In fact some of them are no longer obvious to me any more after decades doing other things. Hopefully I’ve become at least slightly less arrogant and thoughtless by now.
- kyllo 6y ago
  Are you talking about things like very small differences in the results of floating point calculations?
  - recursive 6y ago
    Very small errors in floating point calculations can be made arbitrarily large.
  - hansvm 6y ago
    There are domains where floating point problems being ill-conditioned is the norm rather than an exception. Try running `1e100+1e20-1e100` in python, js, etc....
  - gumby 6y ago
    Some optimization switches are off by default as they can subtly change the semantics of your code. Yes, this can trip up even quite knowledgeable people as with the fp post from Lemire the other day — he thought gcc was making a mistake, but it was doing the correct thing. But there are plenty of others, for example telling the compiler not to worry about aliasing. I might use this on certain modules or individual files which were carefully written for this. Or asking for more aggressive loop unrolling Which could increase code size but improve performance (or not!). 99.9% the defaults are fine.
- gnufx 6y ago
  I used to be a compiler maintainer because I was the maintainer for a large scientific suite -- i.e. I was on both sides of the fence -- and I haven't really changed my opinion. The arrogance I've seen has mainly been from scientific users who lecture you about the code you've worked on and the language standard, neither of which they know. A classic case is ignoring the Fortran storage association rules (that happen to have some significance in GCC history).
gnufx 6y ago
I'd say the moral is to write standard-conforming, well-defined code, and use standard-conforming compiler options, e.g. not the Intel defaults. Also use the available warnings and diagnostics.
- deathanatos 6y ago
  Well-formed code is not sufficient. For example, C11 says this about floating point: > The accuracy of the floating-point operations (+, -, <star>, /) and of the library functions in <math.h> and <complex.h> that return floating-point results is implementation-defined, as is the accuracy of the conversion between floating-point internal representations and string representations performed by the library functions in <stdio.h>, <stdlib.h>, and <wchar.h>. The implementation may state that the accuracy is unknown. There was an article recently talking about the bizarre rounding that happens when the x86 FPU registers get involved. You get one rounding for operations occurring in the register (which is 80-bits wide) and another if you get kicked out of the register and into RAM (which is only 64-bits wide, for doubles).
  - etimberg 6y ago
    I've seen that kind of rounding error in code that went through emscripten. The initial C++ code predated std::log10 so it was implemented using a change of base, i.e. log(value)/log(10). When this was translated into Javascript, it was as Math.log(value) / Math.log(10) which rounds to 64 bits each step of the way. In this particular calculation, this caused error to build up over time. Replacing the C++ code with std::log10 translated into Javascript as Math.log10 and since the browsers natively implemented that, the values would stay in registers and only round once.
  - gnufx 6y ago
    I said "well-defined" for that sort of reason, but there's more. Fortunately I haven't had to discuss -ffloat-store (which may have been originally for m68k) for a long time.

by ISL 6y ago

gumby 6y ago

I used to be a compiler developer and this was no big deal (except when it was an actual code generator bug!). The compiler switches tell the compiler to do different things, so of course some of them change the output semantics. It’s up to the user to decide what she wants — it’s not like the differences are obscure or surprising. Then I went on to other things and realized that developers are experts in their domains, and those “obvious” things were only clear to someone steeped in the code — not obvious at all. In fact some of them are no longer obvious to me any more after decades doing other things. Hopefully I’ve become at least slightly less arrogant and thoughtless by now.

kyllo 6y ago
Are you talking about things like very small differences in the results of floating point calculations?
- recursive 6y ago
  Very small errors in floating point calculations can be made arbitrarily large.
- hansvm 6y ago
  There are domains where floating point problems being ill-conditioned is the norm rather than an exception. Try running `1e100+1e20-1e100` in python, js, etc....
- gumby 6y ago
  Some optimization switches are off by default as they can subtly change the semantics of your code. Yes, this can trip up even quite knowledgeable people as with the fp post from Lemire the other day — he thought gcc was making a mistake, but it was doing the correct thing. But there are plenty of others, for example telling the compiler not to worry about aliasing. I might use this on certain modules or individual files which were carefully written for this. Or asking for more aggressive loop unrolling Which could increase code size but improve performance (or not!). 99.9% the defaults are fine.
gnufx 6y ago
I used to be a compiler maintainer because I was the maintainer for a large scientific suite -- i.e. I was on both sides of the fence -- and I haven't really changed my opinion. The arrogance I've seen has mainly been from scientific users who lecture you about the code you've worked on and the language standard, neither of which they know. A classic case is ignoring the Fortran storage association rules (that happen to have some significance in GCC history).

gnufx 6y ago

I'd say the moral is to write standard-conforming, well-defined code, and use standard-conforming compiler options, e.g. not the Intel defaults. Also use the available warnings and diagnostics.

deathanatos 6y ago
Well-formed code is not sufficient. For example, C11 says this about floating point: > The accuracy of the floating-point operations (+, -, <star>, /) and of the library functions in <math.h> and <complex.h> that return floating-point results is implementation-defined, as is the accuracy of the conversion between floating-point internal representations and string representations performed by the library functions in <stdio.h>, <stdlib.h>, and <wchar.h>. The implementation may state that the accuracy is unknown. There was an article recently talking about the bizarre rounding that happens when the x86 FPU registers get involved. You get one rounding for operations occurring in the register (which is 80-bits wide) and another if you get kicked out of the register and into RAM (which is only 64-bits wide, for doubles).
- etimberg 6y ago
  I've seen that kind of rounding error in code that went through emscripten. The initial C++ code predated std::log10 so it was implemented using a change of base, i.e. log(value)/log(10). When this was translated into Javascript, it was as Math.log(value) / Math.log(10) which rounds to 64 bits each step of the way. In this particular calculation, this caused error to build up over time. Replacing the C++ code with std::log10 translated into Javascript as Math.log10 and since the browsers natively implemented that, the values would stay in registers and only round once.
- gnufx 6y ago
  I said "well-defined" for that sort of reason, but there's more. Fortunately I haven't had to discuss -ffloat-store (which may have been originally for m68k) for a long time.