4 ms·

Spying on floating point behavior of existing unmodified scientific applications

42 points by aluket 6y ago

throwawaybutwhy 6y ago
Wow. Kudos to Adrian for finding this pearl [0]. Source code (non-commercial use license): [1]. [0] http://pdinda.org/Papers/hpdc20.pdf http://pdinda.org/Papers/hpdc20.pdf [1] http://presciencelab.org/FPSpy/fpspy-1.0.tgz http://presciencelab.org/FPSpy/fpspy-1.0.tgz
FartyMcFarter 6y ago
Regarding the "inexact results", how do we know that the application authors didn't take those into account while analyzing their algorithms? If you know that your end result is accurate enough, you don't need to care about those.
- steerablesafe 6y ago
  Inexact is pretty much meaningless. You will get inexact results in any meaningful calculation you do. And floating point rounding error is only one source of error in these kinds of simulations. Numerically solving complex, non-linear partial differential equations brings its own share of error sources.
  - Someone 6y ago
    Also, if I understand the proposed solution correctly, it doesn’t make sense. I think the article says “replace the relatively rare inexact calculations by (more expensive) calls to libraries that produce correct results, then stuff the result in a IEEE float/double, so that the less rare calls can run cheaply. Problems with that approach: - a computation at a given location in the code can be exact zillions of times, only to be inexact on the zillion+1th call, so instrumenting cannot prove a computation to be safe for doing on the hardware. - worse, the “stuff the result in a IEEE float/double” will cause inexact rounding (there may be extremely rare instances where the value of a trig function or something equally complex is exactly representable, but the CPU doesn’t compute that exact number, but those will be extremely rare)
- JimTheMan 6y ago
  From my memories of classes computation fluid dynamics from almost a decade ago: The whole thing was a balancing act of making sure that solutions were convergent and not unstable. Floating point numbers would fall in that category. Ie its OK to be a little a tiny little bit wrong as long as the final result converges on something reasonable. Especially given the other sources of error are magnitudes higher (for instance, picking a timestep that is way too large, relaxation factors when solving... I don't remember exactly but it felt like a bit of an art within engineering to get the thing to be reasonable. https://www.flow3d.com/resources/cfd-101/numerical-issues/numerical-instability/ https://www.flow3d.com/resources/cfd-101/numerical-issues/nu...
Const-me 6y ago
I don’t think these results mean much. Take a look: https://godbolt.org/z/Wd98eG https://godbolt.org/z/Wd98eG When asked to divide 2 float numbers by another 2 numbers, clang compiler used division instruction for 4-wide vectors (for 32-bit floats, modern hardware only implements 4-wide and 8-wide vectors), that code runs 0.0 / 0.0 in the unused higher 2 lanes. That code is correct (we only want 2 lower lanes, and the result there is good), but FPSpy would then show these red lights for that software.
- yorwba 6y ago
  Does the vdivps instruction set the division-by-zero flag? If so, I don't think compiling code that doesn't divide by zero into code that does is correct.
  - Const-me 6y ago
    > Does the vdivps instruction set the division-by-zero flag? Yes. It even triggers a runtime exception when these are enabled in the corresponding bit of MXCSR status register. > I don't think compiling code that doesn't divide by zero into code that does is correct. There’re good arguments both for and against. Personally, I think it’s fine. The optimization clang did almost doubled the performance. Most people don’t touch these bits in MXCSR, and the default values are good. When manually vectorizing, I sometimes manually writing code like that when the alternatives are much slower. In this particular example it’s possible to fix for free by replacing vmovsd with vmovddup, _mm_loaddup_pd in C++. But it’s not always the case, e.g. when dealing with 3D vectors you need extra instructions. In SSE the fastest way is _mm_insert_ps, it can either copy 3-rd lane into 4-th one, or selectively zero out lanes, a single very fast instructions that doesn't touch memory. In AVX however you would need memory access + _mm256_blend_ps.
  - rrss 6y ago
    FYI, clang 10+ has support for -ffp-exception-behavior that enables configuring this behavior. > maytrap: The compiler avoids transformations that may raise exceptions that would not have been raised by the original code. Constant folding performed by the compiler is exempt from this option. https://releases.llvm.org/10.0.0/tools/clang/docs/UsersManual.html https://releases.llvm.org/10.0.0/tools/clang/docs/UsersManua... You can see here that this prevents clang from using vdivps: https://godbolt.org/z/WG7bEE https://godbolt.org/z/WG7bEE.
alasarmas 6y ago
LD_PRELOAD strikes again! I learned about LD_PRELOAD last year from another HN post, and I'm not saying it's a silver bullet for instrumenting binaries on Linux, but it does seem to come in handy. Edit: a SO question about "the LD_PRELOAD trick": https://stackoverflow.com/questions/426230/what-is-the-ld-preload-trick https://stackoverflow.com/questions/426230/what-is-the-ld-pr...
- gnufx 6y ago
  LD_PRELOAD is the normal way to add instrumentation on ELF HPC; it's more-or-less assumed by the MPI profiling facility. I'm surprised that LD_PRELOAD should be news to anyone at this stage, though. For instance, glibc has hooks for interposing that way, like a different malloc. It's definitely not general, specifically not for static binaries (Cray?). Dyninst is the usual alternative.
gnufx 6y ago
You don't need anything special to say you have relevant exceptions. GCC with a benchmark set currently in progress (which has other problems): Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG The "inexact" exception message you got at the end of essentially anything on SunOS at one time was quite annoying. (I can't remember whether that was specific to the SunPro compiler.) Fortran provides control over this with IEEE_ARITHMETIC, but I don't remember whether that's actually part of the standard.

42 points by aluket 6y ago

Wow. Kudos to Adrian for finding this pearl [0]. Source code (non-commercial use license): [1]. [0] http://pdinda.org/Papers/hpdc20.pdf http://pdinda.org/Papers/hpdc20.pdf [1] http://presciencelab.org/FPSpy/fpspy-1.0.tgz http://presciencelab.org/FPSpy/fpspy-1.0.tgz

FartyMcFarter 6y ago

Regarding the "inexact results", how do we know that the application authors didn't take those into account while analyzing their algorithms? If you know that your end result is accurate enough, you don't need to care about those.

steerablesafe 6y ago
Inexact is pretty much meaningless. You will get inexact results in any meaningful calculation you do. And floating point rounding error is only one source of error in these kinds of simulations. Numerically solving complex, non-linear partial differential equations brings its own share of error sources.
- Someone 6y ago
  Also, if I understand the proposed solution correctly, it doesn’t make sense. I think the article says “replace the relatively rare inexact calculations by (more expensive) calls to libraries that produce correct results, then stuff the result in a IEEE float/double, so that the less rare calls can run cheaply. Problems with that approach: - a computation at a given location in the code can be exact zillions of times, only to be inexact on the zillion+1th call, so instrumenting cannot prove a computation to be safe for doing on the hardware. - worse, the “stuff the result in a IEEE float/double” will cause inexact rounding (there may be extremely rare instances where the value of a trig function or something equally complex is exactly representable, but the CPU doesn’t compute that exact number, but those will be extremely rare)
JimTheMan 6y ago
From my memories of classes computation fluid dynamics from almost a decade ago: The whole thing was a balancing act of making sure that solutions were convergent and not unstable. Floating point numbers would fall in that category. Ie its OK to be a little a tiny little bit wrong as long as the final result converges on something reasonable. Especially given the other sources of error are magnitudes higher (for instance, picking a timestep that is way too large, relaxation factors when solving... I don't remember exactly but it felt like a bit of an art within engineering to get the thing to be reasonable. https://www.flow3d.com/resources/cfd-101/numerical-issues/numerical-instability/ https://www.flow3d.com/resources/cfd-101/numerical-issues/nu...

Const-me 6y ago

I don’t think these results mean much. Take a look: https://godbolt.org/z/Wd98eG https://godbolt.org/z/Wd98eG When asked to divide 2 float numbers by another 2 numbers, clang compiler used division instruction for 4-wide vectors (for 32-bit floats, modern hardware only implements 4-wide and 8-wide vectors), that code runs 0.0 / 0.0 in the unused higher 2 lanes. That code is correct (we only want 2 lower lanes, and the result there is good), but FPSpy would then show these red lights for that software.

yorwba 6y ago
Does the vdivps instruction set the division-by-zero flag? If so, I don't think compiling code that doesn't divide by zero into code that does is correct.
- Const-me 6y ago
  > Does the vdivps instruction set the division-by-zero flag? Yes. It even triggers a runtime exception when these are enabled in the corresponding bit of MXCSR status register. > I don't think compiling code that doesn't divide by zero into code that does is correct. There’re good arguments both for and against. Personally, I think it’s fine. The optimization clang did almost doubled the performance. Most people don’t touch these bits in MXCSR, and the default values are good. When manually vectorizing, I sometimes manually writing code like that when the alternatives are much slower. In this particular example it’s possible to fix for free by replacing vmovsd with vmovddup, _mm_loaddup_pd in C++. But it’s not always the case, e.g. when dealing with 3D vectors you need extra instructions. In SSE the fastest way is _mm_insert_ps, it can either copy 3-rd lane into 4-th one, or selectively zero out lanes, a single very fast instructions that doesn't touch memory. In AVX however you would need memory access + _mm256_blend_ps.
- rrss 6y ago
  FYI, clang 10+ has support for -ffp-exception-behavior that enables configuring this behavior. > maytrap: The compiler avoids transformations that may raise exceptions that would not have been raised by the original code. Constant folding performed by the compiler is exempt from this option. https://releases.llvm.org/10.0.0/tools/clang/docs/UsersManual.html https://releases.llvm.org/10.0.0/tools/clang/docs/UsersManua... You can see here that this prevents clang from using vdivps: https://godbolt.org/z/WG7bEE https://godbolt.org/z/WG7bEE.

alasarmas 6y ago

LD_PRELOAD strikes again! I learned about LD_PRELOAD last year from another HN post, and I'm not saying it's a silver bullet for instrumenting binaries on Linux, but it does seem to come in handy. Edit: a SO question about "the LD_PRELOAD trick": https://stackoverflow.com/questions/426230/what-is-the-ld-preload-trick https://stackoverflow.com/questions/426230/what-is-the-ld-pr...

gnufx 6y ago
LD_PRELOAD is the normal way to add instrumentation on ELF HPC; it's more-or-less assumed by the MPI profiling facility. I'm surprised that LD_PRELOAD should be news to anyone at this stage, though. For instance, glibc has hooks for interposing that way, like a different malloc. It's definitely not general, specifically not for static binaries (Cray?). Dyninst is the usual alternative.

gnufx 6y ago

You don't need anything special to say you have relevant exceptions. GCC with a benchmark set currently in progress (which has other problems): Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG The "inexact" exception message you got at the end of essentially anything on SunOS at one time was quite annoying. (I can't remember whether that was specific to the SunPro compiler.) Fortran provides control over this with IEEE_ARITHMETIC, but I don't remember whether that's actually part of the standard.