6 ms·

Many instructions (that likely see heavy use in MKL) are approximations, the exact implementation and behavior of these approximations varies across platforms (

by creato 7y ago

Many instructions (that likely see heavy use in MKL) are approximations, the exact implementation and behavior of these approximations varies across platforms (including possibly architecture versions). The docs you quoted describes the necessary conditions for using an instruction to avoid crashes/illegal instruction issues. That doesn't cover these kinds of more subtle differences that exist. For example, I just made a plot of |1/x - rcpps(x)|: https://snipboard.io/Rei74t.jpg https://snipboard.io/Rei74t.jpg

jchw 7y ago
I'm very much not buying it. Although of course an instruction for something like approximations can vary across different micro-architectures and vendors, there has to be some definition for what the baseline is. Without it, the instruction may as well output junk. Indeed, I just checked and Intel's "Intel 64 and IA-32 Architectures Software Developer’s Manual" contains a bound on the error of the approximation in rcpps: The relative error for this approximation is: |Relative Error| ≤ 1.5 ∗ 2^−12 In fact, the document goes on to further define a lot of the behavior of the instruction in edge cases. Because that's kind of important for an architecture. If the extended instruction set did different things on every different processor, it would be useless. So I don't think this is really the issue. I don't think Intel MKL defines what the error is for many operations, so it is presumably not defined, and therefore it'd be hard for AMD's implementation to run afoul. It's also not going to be reproducibility/consistency, since they cover this in a separate issue: https://software.intel.com/en-us/articles/consistency-of-floating-point-results-using-the-intel-compiler https://software.intel.com/en-us/articles/consistency-of-flo...
- creato 7y ago
  That's the maximum error. The error is often much less than that (as the plot shows). Code might unintentionally be relying on the error being much smaller than the spec requires, and on another architecture, the instruction might not meet this (unintentional!) requirement. Of course an issue stemming from something like this is rare/unlikely. But Intel also can't assume that the consequences for a related error are small. It seems like most people commenting irately about this in this thread are hobbyists or are otherwise working on low consequence software. Yeah, for these uses, it would probably make sense for intel to just look at the CPUID and leave it at that. But they can't know that, and they aren't optimizing solely for these use cases.
  - jchw 7y ago
    I am suggesting that if Intel does not care about their code running on AMD processors, then it should not arbitrarily change behavior based on the presence of the string “GenuineIntel.” End of story. But I suspect Intel does care, in the sense that they certainly care about how they perform and benchmark against AMD. Let’s put it another way. What happens if Intel enables AVX2 for AMD and it doesn’t work? 1. Precision is not as good as intel? 1.a. Many developers care and it impacts AMD negatively. (Probably nothing happens since nothing is actually broken per se.) 1.b. Few developers care and it suggests Intel is not optimizing for the best performance trade-off. 2. Programs using MKL crash on AMD processors. 2.a. There is a bug in MKL: it probably gets fixed with little fanfare. 2.b. There is a bug on AMD processors, and it probably gets fixed in microcode. 3. AMD processors produce wrong results. There seems like there can only be one outcome here, and it’s probably worse for AMD than Intel. The Intel FDIV bug proves that people take these things very seriously. At best it could be fixed in microcode, and at absolute worst AVX2 could be disabled entirely in an update. The best argument for why Intel would still rationally avoid AVX2 on AMD outside of the above is to ensure their customer’s code runs correctly on AMD processors. However, there’s still some problems: - They have to contend with their own processors, too. If they write code they know depends on non-public details of the architecture, they could very well break themselves. - What is the definition for “break?” Intel surely has test suites for their software, but just because the software’s precision may be worse (still pure conjecture) when running under AMD does not suggest it is broken. All we can say is that users may rely on the behavior of precision on their specific configuration, but the problem is that’s not specific to AMD. Intel alludes to the fact that precision can be different amongst Intel processors in several places on MKL’s documentation. Nothing stops someone from improperly relying on this. By playing fast and loose with specifying an architecture then writing software that disregards it, I’m not sure how defensible a position Intel is sitting in from pretty much any perspective. > “otherwise working on low consequence software” No offense but it is hard to respectfully respond to something like this, even though I know it wasn't directed at me personally. I hope you don’t think about coworkers with this mindset.
    - creato 7y ago
      > Let’s put it another way. What happens if Intel enables AVX2 for AMD and it doesn’t work? The "safe" thing to do is rather than work through all of these cases you've considered, is to just give up on it. > No offense but it is hard to respectfully respond to something like this, even though I know it wasn't directed at me personally. I hope you don’t think about coworkers with this mindset. I didn't intend it to be disrespectful. It's not a value judgement. Maybe these aren't the best terms, but I've worked on both what I would call "low consequence" software, and "high consequence" software. The tolerance for decision making about things like swapping out hardware is different. In some cases, this can be done with little hesitation. In other cases, it would cost millions of dollars to test the system to the point where it could be trusted. Maybe the solution to the internet pitchfork wielders is for intel to simply stop trying to serve both of these sets of users with the same software. But I suspect they simply don't care about this problem enough to do anything.
tagrun 7y ago
And how exactly does that plot show that AMD's AVX instructions are faulty?
- creato 7y ago
  It doesn't. It shows that the behavior of instructions like this is very unpredictable and subtle, and that code using it that was tested on one processor might not behave the same on another processor. It doesn't mean one is faulty and the other is not.
  - jchw 7y ago
    A processor would be faulty if the error or behavior fell outside of the architecture specification. The behavior is not unpredictable because the behavior is defined by a specification.
  - tagrun 7y ago
    I completely disagree. Your plot doesn't show CPU instructions are unpredictable (which is not correct). Yes, IEEE floats are not the same as real numbers (which is what your plot is showing): they have inherent precision errors, and float operations are not even associative. That being said, IEEE floats are carefully and consistently defined, and are perfect predictable. The unpredictability you claim is not due to stochastic errors or faulty implementation of CPU vendors, they're a part of the IEEE definition and are deterministic.
    - creato 7y ago
      The '1/x' part of this experiment is described by an IEEE spec. The reciprocal instructions are not. This experiment doesn't cover the difference between theoretical real numbers and float computations, it's the difference between two different ways of working with floats. One of them is IEEE specified, the other is not. In practice, using these kinds of instructions (which are not specified by IEEE) can give massive performance advantages.
      - 7y ago
BeeOnRope 7y ago
Your argument might be worth considering if Intel didn't use the same "cripple AMD" check across a lot of other libraries and their own compiler too - where there are no such "numerical approximation" concerns. They've been doing it long before MKL [1]. --- [1] https://www.agner.org/optimize/blog/read.php?i=49 https://www.agner.org/optimize/blog/read.php?i=49

by creato 7y ago

jchw 7y ago

I'm very much not buying it. Although of course an instruction for something like approximations can vary across different micro-architectures and vendors, there has to be some definition for what the baseline is. Without it, the instruction may as well output junk. Indeed, I just checked and Intel's "Intel 64 and IA-32 Architectures Software Developer’s Manual" contains a bound on the error of the approximation in rcpps: The relative error for this approximation is: |Relative Error| ≤ 1.5 ∗ 2^−12 In fact, the document goes on to further define a lot of the behavior of the instruction in edge cases. Because that's kind of important for an architecture. If the extended instruction set did different things on every different processor, it would be useless. So I don't think this is really the issue. I don't think Intel MKL defines what the error is for many operations, so it is presumably not defined, and therefore it'd be hard for AMD's implementation to run afoul. It's also not going to be reproducibility/consistency, since they cover this in a separate issue: https://software.intel.com/en-us/articles/consistency-of-floating-point-results-using-the-intel-compiler https://software.intel.com/en-us/articles/consistency-of-flo...

creato 7y ago
That's the maximum error. The error is often much less than that (as the plot shows). Code might unintentionally be relying on the error being much smaller than the spec requires, and on another architecture, the instruction might not meet this (unintentional!) requirement. Of course an issue stemming from something like this is rare/unlikely. But Intel also can't assume that the consequences for a related error are small. It seems like most people commenting irately about this in this thread are hobbyists or are otherwise working on low consequence software. Yeah, for these uses, it would probably make sense for intel to just look at the CPUID and leave it at that. But they can't know that, and they aren't optimizing solely for these use cases.
- jchw 7y ago
  I am suggesting that if Intel does not care about their code running on AMD processors, then it should not arbitrarily change behavior based on the presence of the string “GenuineIntel.” End of story. But I suspect Intel does care, in the sense that they certainly care about how they perform and benchmark against AMD. Let’s put it another way. What happens if Intel enables AVX2 for AMD and it doesn’t work? 1. Precision is not as good as intel? 1.a. Many developers care and it impacts AMD negatively. (Probably nothing happens since nothing is actually broken per se.) 1.b. Few developers care and it suggests Intel is not optimizing for the best performance trade-off. 2. Programs using MKL crash on AMD processors. 2.a. There is a bug in MKL: it probably gets fixed with little fanfare. 2.b. There is a bug on AMD processors, and it probably gets fixed in microcode. 3. AMD processors produce wrong results. There seems like there can only be one outcome here, and it’s probably worse for AMD than Intel. The Intel FDIV bug proves that people take these things very seriously. At best it could be fixed in microcode, and at absolute worst AVX2 could be disabled entirely in an update. The best argument for why Intel would still rationally avoid AVX2 on AMD outside of the above is to ensure their customer’s code runs correctly on AMD processors. However, there’s still some problems: - They have to contend with their own processors, too. If they write code they know depends on non-public details of the architecture, they could very well break themselves. - What is the definition for “break?” Intel surely has test suites for their software, but just because the software’s precision may be worse (still pure conjecture) when running under AMD does not suggest it is broken. All we can say is that users may rely on the behavior of precision on their specific configuration, but the problem is that’s not specific to AMD. Intel alludes to the fact that precision can be different amongst Intel processors in several places on MKL’s documentation. Nothing stops someone from improperly relying on this. By playing fast and loose with specifying an architecture then writing software that disregards it, I’m not sure how defensible a position Intel is sitting in from pretty much any perspective. > “otherwise working on low consequence software” No offense but it is hard to respectfully respond to something like this, even though I know it wasn't directed at me personally. I hope you don’t think about coworkers with this mindset.
  - creato 7y ago
    > Let’s put it another way. What happens if Intel enables AVX2 for AMD and it doesn’t work? The "safe" thing to do is rather than work through all of these cases you've considered, is to just give up on it. > No offense but it is hard to respectfully respond to something like this, even though I know it wasn't directed at me personally. I hope you don’t think about coworkers with this mindset. I didn't intend it to be disrespectful. It's not a value judgement. Maybe these aren't the best terms, but I've worked on both what I would call "low consequence" software, and "high consequence" software. The tolerance for decision making about things like swapping out hardware is different. In some cases, this can be done with little hesitation. In other cases, it would cost millions of dollars to test the system to the point where it could be trusted. Maybe the solution to the internet pitchfork wielders is for intel to simply stop trying to serve both of these sets of users with the same software. But I suspect they simply don't care about this problem enough to do anything.

tagrun 7y ago

And how exactly does that plot show that AMD's AVX instructions are faulty?

creato 7y ago
It doesn't. It shows that the behavior of instructions like this is very unpredictable and subtle, and that code using it that was tested on one processor might not behave the same on another processor. It doesn't mean one is faulty and the other is not.
- jchw 7y ago
  A processor would be faulty if the error or behavior fell outside of the architecture specification. The behavior is not unpredictable because the behavior is defined by a specification.
- tagrun 7y ago
  I completely disagree. Your plot doesn't show CPU instructions are unpredictable (which is not correct). Yes, IEEE floats are not the same as real numbers (which is what your plot is showing): they have inherent precision errors, and float operations are not even associative. That being said, IEEE floats are carefully and consistently defined, and are perfect predictable. The unpredictability you claim is not due to stochastic errors or faulty implementation of CPU vendors, they're a part of the IEEE definition and are deterministic.
  - creato 7y ago
    The '1/x' part of this experiment is described by an IEEE spec. The reciprocal instructions are not. This experiment doesn't cover the difference between theoretical real numbers and float computations, it's the difference between two different ways of working with floats. One of them is IEEE specified, the other is not. In practice, using these kinds of instructions (which are not specified by IEEE) can give massive performance advantages.
    - 7y ago

BeeOnRope 7y ago

Your argument might be worth considering if Intel didn't use the same "cripple AMD" check across a lot of other libraries and their own compiler too - where there are no such "numerical approximation" concerns. They've been doing it long before MKL [1]. --- [1] https://www.agner.org/optimize/blog/read.php?i=49 https://www.agner.org/optimize/blog/read.php?i=49