8 ms·

I bet there's more than one integer division unit per core. When I've been micro-optimizing performance-critical code, integer division shows up as a hot spot

I bet there's more than one integer division unit per core. When I've been micro-optimizing performance-critical code, integer division shows up as a hot spot regularly. I assume most developers don't think about the performance implications of coding up a / or % between two runtime values, preventing the compiler from doing any strength reduction. Apple must have seen this in their surely voluminous profiling of real-world applications.

mhh__ 5y ago
The fact that Intel and AMD apparently don't prioritize integer division could suggest that their profiling suggests it's not worth it, but with Apple's transistor budget at the moment they can afford it. Also keep in mind that this Xeon might not be really made for number crunching (not really sure)?
- masklinn 5y ago
  > The fact that Intel and AMD apparently don't prioritize integer division could suggest that their profiling suggests it's not worth it, but with Apple's transistor budget at the moment they can afford it. An other possibility is that Apple has a very different profiling base e.g. iOS applications, whereas Intel and AMD would have more artificial workloads, or be bound by workloads / profiles from scientific computing or the like (video games)?
- criddell 5y ago
  Would it be fair to characterize the M1 as being made for number crunching?
  - mhh__ 5y ago
    Refining number crunching to mean single threaded performance I would say yes, or at least definitely more so than the Intel chip
  - gameswithgo 5y ago
    Not any more than an intel/amd/etc cpu is. Like that XEON cpu is gonna crunch more numbers, just due to more cores.
    - mhh__ 5y ago
      If it was intended to be used in the cloud for example it's going to be doing more work but probably designed around a memory-bound load rather than integer throughput.
- pbsd 5y ago
  Intel greatly improved their divider implementation between Skylake and Icelake. The measurements in the OP are on Skylake-SP, prior to these improvements.
- rodgerd 5y ago
  Apple's chip designers have the advantage, I assume, of being able to wander down a hallway and ask what the telemetry from iOS and MacOS devices are telling them about real-world use.
  - mhh__ 5y ago
    Most of Intel's volume is probably shipped to customers who either don't care or buy a lot of CPUs in one go, so the advantage of this probably isn't quite as apparent as you'd imagine. What can definitely play a role is that (I don't think it's as much of a problem these days, but it definitely has been in the past) is the standard "benchmark" suites that chipmakers can beat each other over the head with e.g. I think it was Itanium that had a bunch of integer functional units mainly for the purpose of getting better SPEC numbers rather than working on the things that actually make programs fast (MEMORY) - I was maybe 1 or 2 when this chip came out, so this is nth-hand gossip, however.
    - anfilt 5y ago
      The Itantium did care about memory its one of the reasons they had massive caches compared to other archs of the era.
  - twoodfin 5y ago
    They also have the advantage of controlling the compiler used by effectively the entire development ecosystem. And have groomed that ecosystem through carrots and sticks to upgrade their compilers and rebuild their applications regularly (or use bitcode!) A compiler code generator that knows about a hypothetical two divide units (or just a much more efficient single unit) could be much more effective statically scheduling around them. I’d guess that the bulk of the software running on the highest margin Intel Xeons was compiled some years ago and tuned for microarchitectures even older.
    - mhh__ 5y ago
      >A compiler code generator that knows about a hypothetical two divide units (or just a much more efficient single unit) could be much more effective statically scheduling around them. I'm still completely blind to how they are actually used but GCC and LLVM both have pretty good internal representations of the microarchitecture they are compiling for. If I ever work it out I'll write a blog post about it, but this is an area where GCC and LLVM are both equally impenetrable.
buildbot 5y ago
I think you got the point most people miss - apple had a unique ability to profile every app on the Mac and iOS App Store, possibly in an automated way, as part of the app submission pipeline. Intel and AMD could go out and profile real work applications, and I’m sure they do, but to get to the same level of breadth is probably not possible.
- pvg 5y ago
  Is that really such a unique advantage for Apple? Intel and AMD can work with Microsoft to achieve something similar, for instance.
  - bch 5y ago
    I’m only speculating, but I’d think they wouldn’t even “have to work with” anybody - couldn’t they just instrument whatever they want?
    - pvg 5y ago
      Sure, I just meant there's a Microsoft store so if the there was something particularly advantageous to having access to an app store, Intel or AMD are not completely in the wilderness here.
      - zsmi 5y ago
        Not completely in the wilderness sure. But they also have many operating systems to consider. The other obvious ones are Linux, BSD, and, until very recently, macOS. Are you sure it's possible to do a micro-optimization for one without nerfing the others?
        ygra 5y ago
        Does the actual machine code doing computations really differ that much based on the OS? I'd guess the differences are just that you call different libraries and maybe a different calling convention, but nothing that would impact code running single-threaded without preemptions much.
CalChris 5y ago
AnandTech says there is one separate integer division unit in the A14 Firestorm which is what the M1 is based on. The article even mentions an increase in latency for integer division for the A14. https://www.anandtech.com/show/16226/apple-silicon-m1-a14-deep-dive/2 https://www.anandtech.com/show/16226/apple-silicon-m1-a14-de... https://images.anandtech.com/doci/16226/Firestorm_575px.png https://images.anandtech.com/doci/16226/Firestorm_575px.png
hctaw 5y ago
I'd quibble with whether folks understand the performance implications. I think most people know division is expensive. It's just non trivial to remove it. If it was that easy to replace, it would be done in hardware and division wouldn't be expensive anymore.
blt 5y ago
I thought "division is slow" was one of the more widely-known performance heuristics. I genuinely wonder how much of the division load in typical programs is an essential part of the problem vs. programmers not knowing/caring about slow division.
rwallace 5y ago
> When I've been micro-optimizing performance-critical code, integer division shows up as a hot spot regularly. Now I'm really curious. In my experience, integer division practically never happens (except by powers of 2 that are easily optimized down to a shift), to the point where I was frowning in puzzlement about why Apple spent resources optimizing it. How is the code you are looking at, bottlenecking on integer division? What on earth is it doing, to make that a frequent operation?

mhh__ 5y ago

The fact that Intel and AMD apparently don't prioritize integer division could suggest that their profiling suggests it's not worth it, but with Apple's transistor budget at the moment they can afford it. Also keep in mind that this Xeon might not be really made for number crunching (not really sure)?

masklinn 5y ago
> The fact that Intel and AMD apparently don't prioritize integer division could suggest that their profiling suggests it's not worth it, but with Apple's transistor budget at the moment they can afford it. An other possibility is that Apple has a very different profiling base e.g. iOS applications, whereas Intel and AMD would have more artificial workloads, or be bound by workloads / profiles from scientific computing or the like (video games)?
criddell 5y ago
Would it be fair to characterize the M1 as being made for number crunching?
- mhh__ 5y ago
  Refining number crunching to mean single threaded performance I would say yes, or at least definitely more so than the Intel chip
- gameswithgo 5y ago
  Not any more than an intel/amd/etc cpu is. Like that XEON cpu is gonna crunch more numbers, just due to more cores.
  - mhh__ 5y ago
    If it was intended to be used in the cloud for example it's going to be doing more work but probably designed around a memory-bound load rather than integer throughput.
pbsd 5y ago
Intel greatly improved their divider implementation between Skylake and Icelake. The measurements in the OP are on Skylake-SP, prior to these improvements.
rodgerd 5y ago
Apple's chip designers have the advantage, I assume, of being able to wander down a hallway and ask what the telemetry from iOS and MacOS devices are telling them about real-world use.
- mhh__ 5y ago
  Most of Intel's volume is probably shipped to customers who either don't care or buy a lot of CPUs in one go, so the advantage of this probably isn't quite as apparent as you'd imagine. What can definitely play a role is that (I don't think it's as much of a problem these days, but it definitely has been in the past) is the standard "benchmark" suites that chipmakers can beat each other over the head with e.g. I think it was Itanium that had a bunch of integer functional units mainly for the purpose of getting better SPEC numbers rather than working on the things that actually make programs fast (MEMORY) - I was maybe 1 or 2 when this chip came out, so this is nth-hand gossip, however.
  - anfilt 5y ago
    The Itantium did care about memory its one of the reasons they had massive caches compared to other archs of the era.
- twoodfin 5y ago
  They also have the advantage of controlling the compiler used by effectively the entire development ecosystem. And have groomed that ecosystem through carrots and sticks to upgrade their compilers and rebuild their applications regularly (or use bitcode!) A compiler code generator that knows about a hypothetical two divide units (or just a much more efficient single unit) could be much more effective statically scheduling around them. I’d guess that the bulk of the software running on the highest margin Intel Xeons was compiled some years ago and tuned for microarchitectures even older.
  - mhh__ 5y ago
    >A compiler code generator that knows about a hypothetical two divide units (or just a much more efficient single unit) could be much more effective statically scheduling around them. I'm still completely blind to how they are actually used but GCC and LLVM both have pretty good internal representations of the microarchitecture they are compiling for. If I ever work it out I'll write a blog post about it, but this is an area where GCC and LLVM are both equally impenetrable.

buildbot 5y ago

I think you got the point most people miss - apple had a unique ability to profile every app on the Mac and iOS App Store, possibly in an automated way, as part of the app submission pipeline. Intel and AMD could go out and profile real work applications, and I’m sure they do, but to get to the same level of breadth is probably not possible.

pvg 5y ago
Is that really such a unique advantage for Apple? Intel and AMD can work with Microsoft to achieve something similar, for instance.
- bch 5y ago
  I’m only speculating, but I’d think they wouldn’t even “have to work with” anybody - couldn’t they just instrument whatever they want?
  - pvg 5y ago
    Sure, I just meant there's a Microsoft store so if the there was something particularly advantageous to having access to an app store, Intel or AMD are not completely in the wilderness here.
    - zsmi 5y ago
      Not completely in the wilderness sure. But they also have many operating systems to consider. The other obvious ones are Linux, BSD, and, until very recently, macOS. Are you sure it's possible to do a micro-optimization for one without nerfing the others?
      - ygra 5y ago
        Does the actual machine code doing computations really differ that much based on the OS? I'd guess the differences are just that you call different libraries and maybe a different calling convention, but nothing that would impact code running single-threaded without preemptions much.

CalChris 5y ago

AnandTech says there is one separate integer division unit in the A14 Firestorm which is what the M1 is based on. The article even mentions an increase in latency for integer division for the A14. https://www.anandtech.com/show/16226/apple-silicon-m1-a14-deep-dive/2 https://www.anandtech.com/show/16226/apple-silicon-m1-a14-de... https://images.anandtech.com/doci/16226/Firestorm_575px.png https://images.anandtech.com/doci/16226/Firestorm_575px.png

hctaw 5y ago

I'd quibble with whether folks understand the performance implications. I think most people know division is expensive. It's just non trivial to remove it. If it was that easy to replace, it would be done in hardware and division wouldn't be expensive anymore.

blt 5y ago

I thought "division is slow" was one of the more widely-known performance heuristics. I genuinely wonder how much of the division load in typical programs is an essential part of the problem vs. programmers not knowing/caring about slow division.

rwallace 5y ago

> When I've been micro-optimizing performance-critical code, integer division shows up as a hot spot regularly. Now I'm really curious. In my experience, integer division practically never happens (except by powers of 2 that are easily optimized down to a shift), to the point where I was frowning in puzzlement about why Apple spent resources optimizing it. How is the code you are looking at, bottlenecking on integer division? What on earth is it doing, to make that a frequent operation?