5 ms·

> Highly efficient assembly showed the least performance drop whereas complex "idiomatic" Swift code showed the greatest performance drop. I wonder what this m

by OskarS 5y ago

> Highly efficient assembly showed the least performance drop whereas complex "idiomatic" Swift code showed the greatest performance drop. I wonder what this means. The efficient assembly probably has fewer instructions that use vector instruction and floating point calculations more, while the "idiomatic" Swift probably has just a larger number of instructions that aren't doing heavy calculation. Does that imply then that the high performance cores does much deeper pipelining, but the the number floating point units or whatever is probably pretty similar across both types?

simondotau 5y ago
My initial guess is that it's because Icestorm CPUs have less L1 and L2 cache, resulting in more frequent cache misses in complex loops. I'm by no means an expert in any of this, so I really have no place hypothesising. Firestorm has 128KB L1 per core and 12MB shared L2. Icestorm has 64KB L1 per core and 4MB shared L2.
- OskarS 5y ago
  Yeah, that makes a ton of sense. Cache misses would be less of a factor in the highly optimized assembly case.
- emn13 5y ago
  I've never written native code for an apple device, but have some low-level experience on other things; my guess would be that idiomatic swift contains safety checks, i.e. predictable branches, but branches nevertheless - and that simpler cores with poor or no branch prediction suffer much more heavily than "smarter" cores from that kind of code. At the end of the day, if all you're doing is basic adding/multiplying, you just don't need a fancy core (and hence: GPGPU, right?) So this microbenchmark result isn't very surprising, although it's nice to see the principles in action.
  - saagarjha 5y ago
    That is correct. Also, the checks have a bit of an unfortunate downside that they need to be emitted for each array access if the loop is unrolled (as it should be here).
- saagarjha 5y ago
  This benchmark is highly likely to fit entirely within the cache even for Icestorm.
- hajile 5y ago
  I doubt the cache is a serious problem. Firestorm has 2x the ports across the board (int, fp, load/store, and simd). To keep those ports busy, it has a massive instruction window (630 entries). Keeping those fed requires large I-cache. x86 instructions are 15-20% more dense than uarch64, but the fastest x86 cores still have only 32kb of I-cache (in fact, AMD went down from 64kb in Zen 1 to 32k in Zen 2/3), so I doubt doubling that is the major limitation here. In this particular micro-benchmark, I doubt even extremely bloated code would get anywhere close to 1k of instructions for the loop let alone 64k. There might be something to say about the size of the dedicated loop cache that many chips have, but that's a completely different animal.
ChrisMarshallNY 5y ago
I know what that means to me (as a practical issue, as opposed to an internal mechanism issue), and I am sort of face-palming. I spent a great deal of time, learning "idiomatic Swift" coding practices. I can do some damn clever HOF stuff (inlined maps and reduces). This tells me that I may be better off, doing "classic C"-style coding, like I did, when I was just getting started with Swift. Sigh...
- OskarS 5y ago
  It's generally true. Both Rust and C++ (and Swift to a smaller extent) are both all about the "zero cost abstractions", but they usually aren't. It's hard to beat the C-style for-loop: they map really closely to the underlying hardware, and modern compilers are starting to become really good at auto-vectorizing them.
  - nicoburns 5y ago
    That's not a general rule. In Rust iterator chains are frequently faster than for loops, and are just as likely to be auto-vectorized (or not at the whim of the compiler) as for-loops.
  - qayxc 5y ago
    > It's hard to beat the C-style for-loop: they map really closely to the underlying hardware, and modern compilers are starting to become really good at auto-vectorizing them. Except with many compilers and depending on the loop, for-loops may optimise much poorer than their C++/Rust iterator based counterparts. That's because modern language constructs can communicate intent much clearer to the compiler than a primitive for-loop can; the compiler has to make fewer assumptions (which can can be wrong). So no, it's not "generally true" at all.
    - deleted 5y ago
      [deleted]
  - Ar-Curunir 5y ago
    That’s not necessarily true; sometimes the higher level abstractions can provide more information to the compiler

by OskarS 5y ago

simondotau 5y ago

My initial guess is that it's because Icestorm CPUs have less L1 and L2 cache, resulting in more frequent cache misses in complex loops. I'm by no means an expert in any of this, so I really have no place hypothesising. Firestorm has 128KB L1 per core and 12MB shared L2. Icestorm has 64KB L1 per core and 4MB shared L2.

OskarS 5y ago
Yeah, that makes a ton of sense. Cache misses would be less of a factor in the highly optimized assembly case.
emn13 5y ago
I've never written native code for an apple device, but have some low-level experience on other things; my guess would be that idiomatic swift contains safety checks, i.e. predictable branches, but branches nevertheless - and that simpler cores with poor or no branch prediction suffer much more heavily than "smarter" cores from that kind of code. At the end of the day, if all you're doing is basic adding/multiplying, you just don't need a fancy core (and hence: GPGPU, right?) So this microbenchmark result isn't very surprising, although it's nice to see the principles in action.
- saagarjha 5y ago
  That is correct. Also, the checks have a bit of an unfortunate downside that they need to be emitted for each array access if the loop is unrolled (as it should be here).
saagarjha 5y ago
This benchmark is highly likely to fit entirely within the cache even for Icestorm.
hajile 5y ago
I doubt the cache is a serious problem. Firestorm has 2x the ports across the board (int, fp, load/store, and simd). To keep those ports busy, it has a massive instruction window (630 entries). Keeping those fed requires large I-cache. x86 instructions are 15-20% more dense than uarch64, but the fastest x86 cores still have only 32kb of I-cache (in fact, AMD went down from 64kb in Zen 1 to 32k in Zen 2/3), so I doubt doubling that is the major limitation here. In this particular micro-benchmark, I doubt even extremely bloated code would get anywhere close to 1k of instructions for the loop let alone 64k. There might be something to say about the size of the dedicated loop cache that many chips have, but that's a completely different animal.

ChrisMarshallNY 5y ago

I know what that means to me (as a practical issue, as opposed to an internal mechanism issue), and I am sort of face-palming. I spent a great deal of time, learning "idiomatic Swift" coding practices. I can do some damn clever HOF stuff (inlined maps and reduces). This tells me that I may be better off, doing "classic C"-style coding, like I did, when I was just getting started with Swift. Sigh...

OskarS 5y ago
It's generally true. Both Rust and C++ (and Swift to a smaller extent) are both all about the "zero cost abstractions", but they usually aren't. It's hard to beat the C-style for-loop: they map really closely to the underlying hardware, and modern compilers are starting to become really good at auto-vectorizing them.
- nicoburns 5y ago
  That's not a general rule. In Rust iterator chains are frequently faster than for loops, and are just as likely to be auto-vectorized (or not at the whim of the compiler) as for-loops.
- qayxc 5y ago
  > It's hard to beat the C-style for-loop: they map really closely to the underlying hardware, and modern compilers are starting to become really good at auto-vectorizing them. Except with many compilers and depending on the loop, for-loops may optimise much poorer than their C++/Rust iterator based counterparts. That's because modern language constructs can communicate intent much clearer to the compiler than a primitive for-loop can; the compiler has to make fewer assumptions (which can can be wrong). So no, it's not "generally true" at all.
  - deleted 5y ago
    [deleted]
- Ar-Curunir 5y ago
  That’s not necessarily true; sometimes the higher level abstractions can provide more information to the compiler