3 ms·

I am really curious about what kind of optimizations are enabled when you know the microarchitectural design of various CPUs, but without writing assembly by ha

I am really curious about what kind of optimizations are enabled when you know the microarchitectural design of various CPUs, but without writing assembly by hand. I only know the basics such as optimizing data structure for better cache locality, add some fast paths, manual unrolling, but have no idea about how to work around things like pipeline stalls. It would be really helpful if you can point to some materials about this!

jandrewrogers 3y ago
The basic principle is that a CPU core is a complex distributed system with varying degrees of parallelism, concurrency, and latency as all of these components communicate with each other. The design of this distributed system varies across microarchitectures as do the available features. Analyzing the optimal codegen is similar to analyzing optimal algorithm design in higher level distributed systems with a fixed hardware specification. Compilers are good at finding very local optimizations targeting specific microarchitectures. If you use godbolt and switch out the architecture flags you can see how the codegen changes. However, as with higher level code, the compiler can’t rewrite your data structures or see patterns spread across too much code. As a simple example, modern cores have 4+ concurrent ALUs with different capabilities. Some of those ALUs are usually idle if there are not enough independent instructions or independent instructions are too far away from the current instruction. You can see large performance improvements simply by reorganizing your C code so that the compiler and CPU can see more opportunities to use more ALUs in parallel. Interestingly, a lot of these code changes are trivial no-ops at the code semantics level, but they bring the ALU concurrency opportunity within view of the CPU. There is an active niche community on the Internet that studies how various instruction sequences interact with various microarchitectures. This is probably the best resource because a lot of detail is not well documented by the CPU companies themselves. It requires a fair amount of experimentation to develop an intuition for how code will run on a given CPU at this level of detail.
- protomolecule 3y ago
  "There is an active niche community on the Internet that studies how various instruction sequences interact with various microarchitectures. This is probably the best resource" Could elaborate on that? I mean where do I find this community?

jandrewrogers 3y ago

The basic principle is that a CPU core is a complex distributed system with varying degrees of parallelism, concurrency, and latency as all of these components communicate with each other. The design of this distributed system varies across microarchitectures as do the available features. Analyzing the optimal codegen is similar to analyzing optimal algorithm design in higher level distributed systems with a fixed hardware specification. Compilers are good at finding very local optimizations targeting specific microarchitectures. If you use godbolt and switch out the architecture flags you can see how the codegen changes. However, as with higher level code, the compiler can’t rewrite your data structures or see patterns spread across too much code. As a simple example, modern cores have 4+ concurrent ALUs with different capabilities. Some of those ALUs are usually idle if there are not enough independent instructions or independent instructions are too far away from the current instruction. You can see large performance improvements simply by reorganizing your C code so that the compiler and CPU can see more opportunities to use more ALUs in parallel. Interestingly, a lot of these code changes are trivial no-ops at the code semantics level, but they bring the ALU concurrency opportunity within view of the CPU. There is an active niche community on the Internet that studies how various instruction sequences interact with various microarchitectures. This is probably the best resource because a lot of detail is not well documented by the CPU companies themselves. It requires a fair amount of experimentation to develop an intuition for how code will run on a given CPU at this level of detail.

protomolecule 3y ago
"There is an active niche community on the Internet that studies how various instruction sequences interact with various microarchitectures. This is probably the best resource" Could elaborate on that? I mean where do I find this community?