4 ms·

> For most code, you don't know how fast it "ought to" be, so you don't even know there is a problem One way to build up intuition about this is to start predi

> For most code, you don't know how fast it "ought to" be, so you don't even know there is a problem One way to build up intuition about this is to start predicting in advance how many clock cycles your newly written functions will take to run. It helps when you keep your functions small and focused on a single task. Before benchmarking some code, try to estimate in advance. "How many arithmetic operations, how many memory fetches, how many cache misses, how many branches, etc." Then add those up, multiply by their respective clock cycle cost, divide by the chips frequency and voila. An estimate. Probably not that accurate, but at least you're building a mental model. Now run the code section through Valgrind, and compare what you got right and wrong. If you keep repeating this exercise, you'll get substantially better at predicting code performance ahead of time. The major benefit of that is that you'll tend to write high performance code, even before you optimize or profile it. Like any skill, the key to improvement is fast and accurate feedback cycles combined with repeated practice.

ncmncm 6y ago
Our collective "deal with the devil" is that, with all the various caches and predictors, ordinary code runs much faster than it has any business running, but it is impossible to be certain, up front, of the effects of any particular choice. Doing something clever is as likely to make code slower as it is to make it faster. Still, it is true that practice improves intuition, and a habit of measuring helps in learning to avoid spending extra time to actually make code worse. But microarchitectures have become so complicated and so poorly documented that it amounts to a guessing game to discover why a change had the effect it did. Compiler optimizers are always a few steps behind, often doing something that would once have been clever, but is now foolish. For example, it was once smart to intersperse arithmetic operations between branch instructions to fill pipeline holes, but those can now prevent micro-op fusion and make an innermost loop take two cycles where it could take one, resulting in a 2x penalty overall. Telling Gcc to optimize for Zen2 seems to result in much slower code, on Zen2, if the code had been tuned before. I have recently got a 2x improvement in Quicksort, applied to random input, which on its face should have been impossible for such a mature algorithm. We have learned to avoid pointer chasing, and special-casing, but now a branch over extra work may take much longer than the work skipped, if it is predicted badly. With the end of Dennard Scaling, just waiting doesn't make our programs faster anymore. As Moore's law, which still operates, lets microarchitecture complexity grow without bound, the improvement possible given extra attention increases, but the amount of skill needed to extract it grows too. As it grows, the number of people who can do it falls faster, and skepticism that any particular individual has it grows, particularly around languages that offer less direct control. As it gets harder, the range of applications that merit such extra work and attention narrows. Often, now, in places where better performance is most needed, no one on hand understands how to get it, or even knows it is available.

ncmncm 6y ago

Our collective "deal with the devil" is that, with all the various caches and predictors, ordinary code runs much faster than it has any business running, but it is impossible to be certain, up front, of the effects of any particular choice. Doing something clever is as likely to make code slower as it is to make it faster. Still, it is true that practice improves intuition, and a habit of measuring helps in learning to avoid spending extra time to actually make code worse. But microarchitectures have become so complicated and so poorly documented that it amounts to a guessing game to discover why a change had the effect it did. Compiler optimizers are always a few steps behind, often doing something that would once have been clever, but is now foolish. For example, it was once smart to intersperse arithmetic operations between branch instructions to fill pipeline holes, but those can now prevent micro-op fusion and make an innermost loop take two cycles where it could take one, resulting in a 2x penalty overall. Telling Gcc to optimize for Zen2 seems to result in much slower code, on Zen2, if the code had been tuned before. I have recently got a 2x improvement in Quicksort, applied to random input, which on its face should have been impossible for such a mature algorithm. We have learned to avoid pointer chasing, and special-casing, but now a branch over extra work may take much longer than the work skipped, if it is predicted badly. With the end of Dennard Scaling, just waiting doesn't make our programs faster anymore. As Moore's law, which still operates, lets microarchitecture complexity grow without bound, the improvement possible given extra attention increases, but the amount of skill needed to extract it grows too. As it grows, the number of people who can do it falls faster, and skepticism that any particular individual has it grows, particularly around languages that offer less direct control. As it gets harder, the range of applications that merit such extra work and attention narrows. Often, now, in places where better performance is most needed, no one on hand understands how to get it, or even knows it is available.