4 ms·

I feel like we are now living in a world in which figuring about the performance of your code is almost impossible. Between hopspot compilers, bytecode, CPU int

by mruts 7y ago

I feel like we are now living in a world in which figuring about the performance of your code is almost impossible. Between hopspot compilers, bytecode, CPU intrinsics, vectorization, caches, pipelining, GC, branch prediction, etc how is one supposed to reason about his code? In whatever language I'm programming, I usually just imagine what the C would look like and make performance decisions from there. But things are so complicated nowadays that's doesn't even come close to capturing the performance. Is this the only way we can get good performance? Is there an alternative paradigm of optimization in hardware or software that would make performance tradeoffs more explicit? I don't know, but the current state of things makes me uneasy because I feel I never understand what's truly going on (and I definitely don't).

theonemind 7y ago
I don't know. In general, it's advisable to write your code first largely ignoring performance considerations, just do the simplest/most readable thing, then profile it and see where it bottlenecks. So, that's the best general wisdom I know. Don't even bother trying to guess where you'll have performance problems. Profile it.
- civility 7y ago
  This just defers the optimization process by one step. It's hard to improve something without measuring it first, but either you know where your inner loop is or the profiler tells you. Now you've still got to improve that bottleneck somehow and take into account all those complicating factors. I've seen very unexpected results doing this. In one case, I put the inner loop in a separate function to enable swapping implementations and make it simpler to time. Surprisingly, the overall program ran faster without any other changes. Just pulling the inner loop into a function of its own sped things up. My best guess was that the compiler was willing to put more effort into optimization on the smaller function. This contradicts common wisdom about function call overhead, and it also shows another level of complexity in the problem: Some compiler optimizations are exponential in their cost, so they punt if you put too much of your code in one block or function. Also note that I wasn't able to prove this is what happened. It's just my best guess based on what I was able to see and infer.
- mruts 7y ago
  I prefer to keep the approximate performance of my code in my head as I write it to ensure that I'm using appropriate data structures and abstractions since it's often costly to change these things after your profile.
civility 7y ago
Yes, and it's also complicated by the translation to micro-ops with out of order dispatch and instruction retirement. I remember seeing one statement saying this is now a complicated enough system that the number of cycles it takes to execute a block of code is literally chaotic in the sense that small changes to initial conditions cause large changes in the number of cycles to complete. A butterfly effect, possibly triggered by whatever the kernel chose to do a millisecond before your code runs. Throw in changing clock rates based on the temperature of the CPU. If your code efficiently uses the ALUs / cores in the processor, it can get hot and slow itself down.

by mruts 7y ago

theonemind 7y ago

I don't know. In general, it's advisable to write your code first largely ignoring performance considerations, just do the simplest/most readable thing, then profile it and see where it bottlenecks. So, that's the best general wisdom I know. Don't even bother trying to guess where you'll have performance problems. Profile it.

civility 7y ago
This just defers the optimization process by one step. It's hard to improve something without measuring it first, but either you know where your inner loop is or the profiler tells you. Now you've still got to improve that bottleneck somehow and take into account all those complicating factors. I've seen very unexpected results doing this. In one case, I put the inner loop in a separate function to enable swapping implementations and make it simpler to time. Surprisingly, the overall program ran faster without any other changes. Just pulling the inner loop into a function of its own sped things up. My best guess was that the compiler was willing to put more effort into optimization on the smaller function. This contradicts common wisdom about function call overhead, and it also shows another level of complexity in the problem: Some compiler optimizations are exponential in their cost, so they punt if you put too much of your code in one block or function. Also note that I wasn't able to prove this is what happened. It's just my best guess based on what I was able to see and infer.
mruts 7y ago
I prefer to keep the approximate performance of my code in my head as I write it to ensure that I'm using appropriate data structures and abstractions since it's often costly to change these things after your profile.

civility 7y ago

Yes, and it's also complicated by the translation to micro-ops with out of order dispatch and instruction retirement. I remember seeing one statement saying this is now a complicated enough system that the number of cycles it takes to execute a block of code is literally chaotic in the sense that small changes to initial conditions cause large changes in the number of cycles to complete. A butterfly effect, possibly triggered by whatever the kernel chose to do a millisecond before your code runs. Throw in changing clock rates based on the temperature of the CPU. If your code efficiently uses the ALUs / cores in the processor, it can get hot and slow itself down.