5 ms·

The core waiting for data to be loaded from RAM is busy. Busy waiting for data. Instructions per cycle can also be misleading. Modern cpu's can do multiple shi

by gens 9y ago

The core waiting for data to be loaded from RAM is busy. Busy waiting for data. Instructions per cycle can also be misleading. Modern cpu's can do multiple shifts per cycle, but something like division takes a long time. It all doesn't matter anyway, as instructions per cycle does not tell you anything specific. Use the cpu-builtin performance counters, use perf. It basically works by sampling every once in a while. It (perf, or any other tool that uses performance counters) shows you exactly what instructions are taking up your processes time. (hint: it's usually the ones that read data from memory; so be nice to your caches) It's not rocket surgery.

brendangregg 9y ago
> The core waiting for data to be loaded from RAM is busy. Busy waiting for data. A) it can run another hyperthread with those cycles, and, B) even without hyperthreads, it misleads the developers as to where to tune. C) also, "busy waiting" sounds like an oxymoron. DRAM is busy. The CPU is waiting. Yes, the OS doesn't differentiate and calls it %CPU. > It all doesn't matter anyway, as instructions per cycle does not tell you anything specific. It provides a clue as to whether %CPU is instruction bound or stall bound. > Use the cpu-builtin performance counters, use perf. It basically works by sampling every once in a while. That's called sampling mode. The example in my post used counting mode. Edit: added (C).
- fulafel 9y ago
  I think A is relatively uncommon, as long as mainstream CPUs only have 2 threads. It may give you a boost when you happen to have between N and N*2 runnable threads, for N = number of cores. And those idle CPU threads were already showing up as idle logical processors in top so you would have had a good reason to try to employ them earlier already. (terminology nitpick: HyperThreading is just Intel's proprietary trademark for SMT)
- gens 9y ago
  >A) it can run another hyperthread with those cycles, and, I don't see hyperthreading useful. Neither do the benchmarks, that usually put hypterthreading off ahead of hypterthreading on. Reasoning is simple. When a cache miss happens the (intel) cpu runs another thread. The question is does that happen often enough to justify the overhead of that "hyper"threading. Remember that hyperthreading does not magically make a new cpu core, so there has to be overhead (and the benchmarks prove so). Even if there is no overhead, there is still the other thread filling the L1 (and L2) cache (programs that don't often read from nor write to memory are very rare). Apparently Agner Fog wrote a post about it [0]. Even without taking into account what i wrote above, hypterthreading would not be a big enough of influence on the sampling (i can draw a hypothetical timeline, if you don't get it). >B) even without hyperthreads, it misleads the developers as to where to tune. No, no it doesn't. How would it ? It shows you exactly what instruction is taking up time. If it is a MOV then your data is not in cache, if it is a divss then.. well you have a division in your code, if it is about evenly spread in a pattern then you should look if your code is serial (not that there is usually much help for that), if it is a branch then.. etc. How the F is that less helpful then a blanket "your code is not executing many instructions per tick" statement ? Instructions per tick only tells you if another process is f-ing with yours (probably by hogging memory bandwidth). >C) also, "busy waiting" sounds like an oxymoron. DRAM is busy. The CPU is waiting. Yes, the OS doesn't differentiate and calls it %CPU. Normally it is, but we are not writing poems here. Remember how old cpu's had a busy loop, that ran when idling ? AFAIK even today's cpus have it. >It provides a clue as to whether %CPU is instruction bound or stall bound. If i read a byte from a random place in a gigabyte array then do some shifts on it and store it back, it will still show a (relatively) high number of instructions but the bottleneck will be memory. On the other side if i load a float from a small array then do some math on it (notably division), it will result in a low number of instructions per tick but the memory will not be the bottleneck. So all in all, instructions per tick doesn't tell you much. Maybe if the program suddenly starts to go slow, to more easily check what other program is hogging the memory access. Or a general rule of thumb that makes you think "this code should execute instructions much faster, hmm". As for finding bottlenecks and dealing with them: If your MOV instruction is a bottleneck, you are either dealing with a huge amount of data that can't be accessed in a predictable manner or you didn't think to make your code/data structures friendly to cache[1] or your code is really complicated or just shit (as 90% of code is). Perf also shows you cache misses... Another great thing about perf (actually about objdump) is that it can interweave code and assembly when showing you where your code is slow. So you don't even have to learn assembly. [0] http://www.agner.org/optimize/blog/read.php?i=6 http://www.agner.org/optimize/blog/read.php?i=6 [1] I have a huge 2D array representing a grid. The grid will be sampled 1-5 points up, down, left, and right. The up and down sampling will, more often then not, result in cache misses (for obvious reasons). Solution ? Cut the grid into squares. PS Hyperthreading sucks. I always turn it off. edit: PPS No benchmarks, hypotheticals, deeper research into the causes ?

by gens 9y ago

brendangregg 9y ago

> The core waiting for data to be loaded from RAM is busy. Busy waiting for data. A) it can run another hyperthread with those cycles, and, B) even without hyperthreads, it misleads the developers as to where to tune. C) also, "busy waiting" sounds like an oxymoron. DRAM is busy. The CPU is waiting. Yes, the OS doesn't differentiate and calls it %CPU. > It all doesn't matter anyway, as instructions per cycle does not tell you anything specific. It provides a clue as to whether %CPU is instruction bound or stall bound. > Use the cpu-builtin performance counters, use perf. It basically works by sampling every once in a while. That's called sampling mode. The example in my post used counting mode. Edit: added (C).

fulafel 9y ago
I think A is relatively uncommon, as long as mainstream CPUs only have 2 threads. It may give you a boost when you happen to have between N and N*2 runnable threads, for N = number of cores. And those idle CPU threads were already showing up as idle logical processors in top so you would have had a good reason to try to employ them earlier already. (terminology nitpick: HyperThreading is just Intel's proprietary trademark for SMT)
gens 9y ago
>A) it can run another hyperthread with those cycles, and, I don't see hyperthreading useful. Neither do the benchmarks, that usually put hypterthreading off ahead of hypterthreading on. Reasoning is simple. When a cache miss happens the (intel) cpu runs another thread. The question is does that happen often enough to justify the overhead of that "hyper"threading. Remember that hyperthreading does not magically make a new cpu core, so there has to be overhead (and the benchmarks prove so). Even if there is no overhead, there is still the other thread filling the L1 (and L2) cache (programs that don't often read from nor write to memory are very rare). Apparently Agner Fog wrote a post about it [0]. Even without taking into account what i wrote above, hypterthreading would not be a big enough of influence on the sampling (i can draw a hypothetical timeline, if you don't get it). >B) even without hyperthreads, it misleads the developers as to where to tune. No, no it doesn't. How would it ? It shows you exactly what instruction is taking up time. If it is a MOV then your data is not in cache, if it is a divss then.. well you have a division in your code, if it is about evenly spread in a pattern then you should look if your code is serial (not that there is usually much help for that), if it is a branch then.. etc. How the F is that less helpful then a blanket "your code is not executing many instructions per tick" statement ? Instructions per tick only tells you if another process is f-ing with yours (probably by hogging memory bandwidth). >C) also, "busy waiting" sounds like an oxymoron. DRAM is busy. The CPU is waiting. Yes, the OS doesn't differentiate and calls it %CPU. Normally it is, but we are not writing poems here. Remember how old cpu's had a busy loop, that ran when idling ? AFAIK even today's cpus have it. >It provides a clue as to whether %CPU is instruction bound or stall bound. If i read a byte from a random place in a gigabyte array then do some shifts on it and store it back, it will still show a (relatively) high number of instructions but the bottleneck will be memory. On the other side if i load a float from a small array then do some math on it (notably division), it will result in a low number of instructions per tick but the memory will not be the bottleneck. So all in all, instructions per tick doesn't tell you much. Maybe if the program suddenly starts to go slow, to more easily check what other program is hogging the memory access. Or a general rule of thumb that makes you think "this code should execute instructions much faster, hmm". As for finding bottlenecks and dealing with them: If your MOV instruction is a bottleneck, you are either dealing with a huge amount of data that can't be accessed in a predictable manner or you didn't think to make your code/data structures friendly to cache[1] or your code is really complicated or just shit (as 90% of code is). Perf also shows you cache misses... Another great thing about perf (actually about objdump) is that it can interweave code and assembly when showing you where your code is slow. So you don't even have to learn assembly. [0] http://www.agner.org/optimize/blog/read.php?i=6 http://www.agner.org/optimize/blog/read.php?i=6 [1] I have a huge 2D array representing a grid. The grid will be sampled 1-5 points up, down, left, and right. The up and down sampling will, more often then not, result in cache misses (for obvious reasons). Solution ? Cut the grid into squares. PS Hyperthreading sucks. I always turn it off. edit: PPS No benchmarks, hypotheticals, deeper research into the causes ?