4 ms·

linux has pressure stall information[0] which tracks the time where some processes couldn't be run because they were waiting for contended resources (cpu/io/mem

linux has pressure stall information[0] which tracks the time where some processes couldn't be run because they were waiting for contended resources (cpu/io/memory). An N-parallel compute-bound job on an M-thread CPU will stay at nearly 0 CPU pressure if N=M (assuming no background tasks) because they're not stepping on each other's toes. At N>M pressure will start to rise. [0] https://docs.kernel.org/accounting/psi.html https://docs.kernel.org/accounting/psi.html

twoodfin 3y ago
Note that for memory, as I understand the documentation, it's less about "couldn't be run" than "couldn't have the real memory allocation necessary to avoid paging". This is in contrast to another contended resource, memory bandwidth, "waiting" for which manifests at the process level as CPU cycles like any other code. It's possible to use profiling tools to distinguish memory waits from other CPU activity, but as far as I know nobody has built the infrastructure to bubble that data up in some form so it could be tracked systemically by the OS. I can guess why: The memory hierarchy is complicated, and what you can derive about it from CPU performance counters is indirect and limited. Still, even some basic estimation for how many cache lines a process was responsible for driving over the memory bus would be helpful for those building high-performance systems and applications.
- Hello71 3y ago
  PSI is exported by the kernel to track context switches: switch to idle process = io wait, switch to kswapd = mem wait, switch to other runnable process = cpu wait. these are already visible to the kernel, it just needs to increment some counters and expose those to userspace. you can get the perf counters with something like `perf stat -ae cache-misses sleep 60`, it doesn't need to be in /proc. additionally, context switches are the same on everything that can run Linux, whereas PMU counters are highly CPU-specific (potentially even different in each CPU stepping), so given the current state of affairs, a generic interface would be very limited.

twoodfin 3y ago

Note that for memory, as I understand the documentation, it's less about "couldn't be run" than "couldn't have the real memory allocation necessary to avoid paging". This is in contrast to another contended resource, memory bandwidth, "waiting" for which manifests at the process level as CPU cycles like any other code. It's possible to use profiling tools to distinguish memory waits from other CPU activity, but as far as I know nobody has built the infrastructure to bubble that data up in some form so it could be tracked systemically by the OS. I can guess why: The memory hierarchy is complicated, and what you can derive about it from CPU performance counters is indirect and limited. Still, even some basic estimation for how many cache lines a process was responsible for driving over the memory bus would be helpful for those building high-performance systems and applications.

Hello71 3y ago
PSI is exported by the kernel to track context switches: switch to idle process = io wait, switch to kswapd = mem wait, switch to other runnable process = cpu wait. these are already visible to the kernel, it just needs to increment some counters and expose those to userspace. you can get the perf counters with something like `perf stat -ae cache-misses sleep 60`, it doesn't need to be in /proc. additionally, context switches are the same on everything that can run Linux, whereas PMU counters are highly CPU-specific (potentially even different in each CPU stepping), so given the current state of affairs, a generic interface would be very limited.