2 ms·

When we replaced a venerable Tru64 cluster with Linux about 15 years ago, the head of the department (who was skeptical of linux) came to me with a problem: he

by dekhn 3y ago

When we replaced a venerable Tru64 cluster with Linux about 15 years ago, the head of the department (who was skeptical of linux) came to me with a problem: he had scheduled 9 "use 100% of a core" processes on an 8 core machine and expected each process to get 8/9ths of a core on the machine, with each process proceeding at the same rate. (If I got that math wrong, basically I mean that all processes should receive the same amount of CPU time over an interval, with the expectation that the CPUs will be at 100%). Instead, he found that one of the 9 processes always went slower than all the others. Tru64 did what he expected. I believe at this point, we were running a kernel before the completely fair scheduler, but I've always wondered if there was a right answer here- I don't think UNIX or Linux truly 'guarantee' fully predictable and evenly divded CPU performance when you have n+1 (or n+whatever) processes running on n cores. (also, he used "yes > /dev/null" as the CPU-generating task and I had to spend a lot of time explaining to him that wasn't really 100% cpu)

ralfd 3y ago
> that wasn't really 100% cpu Can you explain?
- wtallis 3y ago
  That's probably referring to the fact that running 'yes' is going to spend virtually all of its time on syscall and context switch overhead, and minimal real work or IO. So conclusions from such a test may not generalize to any kind of realistic workload.
- dekhn 3y ago
  the yes command, writing to /dev/null, is making IO calls, which interfere with predictable scheduling. If you look at the source code for yes, https://github.com/coreutils/coreutils/blob/master/src/yes.c https://github.com/coreutils/coreutils/blob/master/src/yes.c it builds a buffer of output and then writes that in a for loop while (full_write (STDOUT_FILENO, buf, bufused) == bufused) continue; Inside the kernel, /dev/null is handled by doing nothing. So basically when this runs, the majority of the CPU time it consumes is just context switches from process to kernel and back. It consumes 100% of a core on an idle machine, but what you really want for your basic scheduling test is to have a pure CPU hog with no kernel transitions (I've seen variations- increment a register, do some light flops that don't hit memory, or even just a nop loop (which is basically "increment register, branch if less than"). If you get the expected performance on the cpu hog, then you can proceed to the yes hog.

by dekhn 3y ago

ralfd 3y ago

> that wasn't really 100% cpu Can you explain?

wtallis 3y ago
That's probably referring to the fact that running 'yes' is going to spend virtually all of its time on syscall and context switch overhead, and minimal real work or IO. So conclusions from such a test may not generalize to any kind of realistic workload.
dekhn 3y ago
the yes command, writing to /dev/null, is making IO calls, which interfere with predictable scheduling. If you look at the source code for yes, https://github.com/coreutils/coreutils/blob/master/src/yes.c https://github.com/coreutils/coreutils/blob/master/src/yes.c it builds a buffer of output and then writes that in a for loop while (full_write (STDOUT_FILENO, buf, bufused) == bufused) continue; Inside the kernel, /dev/null is handled by doing nothing. So basically when this runs, the majority of the CPU time it consumes is just context switches from process to kernel and back. It consumes 100% of a core on an idle machine, but what you really want for your basic scheduling test is to have a pure CPU hog with no kernel transitions (I've seen variations- increment a register, do some light flops that don't hit memory, or even just a nop loop (which is basically "increment register, branch if less than"). If you get the expected performance on the cpu hog, then you can proceed to the yes hog.