3 ms·

If the scheduler's purpose is "to make typical desktop loads feel better", how do you benchmark that? There is certainly room for improvement in making the Linu

by jws 17y ago

If the scheduler's purpose is "to make typical desktop loads feel better", how do you benchmark that? There is certainly room for improvement in making the Linux desktop feel good, but I don't know how one would get to a number. Perhaps you'd have to resort to double blind comparison tests. (FWIW, my 1.2GHz PPC running Debian performs unacceptably as a desktop machine. There are sometimes >1000ms lags between a keypress and it being reflected on the X11 display with no background load. But I can't say this is because of the scheduler.)

ars 17y ago
> But I can't say this is because of the scheduler. Yes you can. And you answered your own question: latency is exactly how you benchmark this (with a measure of total throughput added in to make sure it's not low).
- scott_s 17y ago
  You can say it might be the scheduler, but it doesn't have to be. It could easily be something wrong with a device driver.
  - jws 17y ago
    That is my other likely theory. The kernel could just be crawling in a hole and staying there for a while.
jacquesm 17y ago
That sounds like a classical example of starvation.
viraptor 17y ago
> how do you benchmark that? By getting statistics from the scheduler itself: how long did the process have to wait before it started executing. Also by looking at global effects: run N mplayers at the same time and look for dropped frames ; run an app that maps similar amount of work to N children and measure the run time ; measure the time needed for compiling [big project] with concurency == number of cpus ; ...
- Tuna-Fish 17y ago
  run an app that maps similar amount of work to N children and measure the run time ; measure the time needed for compiling [big project] with concurency These are both the kind of tasks Kolivas is willing to totally throw under the bus for better worst case latency. Remember, the single goal of the project is to get worst-case latency for all non-cpu-limited processes on the system as low as it possibly can get. Throughput, and even best- and average case latency are considered entirely expendable. This is why the BFS is hard to benchmark.
  - mquander 17y ago
    Actually, Kolivas states in the FAQ that compiling in particular is measured to be significantly faster with BFS: "For years we've been doing our workloads on linux to have more work than we had CPUs because we thought that the "jobservers" were limited in their ability to utilise the CPUs effectively (so we did make -j6 or more on a quad core machine for example). This scheduler proves that the jobservers weren't at fault at all, because make -j4 on a quad core machine with BFS is faster than any choice of job numbers on CFS." [1] [1] http://ck.kolivas.org/patches/bfs/bfs-faq.txt http://ck.kolivas.org/patches/bfs/bfs-faq.txt
  - scott_s 17y ago
    Then measure worst case latency. I don't see what the difficulty is. If you're purporting to improve the performance of something, but you aren't always measuring the impact your changes have on the actual performance of that something, you're working blind.
  - nailer 17y ago
    > Remember, the single goal of the project is to get worst-case latency for all non-cpu-limited processes on the system as low as it possibly can get. Throughput, and even best- and average case latency are considered entirely expendable. I've just been listing to lectures from MIT's introduction to algorithms, where more emphasis is placed on worst-case scenarios than otherwise, the reasoning being that these make guarantees possible. So more power to BFS.
Confusion 17y ago
A number of benchmarks are mentioned in the original thread on LKML discussing this scheduler: http://www.pubbs.net/kernel/200909/76667/ http://www.pubbs.net/kernel/200909/76667/ At some point in the discussion, someone introduces a benchmark that clearly reproduces a problem with the current CFS scheduler. I think that perfectly illustrates what you are looking for (unfortunately, I have to run and can't pinpoint the exact post in this timeframe).
- nailer 17y ago
  Quite an interesting read. Did anyone else fidn it worrying that Ingo Molnar attempted to benchmark a scheduler that aims for interactive response using: * kernel build * pipe * messaging performance * OLTP performance And then wonders why he doesn't get an improvement? One of the other posters puts it perfectly: "Do you know what a normal desktop PC looks like? No, a more realistic question based on what you chose to benchmark to prove your point would be: Do you know what normal people actually do on them?" In answer to the parent poster, see Nikos Chantziaras' posts.

by jws 17y ago

ars 17y ago

> But I can't say this is because of the scheduler. Yes you can. And you answered your own question: latency is exactly how you benchmark this (with a measure of total throughput added in to make sure it's not low).

scott_s 17y ago
You can say it might be the scheduler, but it doesn't have to be. It could easily be something wrong with a device driver.
- jws 17y ago
  That is my other likely theory. The kernel could just be crawling in a hole and staying there for a while.

jacquesm 17y ago

That sounds like a classical example of starvation.

viraptor 17y ago

> how do you benchmark that? By getting statistics from the scheduler itself: how long did the process have to wait before it started executing. Also by looking at global effects: run N mplayers at the same time and look for dropped frames ; run an app that maps similar amount of work to N children and measure the run time ; measure the time needed for compiling [big project] with concurency == number of cpus ; ...

Tuna-Fish 17y ago
run an app that maps similar amount of work to N children and measure the run time ; measure the time needed for compiling [big project] with concurency These are both the kind of tasks Kolivas is willing to totally throw under the bus for better worst case latency. Remember, the single goal of the project is to get worst-case latency for all non-cpu-limited processes on the system as low as it possibly can get. Throughput, and even best- and average case latency are considered entirely expendable. This is why the BFS is hard to benchmark.
- mquander 17y ago
  Actually, Kolivas states in the FAQ that compiling in particular is measured to be significantly faster with BFS: "For years we've been doing our workloads on linux to have more work than we had CPUs because we thought that the "jobservers" were limited in their ability to utilise the CPUs effectively (so we did make -j6 or more on a quad core machine for example). This scheduler proves that the jobservers weren't at fault at all, because make -j4 on a quad core machine with BFS is faster than any choice of job numbers on CFS." [1] [1] http://ck.kolivas.org/patches/bfs/bfs-faq.txt http://ck.kolivas.org/patches/bfs/bfs-faq.txt
- scott_s 17y ago
  Then measure worst case latency. I don't see what the difficulty is. If you're purporting to improve the performance of something, but you aren't always measuring the impact your changes have on the actual performance of that something, you're working blind.
- nailer 17y ago
  > Remember, the single goal of the project is to get worst-case latency for all non-cpu-limited processes on the system as low as it possibly can get. Throughput, and even best- and average case latency are considered entirely expendable. I've just been listing to lectures from MIT's introduction to algorithms, where more emphasis is placed on worst-case scenarios than otherwise, the reasoning being that these make guarantees possible. So more power to BFS.

Confusion 17y ago

A number of benchmarks are mentioned in the original thread on LKML discussing this scheduler: http://www.pubbs.net/kernel/200909/76667/ http://www.pubbs.net/kernel/200909/76667/ At some point in the discussion, someone introduces a benchmark that clearly reproduces a problem with the current CFS scheduler. I think that perfectly illustrates what you are looking for (unfortunately, I have to run and can't pinpoint the exact post in this timeframe).

nailer 17y ago
Quite an interesting read. Did anyone else fidn it worrying that Ingo Molnar attempted to benchmark a scheduler that aims for interactive response using: * kernel build * pipe * messaging performance * OLTP performance And then wonders why he doesn't get an improvement? One of the other posters puts it perfectly: "Do you know what a normal desktop PC looks like? No, a more realistic question based on what you chose to benchmark to prove your point would be: Do you know what normal people actually do on them?" In answer to the parent poster, see Nikos Chantziaras' posts.