3 ms·

> So, what this will end up doing is just increase the number of context switches, possibly also adding a performance degradation. I think this is a jumping to

> So, what this will end up doing is just increase the number of context switches, possibly also adding a performance degradation. I think this is a jumping to conclusions a bit. You’d need to be reading /proc/cpuinfo in a fairly tight loop in order for the time spend doing any context switching to dominate the runtime cost of a program. Is this a common practice in some types of computation? Or am I misunderstanding this quote from the article?

EE84M3i 6y ago
The article is saying if you make too many processes (more than the number of hardware threads available) the OS will have to context switch between them to keep them all running.
kcbanner 6y ago
They mean that you read it once, then spawn more threads than you have actual cores. Those threads then need to context switch back and forth to do work.
jasontedor 6y ago
I think that the full context of the sentence you quoted will help: > In this case, if you base your number of threads off grepping lscpu you take another dependency (on the util-linux package), which isn’t needed. You also get the wrong answer, as you do by grepping /proc/cpuinfo. So, what this will end up doing is just increase the number of context switches, possibly also adding a performance degradation. The idea is that a common way to size an application that exploits concurrency is to base the number of application threads on the hardware threads available. This portion of the article is saying that if you do so on the basis of `lscpu` or grepping `/proc/cpuinfo` you could mis-size relative to the actual number of cores available to schedule the threads of your application on. Doing so will lead to excessive context switching for your application, and that can hurt.
stewartesmith 6y ago
I wasn't talking about the reading of the number of CPUs you could use, but rather how many threads you'd spawn based on that information. There may be 10 CPUs in the machine, but if your in your container which is restricted to using only 1 of them, it's no use launching 10 threads in that container over just having one.
- catlifeonmars 6y ago
  Oh I see. My mistake. This makes more sense.

EE84M3i 6y ago

The article is saying if you make too many processes (more than the number of hardware threads available) the OS will have to context switch between them to keep them all running.

kcbanner 6y ago

They mean that you read it once, then spawn more threads than you have actual cores. Those threads then need to context switch back and forth to do work.

jasontedor 6y ago

I think that the full context of the sentence you quoted will help: > In this case, if you base your number of threads off grepping lscpu you take another dependency (on the util-linux package), which isn’t needed. You also get the wrong answer, as you do by grepping /proc/cpuinfo. So, what this will end up doing is just increase the number of context switches, possibly also adding a performance degradation. The idea is that a common way to size an application that exploits concurrency is to base the number of application threads on the hardware threads available. This portion of the article is saying that if you do so on the basis of `lscpu` or grepping `/proc/cpuinfo` you could mis-size relative to the actual number of cores available to schedule the threads of your application on. Doing so will lead to excessive context switching for your application, and that can hurt.

stewartesmith 6y ago

I wasn't talking about the reading of the number of CPUs you could use, but rather how many threads you'd spawn based on that information. There may be 10 CPUs in the machine, but if your in your container which is restricted to using only 1 of them, it's no use launching 10 threads in that container over just having one.

catlifeonmars 6y ago
Oh I see. My mistake. This makes more sense.