4 ms·

May I ask, what do you gain by setting it to +1 cores?

by thret 13y ago

csense 13y ago
Even though CPU is usually the resource that limits throughput for video processing in my experience, each process will presumably do some amount of I/O as well. If you have 8 cores running 8 jobs, then whenever one of those jobs needs to do I/O you have a core sitting idle while that job waits for the disk. If you have 8 cores running 9 jobs, then all your cores will still be fully utilized when a single job is doing I/O. I think you actually want to do plus a small percentage, perhaps 5%, so ceil(1.05*N) jobs on N cores. That is, I have a gut feeling that 64 cores doing 65 jobs would still result in underutilization, 67-68 jobs would be better (provided the workload doesn't become I/O-limited with that much CPU power, and provided the number of videos to be converted is still much larger than the number of cores, so you don't run afoul of Amdahl's law). These are really just rules of thumb based on my gut feelings and mental models of how the system works; it might be fun to actually benchmark it and see if my ideas correspond to reality. (You probably want to reboot first, or a bunch of unrelated I/O, to flush the disk cache.)
- icambron 13y ago
  I'd like to get a better sense of how well hyperthreading works in practice on stuff like this. Is it really the case that my physical cores' pipelines are < 50% utilized by video processing, such that the hyperthreads are equivalent to having a whole other core? Because I'd have guessed that, IO aside, you really want slightly fewer processes than virtual cores.
- nucleardog 13y ago
  I definitely agree with the n+1, however when you're dealing with HyperThreading I don't know that I'd bother. The extra 'cores' provided by HyperThreading are not full cores. The idea behind HyperThreading is basically the same as your n+1 idea. A HyperThreaded core has duplicated circuitry to handle state (registers, etc) but not the execution resources. When one thread is stalled/not using the execution resources (e.g., waiting on disk IO, waiting on a fetch from RAM, etc), the other thread is already in processor ready to go and will be executed instead. Obviously this is a bit of a simplification, but your n+1 idea is essentially already implemented in the hardware. That said, if enough of your cores are still stalled waiting on disk/memory access and are stalled long enough to make the context switch worth it, it may be beneficial. If not, however, you might actually end up seeing some minor slow down as the processor is forced to switch between the competing processes. I'd benchmark it.

May I ask, what do you gain by setting it to +1 cores?

by thret 13y ago

May I ask, what do you gain by setting it to +1 cores?

Even though CPU is usually the resource that limits throughput for video processing in my experience, each process will presumably do some amount of I/O as well. If you have 8 cores running 8 jobs, then whenever one of those jobs needs to do I/O you have a core sitting idle while that job waits for the disk. If you have 8 cores running 9 jobs, then all your cores will still be fully utilized when a single job is doing I/O. I think you actually want to do plus a small percentage, perhaps 5%, so ceil(1.05*N) jobs on N cores. That is, I have a gut feeling that 64 cores doing 65 jobs would still result in underutilization, 67-68 jobs would be better (provided the workload doesn't become I/O-limited with that much CPU power, and provided the number of videos to be converted is still much larger than the number of cores, so you don't run afoul of Amdahl's law). These are really just rules of thumb based on my gut feelings and mental models of how the system works; it might be fun to actually benchmark it and see if my ideas correspond to reality. (You probably want to reboot first, or a bunch of unrelated I/O, to flush the disk cache.)

icambron 13y ago
I'd like to get a better sense of how well hyperthreading works in practice on stuff like this. Is it really the case that my physical cores' pipelines are < 50% utilized by video processing, such that the hyperthreads are equivalent to having a whole other core? Because I'd have guessed that, IO aside, you really want slightly fewer processes than virtual cores.
nucleardog 13y ago
I definitely agree with the n+1, however when you're dealing with HyperThreading I don't know that I'd bother. The extra 'cores' provided by HyperThreading are not full cores. The idea behind HyperThreading is basically the same as your n+1 idea. A HyperThreaded core has duplicated circuitry to handle state (registers, etc) but not the execution resources. When one thread is stalled/not using the execution resources (e.g., waiting on disk IO, waiting on a fetch from RAM, etc), the other thread is already in processor ready to go and will be executed instead. Obviously this is a bit of a simplification, but your n+1 idea is essentially already implemented in the hardware. That said, if enough of your cores are still stalled waiting on disk/memory access and are stalled long enough to make the context switch worth it, it may be beneficial. If not, however, you might actually end up seeing some minor slow down as the processor is forced to switch between the competing processes. I'd benchmark it.