3 ms·

Pinning to hardware threads is most certainly not "always" better with fibers. I can say with certainty after very thorough performance testing across dozens of

Pinning to hardware threads is most certainly not "always" better with fibers. I can say with certainty after very thorough performance testing across dozens of architectures and dozens of workloads. The difference can be dramatic and it's difficult to determine in advance which will.be faster (heuristics are necessary).

deleted 6y ago
[deleted]
chrisseaton 6y ago
> Pinning to hardware threads is most certainly not "always" better with fibers. The article is recommending pinning the worker (N) threads to cores, not pinning fibres (M) to cores. If it makes sense to schedule a fibre on a different core to the last one it ran on (perhaps because the data it needs is already on the other core) then you do that yourself when you pick which N to run it on, but if the N threads weren't pinned how would you keep track of which is the right one to run on to get your fibre onto a particular core? What would you do if you selected the right N for a core... but then it got moved when you went to actually run it? That's why we pin.
- emerged 6y ago
  I was talking about pinning workers, not fibers. Empirically, it is often better for performance to allow the OS to manage the hardware threads. Sometimes its worth pinning to a NUMA node, sometimes to a core, rarely to a cpu. Just reporting our results for many thousands of hours across a wide spectrum of test environments. I can understand it being counterintuitive, believe me it has been a shock. But it's true.
- matheusmoreira 6y ago
  > perhaps because the data it needs is already on the other core I'm curious. How does the scheduler know that?
  - chrisseaton 6y ago
    If your scheduler knows about data dependencies between tasks, such as in a dataflow model, or maybe an actor model, then you can try to run tasks that depend on data produced by a worker on the same core that the producer ran on, to catch data already in that core's cache. For example, read http://www.cs.cmu.edu/~blelloch/papers/locality2000.pdf http://www.cs.cmu.edu/~blelloch/papers/locality2000.pdf.

deleted 6y ago

[deleted]

chrisseaton 6y ago

> Pinning to hardware threads is most certainly not "always" better with fibers. The article is recommending pinning the worker (N) threads to cores, not pinning fibres (M) to cores. If it makes sense to schedule a fibre on a different core to the last one it ran on (perhaps because the data it needs is already on the other core) then you do that yourself when you pick which N to run it on, but if the N threads weren't pinned how would you keep track of which is the right one to run on to get your fibre onto a particular core? What would you do if you selected the right N for a core... but then it got moved when you went to actually run it? That's why we pin.

emerged 6y ago
I was talking about pinning workers, not fibers. Empirically, it is often better for performance to allow the OS to manage the hardware threads. Sometimes its worth pinning to a NUMA node, sometimes to a core, rarely to a cpu. Just reporting our results for many thousands of hours across a wide spectrum of test environments. I can understand it being counterintuitive, believe me it has been a shock. But it's true.
matheusmoreira 6y ago
> perhaps because the data it needs is already on the other core I'm curious. How does the scheduler know that?
- chrisseaton 6y ago
  If your scheduler knows about data dependencies between tasks, such as in a dataflow model, or maybe an actor model, then you can try to run tasks that depend on data produced by a worker on the same core that the producer ran on, to catch data already in that core's cache. For example, read http://www.cs.cmu.edu/~blelloch/papers/locality2000.pdf http://www.cs.cmu.edu/~blelloch/papers/locality2000.pdf.