An interesting tidbit from the Erlang world: We have CPU affinity controls (erl(1) options +sbt {db,tnnps}) which will spread schedulers (Erlang's name for a P)

An interesting tidbit from the Erlang world: We have CPU affinity controls (erl(1) options +sbt {db,tnnps}) which will spread schedulers (Erlang's name for a P) over a NUMA-like topology and keep track of the relative price of handoff/stealing between CPUs. Another option defines if you want load compaction and load balancing (+scl, +sub). Some workloads are more efficient if you can keep the work mostly on a few cores rather than spread the work over all available cores in the system because communication will be faster that way. If one core can process all work and get back to an idling state quickly, it usually beats spreading the load out due to caching. For some workloads and hardware configurations, the gain from doing this is quite high because you can avoid schedulers jumping between cores and in the process destroying cache layers and TLBs. OTOH, when running in a typical cloud environment, you are at the mercy of the underlying hypervisor and thus your load profile is different. In that case, affinity has little to no effect.