3 ms·

Run two processes. Process #0 is on HardwareThread#0 through HardwareThread#15 inclusive. These will run faster because of L3 cache (96MB IIRC). Process #1 is

Run two processes. Process #0 is on HardwareThread#0 through HardwareThread#15 inclusive. These will run faster because of L3 cache (96MB IIRC). Process #1 is on HardwareThread#16 through HardwareThread#31. These will run slower because they don't have the cache (only 32MB). Load balancer checks queue-depth of Process#0 vs Process#1 and distributes as appropriate. Done. You probably should be doing this under NUMA-conditions anyway (which will have different memory-access speeds and different PCIe access speeds / asymmetries. NUMA#0 will talk to PCI#0 faster, and if PCI#0 is Ethernet, then NUMA#1/#2/#3 will be penalized), but its no different for asymmetric cores.

wmf 4y ago
Yeah, this gets to the crux of our disagreement. People shouldn't do this IMO and they shouldn't buy NUMA either (or turn it off in the case of Epyc). These optimizations are a waste of time when the alternative is to buy hardware that simply doesn't require them.
- toast0 4y ago
  > they shouldn't buy NUMA either (or turn it off in the case of Epyc). These optimizations are a waste of time when the alternative is to buy hardware that simply doesn't require them. Turning off NUMA reporting doesn't make NUMA go away. Whether you can buy hardware that doesn't need this kind of optimization really depends on your needs and budget. The 7900X3D and 7950X3D don't have a very large niche, IMHO; but if you were thinking of a 7900X for a VM system, and you have some loads that would benefit with more cache and some that wouldn't, it could make sense. Depending on your needs, you might be better served with a 7950X or maybe two 7800X3D systems or ??? there's lots of options. I think if you're going with Epyc, you're needing or wanting lots of cores, lots of memory channels/capacity, and/or lots of pci-e; if you need that in a single machine, you're just going to have to deal with the fallout of NUMA that results, because it's not like you have a choice. Sure, if you can manage to split up your load into multiple small UMA machines, that'll be easier to manage, but then you do have to manage more machines and balancing at different levels. It's all tradeoffs.
- dragontamer 4y ago
  > These optimizations are a waste of time when the alternative is to buy hardware that simply doesn't require them. Its cheaper to own/maintain 50 servers, rather than 100 servers. Running 2-processes per server is just cheaper. You have half the motherboards, your RAM is consolidated, your SSDs / Hard Drives / Ethernet ports are consolidated. This leads to lower power and more efficiency (its easier for threads to migrate over if possible).

wmf 4y ago

Yeah, this gets to the crux of our disagreement. People shouldn't do this IMO and they shouldn't buy NUMA either (or turn it off in the case of Epyc). These optimizations are a waste of time when the alternative is to buy hardware that simply doesn't require them.

toast0 4y ago
> they shouldn't buy NUMA either (or turn it off in the case of Epyc). These optimizations are a waste of time when the alternative is to buy hardware that simply doesn't require them. Turning off NUMA reporting doesn't make NUMA go away. Whether you can buy hardware that doesn't need this kind of optimization really depends on your needs and budget. The 7900X3D and 7950X3D don't have a very large niche, IMHO; but if you were thinking of a 7900X for a VM system, and you have some loads that would benefit with more cache and some that wouldn't, it could make sense. Depending on your needs, you might be better served with a 7950X or maybe two 7800X3D systems or ??? there's lots of options. I think if you're going with Epyc, you're needing or wanting lots of cores, lots of memory channels/capacity, and/or lots of pci-e; if you need that in a single machine, you're just going to have to deal with the fallout of NUMA that results, because it's not like you have a choice. Sure, if you can manage to split up your load into multiple small UMA machines, that'll be easier to manage, but then you do have to manage more machines and balancing at different levels. It's all tradeoffs.
dragontamer 4y ago
> These optimizations are a waste of time when the alternative is to buy hardware that simply doesn't require them. Its cheaper to own/maintain 50 servers, rather than 100 servers. Running 2-processes per server is just cheaper. You have half the motherboards, your RAM is consolidated, your SSDs / Hard Drives / Ethernet ports are consolidated. This leads to lower power and more efficiency (its easier for threads to migrate over if possible).