2 ms·
This processor is terrible for servers because of the asymmetric design.
by wmf 4y ago
This processor is terrible for servers because of the asymmetric design.
- alberth 4y agoCan you link to a better performing server for the same price.
- Arrath 4y agoIt seems poorly suited to a lot of workloads absent some scheduler improvements. Take the work-around for gaming: park the CCD without the 3d cache. Well at that point why not just use a 7800x3d?
- dragontamer 4y agoPark the processes that need the cache on the cache cores. Park the processes that don't need it on the smaller-cores. https://man7.org/linux/man-pages/man1/taskset.1.html https://man7.org/linux/man-pages/man1/taskset.1.html These commands exist in both Linux and Windows for a reason.
- dragontamer 4y agoSurely this is a problem that can be fixed with a bit of Thread Affinity?
- wmf 4y agoMany servers only run one type of workload. Do you want to just use half the cores? That's not cost effective.
- dragontamer 4y agoRun two processes. Process #0 is on HardwareThread#0 through HardwareThread#15 inclusive. These will run faster because of L3 cache (96MB IIRC). Process #1 is on HardwareThread#16 through HardwareThread#31. These will run slower because they don't have the cache (only 32MB). Load balancer checks queue-depth of Process#0 vs Process#1 and distributes as appropriate. Done. You probably should be doing this under NUMA-conditions anyway (which will have different memory-access speeds and different PCIe access speeds / asymmetries. NUMA#0 will talk to PCI#0 faster, and if PCI#0 is Ethernet, then NUMA#1/#2/#3 will be penalized), but its no different for asymmetric cores.
- wmf 4y agoYeah, this gets to the crux of our disagreement. People shouldn't do this IMO and they shouldn't buy NUMA either (or turn it off in the case of Epyc). These optimizations are a waste of time when the alternative is to buy hardware that simply doesn't require them.
- toast0 4y ago> they shouldn't buy NUMA either (or turn it off in the case of Epyc). These optimizations are a waste of time when the alternative is to buy hardware that simply doesn't require them. Turning off NUMA reporting doesn't make NUMA go away. Whether you can buy hardware that doesn't need this kind of optimization really depends on your needs and budget. The 7900X3D and 7950X3D don't have a very large niche, IMHO; but if you were thinking of a 7900X for a VM system, and you have some loads that would benefit with more cache and some that wouldn't, it could make sense. Depending on your needs, you might be better served with a 7950X or maybe two 7800X3D systems or ??? there's lots of options. I think if you're going with Epyc, you're needing or wanting lots of cores, lots of memory channels/capacity, and/or lots of pci-e; if you need that in a single machine, you're just going to have to deal with the fallout of NUMA that results, because it's not like you have a choice. Sure, if you can manage to split up your load into multiple small UMA machines, that'll be easier to manage, but then you do have to manage more machines and balancing at different levels. It's all tradeoffs.
- dragontamer 4y ago> These optimizations are a waste of time when the alternative is to buy hardware that simply doesn't require them. Its cheaper to own/maintain 50 servers, rather than 100 servers. Running 2-processes per server is just cheaper. You have half the motherboards, your RAM is consolidated, your SSDs / Hard Drives / Ethernet ports are consolidated. This leads to lower power and more efficiency (its easier for threads to migrate over if possible).
- sudosysgen 4y agoNot really. If you're going for performance, you're going to be using NUMA aware code, and your compute is going to stay on a single CCD (if you set up your NUMA domain correctly). Then it's up to you to decide if you benefit more from cache or frequency. EDIT: AMD CPUs with nonuniform cache (such as this one), with the right motherboard, support L3 as NUMA, which means that each CCX is exposed as a NUMA domain.
- dragontamer 4y agoI don't think the 7950x is NUMA? But I don't have such a system, nor have I played with the BIOS settings of that chip. But it only has one memory controller, so I doubt there's a NUMA setting. Correct me if I'm wrong of course.
- wmf 4y agoEpyc has a NUMA node per CCX mode but I don't know if Ryzen "BIOS" includes any NUMA support. That also doesn't help if you're running a uniform workload.
- sudosysgen 4y agoThere are consumer motherboards which do expose this setting, so I would expect it to also be available here. There are very few workloads where you will have less per-core performance on a 7950X3D than, say, an EPYC CPU. Even those relatively lower clocks are pretty high by server standards.
- rektide 4y agoThe memory access is through the same IOD (so memory access to main memory is fairly uniform) but there are two different CCD's, two different NUMA zones. Communication between them is slower than communication within the CHIP. And one chip has vastly more L3 cache than the other.
- wmf 4y agoTechnically it's NUCA so treating CCXes as NUMA nodes is kind of misusing the concept. Cluster-aware scheduling might be a better way to handle it: https://www.phoronix.com/news/AMD-Linux-L2-Cluster-Scheduler https://www.phoronix.com/news/AMD-Linux-L2-Cluster-Scheduler
- tedunangst 4y agoI don't think this inevitably follows. It means, for a web server, some requests will be faster than others, but some faster is probably still better than none faster. If you have a very specific need for consistency, then slower is better, but that's rather niche. Do companies like google and Facebook replace 100% of their servers all at once? Or do they upgrade in stages, allowing some requests to be processed faster than others?
- wmf 4y agoFor most workloads the 7950X3D is slower and costs more than the 7950X. The performance variability is just icing on the cake. It's really a gaming processor.
- tedunangst 4y agoOkay, sure, median performance per dollar is a useful metric. I don't think variability alone is a deal breaker.
- cyber_kinetist 4y agoAnd if you really want to buy a gaming CPU then the 7800X3D is basically the same (or better!) performance for much less money.
- karussell 4y agoIMO the predictability, that a certain request for a single server should have the same response time, is very important to find performance regressions. I'm still unsure how others that use Hetzner deal with this: https://news.ycombinator.com/item?id=35165764 https://news.ycombinator.com/item?id=35165764
- justinclift 4y agoHmmm, we tend to use LXC (similar to Docker) to partition dedicated servers into smaller pieces. eg database server, app servers, etc. It's fairly likely the high cache cores would be good for PostgreSQL, while the standard cores would be good for general application servers. I'd be trying these out if we needed new dedicated servers atm. And might do so depending on how things go over time. :)