6 ms·
An EPYC NixOS Build Farm
- tmd83 8y agoThis is quite surprising. I assumed they would get an EPYC server with higher core at similar price (due to the cost of the CPU) and get advantage in performance/dollar. I didn't expect EPYC to beat a similar physical core count intel server running at similar core count let alone by this much margin. Am I missing something here? I'm saying this as someone who really wants AMD to succeed and push the ecosystem forward. I just want to make sure that I'm reading the results correctly here. They are effectively getting 35% extra performance 60% of the price that's just beyond impressive.
- boris 8y agoEPYC is compared to a previous generation Xeon with lower TDP (105w vs 155/170w) and slower RAM (2400 vs 2666). The Xeon server is more expensive probably because it has 4x the amount of RAM. Edit: actually, this is a dual Xeon machine so its TDP is slightly higher than EPYC's.
- yjftsjthsd-h 8y agoDoesn't that make it more impressive? The AMD build has a quarter of the RAM and still came out ahead!?
- wereHamster 8y agoUnsurprisingly, the amount of RAM has little effect on a CPU-heavy workload.
- stefan_ 8y agoFor this workload, they could have had 16 GiB of RAM and get pretty much identical results.
- blattimwind 8y ago> EPYC is compared to a previous generation Xeon with lower TDP ... Edit: actually, this is a dual Xeon machine so its TDP is slightly higher than EPYC's. Comparing TDPs between manufacturers is invalid.
- kingbirdy 8y ago> Comparing TDPs between manufacturers is invalid. Why? It's all Watts
- blattimwind 8y ago"TDP" is not a figure resulting from a standardized process. Each manufacturer has a different (and sometimes changing) definition of it. E.g. for Intel it is a supposed power consumption given a certain load at base clock. Meanwhile with AMD it basically encodes the standard thermal parameters (Tamb, Tj(max), Rthjc).
- mrb 8y ago«EPYC is compared to a previous generation Xeon» That's not enough to explain. I highly doubt replacing the 12-core Broadwell Xeon CPUs with a current-generation 12-core Xeon (which would be Skylake) would catch up with the 30-40% performance lead that EPYC displays in their workload. The Zen microarchitecture of the EPYC is quite impressive and it could just very well be a workload that Zen is particularly good at. For example the fact EPYC can pack so many cores on a single socket means all the RAM and all its 64MB of last-level cache are "local" to every core (at worst accesses go through the Infinity Fabric.) By comparison, in Intel's case when one of the Xeon cores need to access RAM or cache on the other socket, accesses have to go through the QPI link between the 2 sockets, and it has higher latencies and worse throughput than AMD EPYC's Infinity Fabric.
- phkamp 8y agoThe EPYC has twice as many memory busses.
- blattimwind 8y agoFalse. The E5-2650 v4 has four memory channels, and with two sockets populated, you get eight memory channels, same as EPYC.
- ksec 8y ago>They are effectively getting 35% extra performance 60% of the price that's just beyond impressive. Lets just ignore the price point for now because the memory capacity are different. But the performance, 24 Core AMD EPYC Zen is ~35% faster than 24 Core Intel Broadwell Xeon under the same frequency? Broadwell isn't even that old because Intel has only launched Skylake Xeon E5 not long ago. And the different between Skylake and Broadwell IPC aren't that much either. the max turbo speed are similar, and the Intel platform had the advantage of two CPU using two heatsink, higher TDP, higher heat dissipation more time for turbo. And most testing has shown Zen to be about Sandybridge IPC performance. Something seems wrong here ( Or may be too good to be true )
- uluyol 8y agoI suspect that the Xeon processors are running at a lower frequency under load compared to the EPYC ones. That's just a hunch though.
- ksec 8y agoIt isn't, because according to the spec sheet its all Core frequency are about the same as AMD.
- praseodym 8y agoMaybe their workloads were I/O bound and the EPYC server has faster SSDs?
- bearjaws 8y agoFor anyone confused by instance types, this is for Packet.net So these two servers are pretty similar, I originally thought they were comparing a Amazon m1.xlarge vs EPYC...
- chris_st 8y agoI'm curious if anyone knows the test tool they're using here, in particular for lines like: $machine->waitForWindow("^Desktop "); It looks incredibly useful. Thanks!
- O_H_E 8y agoWell there seems to be something called nix-shell, but I never heard of it before. It seams pretty useful. The link to plasma tests is provided in the article. https://github.com/NixOS/nixpkgs/blob/49a6964a4250d98644da61f24dcc11ee0b28c4f9/nixos/tests/plasma5.nix https://github.com/NixOS/nixpkgs/blob/49a6964a4250d98644da61...
- chris_st 8y agoThanks!
- ris 8y agoThis isn't nix-shell. nix-shell is a tool for spawning ephemeral software environments composed of nix packages, something a little like python's virtualenvs. Digging a little bit, nixos' tests appear to be written in perl and are executed in a context that has access to a bunch of useful primitives for asserting these sorts of things. https://nixos.org/nixos/manual/#sec-writing-nixos-tests https://nixos.org/nixos/manual/#sec-writing-nixos-tests
- grhmc 8y agoThe tooling used by NixOS is custom to NixOS. You can check out the test directory for many more examples. I'll be writing more soon about how these tests work and how neat they are. They really help NixOS quickly release really high quality updates!
- noneucat 8y agoThe tests are written in Perl and are invoked by Nix's build tools. You can skim the API at https://nixos.org/nixos/manual/#sec-nixos-tests https://nixos.org/nixos/manual/#sec-nixos-tests In this case, the test makes use of both xdotool and Nix's own testing library.
- davidmr 8y agoI wonder if there were NUMA effects at play since the Intel server was dual-socket and the EPYC was not. It's not entirely surprising that a CPU-heavy workload reading from filesystem cache (i.e. memory) could cause a substantial amount of nonlocal reads that the single-socket server wouldn't have to deal with. That said, that's just the first thing that popped into my head to explain the result. Either way, the results here are extremely encouraging!
- wmf 8y agoA "single" EPYC is eight 4-core processors so there are even more NUMA effects there.
- h4b4n3r0 8y agoI think it’s the other way around: four 8-core processors
- blattimwind 8y agoIt's four dice with two groups of cores of four each connected by a callosum. In NUMA-speak, each group is (probably) a node with neighbouring nodes having a lower distance.
- davidmr 8y agoWe're now well out of my knowledge base, but your comment certainly makes things interesting. If they have similar NUMA latencies (big if!), I guess maybe the edge would go to whichever chip had higher NUMA bandwidth? Just sort of thinking out loud... Or perhaps it has absolutely nothing to do with memory speeds, but I can't think of what else would have such a dramatic effect. I'd love to see the same test with a single Xeon.
- slededit 8y agoUnless your buying top of the line Intel parts AMD is going to have more performance at most price points. It's the only logical decision unless you use AVX-512 or your workload is single threaded.
- newnewpdro 8y agoThis is comparing far more than Xeon vs. EPYC, not even a single mention of the storage subsystem on these respective machines. It's not like the tasks being timed are 100% CPU-bound...
- cjhanks 8y agoThey are testing performance of a heavily modified and secure Xen type 1 hypervisor against a native computer with unspecified security policies? This seems like a meaningless measurement.