3 ms·
Somehow, similarly bad numbers were achieved with 56 cores working with disjoint parts of 512 GB array. I didnt pin RAM to cores, though. Maybe thats the missin
by ashvardanian 5y ago
Somehow, similarly bad numbers were achieved with 56 cores working with disjoint parts of 512 GB array. I didnt pin RAM to cores, though. Maybe thats the missing piece, but it’s sad if thats the only way to meet the spec. Almost no application can do that in modern cloud deployment realities.
- lostmsu 5y agoOn the contrary. Having multiple independent services running on a single physical machine will likely give you full performance. The easiest way is probably just splitting the physical machine into the number of VMs/pods equal or larger than CCX count.
- dragontamer 5y agohttps://people.freebsd.org/~gallatin/talks/euro2019.pdf https://people.freebsd.org/~gallatin/talks/euro2019.pdf Here's a talk about a lot of the NUMA-issues brought up by Netflix + FreeBSD. Its 200 Gb/s (bits), not bytes, but I'd expect you to be running into a lot of the similar problems. Its 200 Gb/s because they're reading from SSDs. But still, careful attention to NUMA was the only way they could reach full PCIe-bandwidths, which is a similar issue to yours (where NUMA is needed to reach full Memory/DRAM bandwidth). -------- Did you set your BIOS into NUMA mode? If you're in default BIOS-settings, your RAM is being split up between all 4 nodes (Uniform-memory-access mode). At least, that's how it was for my Threadripper 1950x (I know you've got a totally different model, but who knows?) I forgot exactly the name of the BIOS setting. But you gotta boot your computer up with 4-nodes-per-socket (or was it 8-nodes per socket for your computer?) or some similar setting. There were some programs on Linux (and Windows) that help you see what the OS is seeing.
- kristjansson 5y agoSee also the a more recent talk[0] on progress made on the same topic. On EPYC too, so perhaps more applicable to OP? [0]: https://people.freebsd.org/~gallatin/talks/euro2021.pdf https://people.freebsd.org/~gallatin/talks/euro2021.pdf
- dragontamer 5y agoThe original talk was on 2019 era hardware, for a 2019-era talk. Xeon and EPYC are both discussed, albeit somewhat dated now but still relevant IMO. The newer talk is 2021 hardware. IMO, the newer talk seems to focus more on the specific details of kTLS and PCIe TLS... but there's still stuff in there about NUMA for sure. But yeah, the 2021 talk definitely is on more recent hardware: 2021 Xeons and EPYCs.
- kristjansson 5y agoAh, dang, that's on me, I forgot the earlier talk discussed EPYC chips too.
- bradfa 5y agoAre you thinking of "NUMA per socket" setting in the BIOS? Dell has a short overview of this focused on Epyc 7000 series: https://downloads.dell.com/manuals/common/dell-emc-dfd-numa-amd-epyc-2ndgen.pdf https://downloads.dell.com/manuals/common/dell-emc-dfd-numa-...
- dragontamer 5y agoThat sounds like the one. Different motherboards call these settings different names in their BIOS programs / setup however. So just keep an eye out for it.
- dragontamer 5y ago> Somehow, similarly bad numbers were achieved with 56 cores working with disjoint parts of 512 GB array Oh, that's not NUMA at all, now that I'm more carefully reading your post. NUMA would involve a "copy" step, ensuring that those elements are in NUMA-local memory before reading. Much like how in GPU-programming, you have to worry about the physicality of memory, NUMA-aware programming you have to memcpy data to the right location before it achieves high speeds. Each of the 56-cores needs its ~10GBs in "NUMA-local" memory _BEFORE_ you start the benchmark. Yeah, I realize this isn't practical. But... who ever said that NUMA use cases are practical? Lol. A lot of cases, it makes more sense to just take advantage of infinity fabric for simplicity (although its slower, its definitely more convenient).
- toast0 5y agoNUMA is part of the reality. There's lots of modern computing that doesn't acknowledge reality, but that doesn't tend to do good things for performance. Communication isn't cheap, and implicit communication makes it hard to tell what's expensive. Reading from a pointer could be from L1 cache, or it could be from a pci-e card attached to another socket. The code looks the same, but the cost is radically different. Organizing how your code operates in memory to avoid cross-numa and cross-cpu in general isn't always easy, but it's required if you want to get the performance you paid for. OTOH, smaller nodes get this for free: my dual core desktop 'servers' don't have to worry about NUMA and don't have a lot of cross-cpu contention either.
- dragontamer 5y ago> Reading from a pointer could be from L1 cache, or it could be from a pci-e card attached to another socket. The fun one is TLB (translation lookaside buffers) and the virtual memory system. Today's AMD core's have more L3 cache than what the TLB can handle with 4k-pages. You need to enable 2MB hugepages or 1GB hugepages to even access L3 cache at full speeds in practice... EDIT: Milan-X has 96MB L3 cache per CCX. 4kB-pages would require 24,000 (24-thousand) TLB-entries. IIRC, Milan only has 2000-TLB-entries. Hurraaahhhhhh.... ------ CPUs are devilishly complicated. It makes optimization "fun". Apparently, running "memcpy" requires Ph.D levels of study before you can "memcpy" at full speeds these days.