21 ms·
Zen4's AVX512 Teardown
- causi 4y agoExciting stuff. AVX512 isn't just for specialized work projects. It's also a huge performance boost for game console emulation.
- stagger87 4y agoThat sounds like a specialized project :)
- marginalia_nu 4y agoAny project becomes specialized if you work long enough on it.
- mmastrac 4y agoWhen I was doing some work on Dolphin's JIT, AVX implementations were always back of mind. It's a massive tradeoff in so many cases but having access to these is amazing.
- dtech 4y agoInteresting! Any reason why they're specifically good for emulation?
- causi 4y agoI don't have a deep understanding of the implementation, but it gets you a 30% performance boost in Playstation 3 emulation. https://www.tomshardware.com/news/ps3-emulator-avx-512-30-percent-performance-boost https://www.tomshardware.com/news/ps3-emulator-avx-512-30-pe...
- xani_ 4y agohttps://whatcookie.github.io/posts/why-is-avx-512-useful-for-rpcs3/ https://whatcookie.github.io/posts/why-is-avx-512-useful-for...
- dragontamer 4y agoExcellent Teardown by "Mysticial" from mersenneforum.org. Cliffnotes: * Zen4 AVX512 is mostly double-pumped: a 256-bit native hardware that processes two halves of the 512-bit register. * No throttling observed * 512-bit shuffle pipeline (!!). A powerful exception to the "double-pumping" found in most other AVX512 instructions. * AMD seemingly handles the AVX512 mask registers better than Intel. * Gather/Scatter slow on AMD's Zen4 implementation. * Intel's 512-bit native load/store unit has clear advantages over AMD's 256-bit load-store unit when reading/writing to L1 cache and beyond.
- daniel-cussen 4y agoShuffle is the SIMD's killer app. It's apparently an interesting but expensive circuit, but it's smart to prioritize it. Absolute best instruction, hands down. So double-pumping yes isn't full speed meaning single cycle, but that increases the compatibility with AVX512 code. I guess if a program executed itself as a function of its runtime from CPUID it might not, and of course there's all kinds of...but for pedestrian purposes, meaning everything on github, it's a step. Hey 40% speedup on Cinebench, that's buen.
- giyanani 4y agoWhy do you say shuffle is “SIMD’s killer app”? I’ve only dabbled in vector instructions from a learning perspective, and seen others mention it’s important too, but have yet to understand why.
- Veliladon 4y agoBecause you can do things using bitmasks and single instructions instead of brute forcing using multiple instructions. Let's say you have a whole heap of 8-bit numbers you want to multiply by 2 and you have a set of 256-bit registers and a nice SIMD multiply command. If you don't have a shuffle you need to assemble your series of 2s for the second operand for each lane before you can even start. This is going to take hundreds of instructions and hundreds of clocks. Shuffle means you load up lane 0 with the "2" and then splat the contents of lane 0 across the other 31 lanes in two instructions and a few clocks using the shuffle unit. N.B. Shuffle isn't just about splatting. There's a whole heap of different operations it can do that are useful. I just picked a simple example with an obvious massive performance increase for illustrative purposes.
- bufo 4y agoThe BF16 and VNNI instructions are finally going to make AMD competitive for neural network inference.
- freemint 4y agoCompetitive with what? GPUs? Only for networks that fit in a cache and even then only in latency not in throughput. Still valuable but not competitive in the majority of use cases.
- formerly_proven 4y ago> For all practical purposes, under a suitable all-core load, the 7950X will be running at 95C Tj.Max all the time. If you throw a bigger cooler on it, it will boost to higher clock speeds to get right back to Tj.Max. Because of this, the performance of the chip is dependent on the cooling. And it is basically impossible to hit the 230W power limit at stock without direct die or sub-ambient. > If 95C sounds scary, wait to you see the voltages involved. AMD advertises 5.7 GHz. In reality, a slightly higher value of 5.75 GHz seems to be the norm - often across half the cores simultaneously. So it's not just a single core load. The Fmax is 5.85 GHz, but I have never personally seen it go above 5.75. 5.75 GHz is reached with 1.5 V Vcore. The +50 MHz bump over advertised boost clocks was also present in Zen 3, likely in response to the poor reception of Zen 2 behavior, which would usually fail to achieve the advertised clocks.
- pella 4y agophoronix:AMD Zen 4 AVX-512 Performance Analysis On The Ryzen 9 7950X https://www.phoronix.com/review/amd-zen4-avx512 https://www.phoronix.com/review/amd-zen4-avx512 "On average for the tested AVX-512 workloads, making use of the AVX-512 instructions led to around 59% higher performance compared to when artificially limiting the Ryzen 9 7950X to AVX2 / no-AVX512. From these results I am rather impressed by the AVX-512 performance out of the AMD Ryzen 9 7950X. While initially being disappointed when hearing of their "double pumping" approach rather than going for a 512-bit data path, these benchmark results speak for themselves. For software that can effectively make use of AVX-512 (and compiled so), there is significant performance uplift to enjoy while no negative impact in terms of reduced CPU clock speeds / higher power consumption (with oneDNN being one of the only exceptions seen so far in terms of higher power draw). AVX-512 is looking good on the Ryzen 7000 series and I'll continue running more benchmarks over the weeks ahead. These AVX-512 results make me all the more excited for AMD EPYC "Genoa" where AVX-512 can be a lot more widely-used among HPC/server workloads. "
- phire 4y agoI wonder how much of that 59% gain comes from the 512bit registers/instructions themselves, and how much comes from the new instructions and modes that come with AVX-512, and can still be used with the narrower 256bit and 128bit registers. Would be interesting to modify some of the benchmarks to be limited to 256bit AVX-512 and see how they compare.
- TinkersW 4y agoMysticals report indicates much of it does come from wider instructions, because it can saturate the core easier. Zen 3 was front end bottlenecked, so on Zen4 running AVX512 it can more often hit 4x256. The new instructions are useful and some help perf, but mostly only for pretty specialized stuff. Masking is nice but I think people really exaggerate the improvement from it, vblend was only 2 cycles.
- dis-sys 4y agoAMD EPYC Genoa will be a killing machine, with almost 1TBytes/sec memory bandwidth and this avx512 extension... good luck for intel's xeon.
- sekh60 4y agoI've updated most of my home lab to AMD EPYC Rome processors. Really can't beat the core counts for private cloud and the price is amazing compared to Intel. Looking forward to Genoa myself, though moving past Rome will be a ways away for my lab.
- sliken 4y agoI'm hearing 12 channels @ 5200 MT/sec or so. Sounds like 500GB/sec, not 1TB/sec. Oh maybe you meant in a dual socket config?
- xani_ 4y agoGenuinely good luck, it's never good when there is no competition
- thejosh 4y agoYes. There needs to be a competition or we'll go back to the days of Intel just stalling. It's why I want Intel Arc to be decent, there needs to be more players in the GPU space.
- mastax 4y ago> And it is basically impossible to hit the 230W power limit at stock without direct die or sub-ambient. Almost, but not quite. In GamersNexus' review they recorded 250.8W measured at the EPS12V cables, while using an Arctic Cooling Liquid Freezer II 360mm AIO with the fans at 100%. At 230W/1.5V=153A a good VRM will generate about 17W of heat. That leaves you a few watts for board power plane and socket resistive losses (I don't have an estimate for that). Not a very practical cooling solution for a day-to-day workstation, but I do wonder if you could reduce the fan speeds a bit while still maxing out the power limit.
- philjohn 4y agoThen again, I have 2 420mm Black Ice Nemesis radiators in my custom loop - even at relatively low speeds it can keep the 5800X in there and 3080 Ti cool under constant high loads.
- pclmulqdq 4y agoMy mini-ITX work desktop has no problems with a 5900x and a Radeon VII pro running Rocm work, using only a tiny heatsink on the 5900x (and some high-airflow fans, but nothing too incredibly loud). It doesn't thermal throttle, but tops out around 80-90 degrees C. The 7000-series seems to be a different story: you really need a big cooler for those chips.
- adrian_b 4y agoI have the same CPU + GPU combination, but used on an ATX MB with a Noctua cooler with a double 120-mm fan. While the larger case and cooler makes the cooling easier, the fans are normally inaudible and the CPU stays under 45 Celsius degrees when not doing heavy work, and the temperature may raise up to a little over 60 degrees Celsius when 100% busy. From what I have seen until now, cooling will no longer be so easy for the 7000 series, unless you choose to run them in the Eco mode.
- flumpcakes 4y agoWhat computer case/chasis are you using that fits those radiators? Was it an easy build?
- asbeb 4y ago
- asbeb 4y ago
- smat 4y agoVery interesting read. The author notes that double pumping the 512 bit instructions to 256 bit execution units appears to be a good trade-off. As far as I understood ARMs new SIMD instruction set is able to map to execution units of arbitrary width. So it sounds to me like ARM is ahead of x86 in flexibility here and might be able to profit in the future. Maybe somebody with more in-depth knowledge could respond whether my understanding is correct.
- adrian_b 4y agoWith any traditional ISA with wide registers and instructions, a.k.a. SIMD instructions, it is possible to implement the execution units with any width desired, regardless which is the architectural register and instruction width. Obviously, it only makes sense for the width of the execution units to be a divisor of the architectural width, otherwise they would not be used efficiently. Thus it is possible to choose various compromises between the cost and the performance of the execution units. However, if the ISA specifies e.g. 32 512-bit registers, then even the cheapest implementation must include at least that amount of physical registers, even if the execution units may be much narrower. What is new in the ARM SVE/SVE2 and which gives the name "Scalable" to that vector extension, is that here the register width is not fixed by the ISA, but it may be different between implementations. Thus a cheap smartphone CPU may have 128-bit registers, while an expensive server CPU for scientific computation applications might have 1024-bit registers. With SVE/SVE2, it is possible to write a program without knowing which will be the width of the registers on the target CPU. Nevertheless, the scalability feature is not perfect, thus some programs may still be made faster if a certain register width is assumed before compilation, which may make them run slower than possible on a CPU that in fact has wider registers than assumed.
- bee_rider 4y agoARM's SVE is definitely interesting, but I do wonder if it is slowly be honing in on CRAY style vector processing. Which is definitely a cool idea, but a little different from the now-popular fixed-width SIMD. I don't know that it makes sense to call one ahead of the other yet -- ARM's documentation is clear that SVE2 doesn't replace NEON. "Mostly scalar but let's sprinkle in some SIMD" coding will probably always be with us (until ML somehow turns all programs into dot products I guess!) RISC-V also has a variable length vector extension.
- boundchecked 4y agoNot many people realize is that recent glibc brought AVX-512 optimized str* and mem* functions to the ifunc dispatch table, your C code may have been using fancy mask registers on someone's Intel laptop!
- fefe23 4y agoI find the vpmullq part the most stunning. This instruction is used in some bignum code, for example if you are implementing RSA. Yet AMD implemented it three times faster than Intel. I'm also fascinated by AMD now making AVX512 worthwhile on consumer devices (where they would until quite recently artificially slow down Intel CPUs that had it), which presumably will lead to widespread adoption where it matters. Intels strategy of turning off AVX512 in the recent consumer devices because their energy efficiency cores don't have it may turn out to be a monumental mistake.
- ComputerGuru 4y agoNo one is going to be able to seriously use and support AVX512 (or be sufficiently motivated to implement support for it in their libraries and especially applications) until Intel finally gets its act together with regards to AVX512 and decides it actually wants to commit to it being a thing. The AVX2 rollout was (comparatively) flawless. The gains AVX512 brings over AVX2 are, for most people w/ specialty libs excluded, not worth dealing with the terrible CPU support. And Intel just keeps making the situation worse, taking one step forward and two back.
- jackmott42 4y agoImagine next gen consoles, suppose they stick with AMD. Then every game studio and game engine studio is going to love flinging some AVX-512 around. Developers will get more experience with it, any game that runs on PC and Console is going to look slow on PC if you have intel cpus with bad support. More libraries and tools will get created that people will want to use. Adoption could accelerate quick!
- kllrnohj 4y agoNext-next gen consoles are probably still a good 5+ years away. AVX-512 for consumers will either have already become "a thing" or it'll be dead & buried by then.
- 4y ago
- loser777 4y agoI'm genuinely curious of the details of how the 1.5v vCore measurement was obtained. CPU-Z and software measurements in general don't have the greatest reputation of being accurate, especially with just-released generations of CPUs. Conventional wisdom has been with newer manufacturing processes, less voltage is required (and tolerated), and 1.5v vCore sounds truly insane in 2022 for a "4nm" chip. For reference, I haven't heard of 1.5v being a safe "24/7" voltage since the days of 90nm-130nm+ CPUs circa 2005-2006. IIRC casual overclockers in the forums weren't really comfortable with 1.5v even with 65nm Core 2, and this was back when it was common to e.g., safely overclock your 2.4 GHz Core 2 Quad to 3.4 GHz.
- xani_ 4y agoProbably used same registers as previous. Would be simple to confirm with some scope probing CPU power.
- magila 4y agoThe problem is the CPU itself isn't the one measuring voltage, it gets that information from the motherboard's VRM controller. The accuracy of the reported value can vary depending on the controller, how it's configured by the motherboard's firmware, and the physical circuit design. That being said, with new motherboards generally using fully digital VRM controllers the reported value should be pretty close in most cases.
- userbinator 4y ago1.5V was the normal operating voltage of the 130nm "Northwood" Pentium 4s. For 65nm processors it's usually around 1.3-1.4. The problem with high voltages is that while the CPU may appear to work fine for a while, it may start becoming unstable and then suddenly die. Part of me wonders whether the CPU manufacturers decided to, after a very long period of being conservative with lifetime and seeing their products last too long to their liking combined with the diminishing increases in performance with each new model, go all-out with voltages that are guaranteed to cause failure as long as they could make almost all of them happen just outside of warranty; not unlike what the LED lighting market has done.
- paulmd 4y agoHaha, as someone who has been shouting "no, really, AVX-512 is good, even if it's double-pumped, just wait for it guys" into the void for years now, glad to see it finally hit the desktop for real and that the AVX people are already leaning into it. Years and years of "nobody needs AVX-512" and "linus says it's just for benchmarks, he worked at transmeta two decades ago, he knows better than Lisa Su" hot takes down the tubes ;)
- phrygian 4y ago(Tiit
- ncmncm 4y agoThis reminds me of the Z-80, which replaced the 8080's 8-bit ALU with a 4-bit unit. Evidently FF recognized that it had no need to produce an answer in one cycle, because it would not be needed so soon, so why waste logic trying? At the time, a 4MHz Z-80 ran about as fast as a 1MHz 6502, so that suggests opposite ends of that tradeoff range. 6502 had to do all its work in a cycle, but cycles were longer, so there was more time to do it in. Z-80 got half the work done, then the other half, with two more cycles left over for whatever. This is what AVX2 used to do, too, working on 128 bits at a time. Probably when 3nm comes along, they will make it 512 bits wide if they can't think of and justify any better way use up their area budget allocation.
- Michal1277 4y ago
- ChoGGi 4y agoHe posted benchmarks on a separate thread: https://www.mersenneforum.org/showthread.php?t=28107 https://www.mersenneforum.org/showthread.php?t=28107