5 ms·

His idea that AVX-512 is something that's exclusively for floating-point is completely off-base, AVX has included integer operations since AVX2. Widely used in

by paulmd 6y ago

His idea that AVX-512 is something that's exclusively for floating-point is completely off-base, AVX has included integer operations since AVX2. Widely used in JIT, database, etc. Furthermore, AVX-512 is about much more than doubling the vector width, it is a significant overhaul to the instruction set and adds many new operations and "fills in gaps" that were missing from previous instruction sets. It in fact would be perfectly valid and good to implement AVX-512 with a 256-bit unit that takes twice as long to run 512-bit width instructions. This completely negates all his points about die space utilization right from the start - AVX-512 support does not imply a significantly larger use of space than previous AVX instructions. This would also fix some of the power-related problems on Skylake-SP - after all if you go from 2 512-bit wide units to 2x256 gangable units or 1x256 running at half-rate, that reduces power correspondingly and you no longer need to drop clocks so strongly to offset this, but you keep the functionality added in AVX-512. Furthermore, it's not like there are massive gains in general IPC that haven't been tapped. AVX-512 has taken 25% of the die area in some instances, if you dropped that to AVX2 (assume 12.5% of die area) then it's not like the processor would be 12.5% faster in general, that would translate to maybe 2-3% faster in general and a 30%+ loss in specialty applications. Once you've mostly explored general-purpose gains and are into diminishing returns territory (which modern processors certainly are), it makes sense to start looking at "specialty units", like AVX, or on GPUs you've got tensor cores and BVH traversal units, and so on. These can provide big speedups in key tasks at the cost of very little "general" performance (since that's already in diminishing returns territory). The biggest thing slowing down AVX-512 adoption has been, yet again, 10nm. Right now it is only available on Skylake-X and Skylake-SP products, and more recently Ice Lake (which came out September of last year, in only the ultrabook segment, supplemented by 14nm in the mobile workstation segment as well as the ultrabook segment). So right now it is available in less than 1% of the desktop "fleet" and probably less than 1/8th of the laptop "fleet". There is very little reason to implement code paths for an instruction set that nobody can execute. Over time, as Ice Lake and Tiger Lake build share of the laptop "fleet", Rocket Lake implements it on desktop, and AMD implements it whenever, it will see more usage, just like prior AVX sets. It really is wider-market than people realize. I have seem many people scoff and say "well you'll never see it used in games or whatever", but for many years now there have been games that simply will not run if you don't have AVX (notably many Ubisoft titles), there is no fallback SSE/scalar codepath. In another 10 years you will probably have AVX-512 mandatory games as well. With all due respect to his long career in software engineering, that doesn't necessarily translate to processor design. This is just one person's opinion and you are under no obligation to accept it as gospel just because it's Torvalds'. See also: his weird ZFS rant. (This seems to be a common thing with software engineers in particular, including many on this site - can't count how many "one weird fix from a software engineer to fix [complex domain problem] in [chemical/materials/aerospace engineering]" I've seen. I of course have no particular expertise in processor design either, but the engineers at Intel presumably do, and they thought it was a good idea.

R0b0t1 6y ago
His experience does translate pretty well to typical computer workload analysis. Realize he has to digest, at least partially, most of the kernel commits. Someone made a comment about special purpose instructions being the future. I think that is true, in part, but those instructions might be better off on an accelerator of some kind, or at least interfaced to the chip in a way that is better than how AVX512 must be used.
- paulmd 6y ago
  > Someone made a comment about special purpose instructions being the future. I think that is true, in part, but those instructions might be better off on an accelerator of some kind, or at least interfaced to the chip in a way that is better than how AVX512 must be used. The problem with the coprocessor model is that it adds an enormous amount of latency. A lot of things aren't worth pushing off to a GPU because it adds so much time, and because there is a limited amount of bandwidth to push it all there and back. Furthermore, the farther you are sending something, the more power you need to do it. Sending it off-chip is significantly less efficient than doing it on-chip, all things equal. RTX is sort of a microcosm of these problems - people really wanted NVIDIA to implement RTX as an off-die "accelerator chip" (and some dumb rumors are even bringing this topic back for the 3000 series despite all the reasons it doesn't work). But it actually makes no sense to separate this - you can't just push it off to the RT core and be done with it, after RT finds collisions you need to go back to shaders and do stuff with it, so it needs to be feasible to flip back and forth between shading, RT, and back to shading. Moving it off die would mean that you now have some kind of async thing or buffer going on and that would kill latency and make it infeasible for real-time shading at least. Furthermore you have cache benefits if everything stays paged in while you are finishing up one tile. Or another example would be Fusion - sure the idea of having coherent memory and triggering the GPU "like an AVX unit" sounds great but it's only unidirectional cache coherency - the GPU reads an up-to-date picture of the CPU but the CPU is not optimized to read out of the GPU, so while it can work in this mode the performance is "singularly poor". This means that "flipping back and forth" does not really work here either, it is structured as a CPU-to-GPU pipeline, not a general purpose coprocessor. With as much as they were pushing Heterogeneous Systems Architecture, presumably they couldn't reasonably implement that without trashing GPU performance, cause if they could I bet they would. https://www.realworldtech.com/fusion-llano/3/ https://www.realworldtech.com/fusion-llano/3/ Integrating specialty units into the core is the tradeoff between latency and performance - it's nowhere near as much performance as a dedicated accelerator, but it's nowhere near as much latency either. Depending on how much work each item needs done, it may be better to do it on-core and keep trucking. And often this is not particularly expensive - things like crypto acceleration instructions are basically uncontroversial. And that is analogous to something like BVH traversal units on NVIDIA, where a ~5% die area expenditure speeds up that segment of the program by 8x or so and results in 2x performance improvements program-wide in some cases. The NVIDIA equivalent to AVX-512 would be tensor cores I suppose - not all that useful to general workloads (apart from DLSS), consuming a fairly significant amount of die area (around 10-15% iirc) but greatly desired by some subset of HPC customers to whom NVIDIA really wants to cater.

by paulmd 6y ago

R0b0t1 6y ago

His experience does translate pretty well to typical computer workload analysis. Realize he has to digest, at least partially, most of the kernel commits. Someone made a comment about special purpose instructions being the future. I think that is true, in part, but those instructions might be better off on an accelerator of some kind, or at least interfaced to the chip in a way that is better than how AVX512 must be used.

paulmd 6y ago
> Someone made a comment about special purpose instructions being the future. I think that is true, in part, but those instructions might be better off on an accelerator of some kind, or at least interfaced to the chip in a way that is better than how AVX512 must be used. The problem with the coprocessor model is that it adds an enormous amount of latency. A lot of things aren't worth pushing off to a GPU because it adds so much time, and because there is a limited amount of bandwidth to push it all there and back. Furthermore, the farther you are sending something, the more power you need to do it. Sending it off-chip is significantly less efficient than doing it on-chip, all things equal. RTX is sort of a microcosm of these problems - people really wanted NVIDIA to implement RTX as an off-die "accelerator chip" (and some dumb rumors are even bringing this topic back for the 3000 series despite all the reasons it doesn't work). But it actually makes no sense to separate this - you can't just push it off to the RT core and be done with it, after RT finds collisions you need to go back to shaders and do stuff with it, so it needs to be feasible to flip back and forth between shading, RT, and back to shading. Moving it off die would mean that you now have some kind of async thing or buffer going on and that would kill latency and make it infeasible for real-time shading at least. Furthermore you have cache benefits if everything stays paged in while you are finishing up one tile. Or another example would be Fusion - sure the idea of having coherent memory and triggering the GPU "like an AVX unit" sounds great but it's only unidirectional cache coherency - the GPU reads an up-to-date picture of the CPU but the CPU is not optimized to read out of the GPU, so while it can work in this mode the performance is "singularly poor". This means that "flipping back and forth" does not really work here either, it is structured as a CPU-to-GPU pipeline, not a general purpose coprocessor. With as much as they were pushing Heterogeneous Systems Architecture, presumably they couldn't reasonably implement that without trashing GPU performance, cause if they could I bet they would. https://www.realworldtech.com/fusion-llano/3/ https://www.realworldtech.com/fusion-llano/3/ Integrating specialty units into the core is the tradeoff between latency and performance - it's nowhere near as much performance as a dedicated accelerator, but it's nowhere near as much latency either. Depending on how much work each item needs done, it may be better to do it on-core and keep trucking. And often this is not particularly expensive - things like crypto acceleration instructions are basically uncontroversial. And that is analogous to something like BVH traversal units on NVIDIA, where a ~5% die area expenditure speeds up that segment of the program by 8x or so and results in 2x performance improvements program-wide in some cases. The NVIDIA equivalent to AVX-512 would be tensor cores I suppose - not all that useful to general workloads (apart from DLSS), consuming a fairly significant amount of die area (around 10-15% iirc) but greatly desired by some subset of HPC customers to whom NVIDIA really wants to cater.