4 ms·

His experience does translate pretty well to typical computer workload analysis. Realize he has to digest, at least partially, most of the kernel commits. Some

• paulmd 6y ago
  > Someone made a comment about special purpose instructions being the future. I think that is true, in part, but those instructions might be better off on an accelerator of some kind, or at least interfaced to the chip in a way that is better than how AVX512 must be used. The problem with the coprocessor model is that it adds an enormous amount of latency. A lot of things aren't worth pushing off to a GPU because it adds so much time, and because there is a limited amount of bandwidth to push it all there and back. Furthermore, the farther you are sending something, the more power you need to do it. Sending it off-chip is significantly less efficient than doing it on-chip, all things equal. RTX is sort of a microcosm of these problems - people really wanted NVIDIA to implement RTX as an off-die "accelerator chip" (and some dumb rumors are even bringing this topic back for the 3000 series despite all the reasons it doesn't work). But it actually makes no sense to separate this - you can't just push it off to the RT core and be done with it, after RT finds collisions you need to go back to shaders and do stuff with it, so it needs to be feasible to flip back and forth between shading, RT, and back to shading. Moving it off die would mean that you now have some kind of async thing or buffer going on and that would kill latency and make it infeasible for real-time shading at least. Furthermore you have cache benefits if everything stays paged in while you are finishing up one tile. Or another example would be Fusion - sure the idea of having coherent memory and triggering the GPU "like an AVX unit" sounds great but it's only unidirectional cache coherency - the GPU reads an up-to-date picture of the CPU but the CPU is not optimized to read out of the GPU, so while it can work in this mode the performance is "singularly poor". This means that "flipping back and forth" does not really work here either, it is structured as a CPU-to-GPU pipeline, not a general purpose coprocessor. With as much as they were pushing Heterogeneous Systems Architecture, presumably they couldn't reasonably implement that without trashing GPU performance, cause if they could I bet they would. https://www.realworldtech.com/fusion-llano/3/ https://www.realworldtech.com/fusion-llano/3/ Integrating specialty units into the core is the tradeoff between latency and performance - it's nowhere near as much performance as a dedicated accelerator, but it's nowhere near as much latency either. Depending on how much work each item needs done, it may be better to do it on-core and keep trucking. And often this is not particularly expensive - things like crypto acceleration instructions are basically uncontroversial. And that is analogous to something like BVH traversal units on NVIDIA, where a ~5% die area expenditure speeds up that segment of the program by 8x or so and results in 2x performance improvements program-wide in some cases. The NVIDIA equivalent to AVX-512 would be tensor cores I suppose - not all that useful to general workloads (apart from DLSS), consuming a fairly significant amount of die area (around 10-15% iirc) but greatly desired by some subset of HPC customers to whom NVIDIA really wants to cater.
  • R0b0t1 6y ago
    >Or another example would be Fusion - sure the idea of having coherent memory and triggering the GPU "like an AVX unit" sounds great As we see though, if you need to "trigger" the unit you may have problems. In this case AVX triggers a cooldown. Running the AVX instructions right now is just as if you triggered an alternate mode. The examples you give citing NVIDIA seem to be more inline with what I'm suggesting. The shared memory fabric allows fast offload the the extremely specialized processing units. The issue is these terms are fluid. It used to be that PCIe was the fabric, but now that's too slow, so what is on the die is the fabric, etc.
    • paulmd 6y ago
      > As we see though, if you need to "trigger" the unit you may have problems. In this case AVX triggers a cooldown. Running the AVX instructions right now is just as if you triggered an alternate mode. Not all AVX instructions trigger a cooldown to build voltage, only 512-bit ones or 256-bit multiplies. So the penalty in these cases is in fact zero unless you are doing a lot of them and triggering cumulative power cooldowns (in which case you are seeing a large benefit in throughput, although potentially you should consider offloading them at that point, yes). Also, AMD doesn't have cumulative power cooldowns or clockdowns at all, that is an Intel implementation detail, not a fault of the instruction set. > The examples you give citing NVIDIA seem to be more inline with what I'm suggesting. The shared memory fabric allows fast offload the the extremely specialized processing units. No, actually, despite some confusing block diagrams, RTX and tensor cores are actually part of the SM engine, not "standalone units" in a different part of the chip. They live right next to the shaders. There is no "fabric" there, it is execution units working on registers just like any other instruction.
      • R0b0t1 6y ago
        > Not all AVX instructions trigger a cooldown to build voltage Ok, but the AVX instructions that do trigger the cooldown may be the ones you wish to use, and the statement is still valid. I never mentioned AMD. > not "standalone units" I didn't say they were standalone, in fact, the opposite. It just depends on what you want to call a boundary. The tighter integration is what is driving these special purpose units but up until now they've not affected package TDP so heavily.