3 ms·

>Or another example would be Fusion - sure the idea of having coherent memory and triggering the GPU "like an AVX unit" sounds great As we see though, if you n

by R0b0t1 6y ago

>Or another example would be Fusion - sure the idea of having coherent memory and triggering the GPU "like an AVX unit" sounds great As we see though, if you need to "trigger" the unit you may have problems. In this case AVX triggers a cooldown. Running the AVX instructions right now is just as if you triggered an alternate mode. The examples you give citing NVIDIA seem to be more inline with what I'm suggesting. The shared memory fabric allows fast offload the the extremely specialized processing units. The issue is these terms are fluid. It used to be that PCIe was the fabric, but now that's too slow, so what is on the die is the fabric, etc.

paulmd 6y ago
> As we see though, if you need to "trigger" the unit you may have problems. In this case AVX triggers a cooldown. Running the AVX instructions right now is just as if you triggered an alternate mode. Not all AVX instructions trigger a cooldown to build voltage, only 512-bit ones or 256-bit multiplies. So the penalty in these cases is in fact zero unless you are doing a lot of them and triggering cumulative power cooldowns (in which case you are seeing a large benefit in throughput, although potentially you should consider offloading them at that point, yes). Also, AMD doesn't have cumulative power cooldowns or clockdowns at all, that is an Intel implementation detail, not a fault of the instruction set. > The examples you give citing NVIDIA seem to be more inline with what I'm suggesting. The shared memory fabric allows fast offload the the extremely specialized processing units. No, actually, despite some confusing block diagrams, RTX and tensor cores are actually part of the SM engine, not "standalone units" in a different part of the chip. They live right next to the shaders. There is no "fabric" there, it is execution units working on registers just like any other instruction.
- R0b0t1 6y ago
  > Not all AVX instructions trigger a cooldown to build voltage Ok, but the AVX instructions that do trigger the cooldown may be the ones you wish to use, and the statement is still valid. I never mentioned AMD. > not "standalone units" I didn't say they were standalone, in fact, the opposite. It just depends on what you want to call a boundary. The tighter integration is what is driving these special purpose units but up until now they've not affected package TDP so heavily.
  - paulmd 6y ago
    > Ok, but the AVX instructions that do trigger the cooldown may be the ones you wish to use, and the statement is still valid. I never mentioned AMD. Again, it's not that AVX instructions inherently must be clocked down for. It is an implementation detail of Intel's processor. AMD already has different boost behavior for AVX2 and will probably have different boost behavior for AVX-512 when they implement that. > It just depends on what you want to call a boundary What you were saying is that they are units attached over some "fabric", which isn't the case. > The shared memory fabric allows fast offload the the extremely specialized processing units. They aren't "multiple processors that share memory", it is an execution unit within the Streaming Multiprocessor. It is no different from an AVX instruction, they are driven by registers, not by a fabric and not by memory. At risk of reigniting the old Bulldozer "what is a core really anyway", AVX and tensor cores don't really represent an independent core, they are an execution unit within a larger core. They can't operate independently without the other execution resources provided by the core.

by R0b0t1 6y ago

paulmd 6y ago

> As we see though, if you need to "trigger" the unit you may have problems. In this case AVX triggers a cooldown. Running the AVX instructions right now is just as if you triggered an alternate mode. Not all AVX instructions trigger a cooldown to build voltage, only 512-bit ones or 256-bit multiplies. So the penalty in these cases is in fact zero unless you are doing a lot of them and triggering cumulative power cooldowns (in which case you are seeing a large benefit in throughput, although potentially you should consider offloading them at that point, yes). Also, AMD doesn't have cumulative power cooldowns or clockdowns at all, that is an Intel implementation detail, not a fault of the instruction set. > The examples you give citing NVIDIA seem to be more inline with what I'm suggesting. The shared memory fabric allows fast offload the the extremely specialized processing units. No, actually, despite some confusing block diagrams, RTX and tensor cores are actually part of the SM engine, not "standalone units" in a different part of the chip. They live right next to the shaders. There is no "fabric" there, it is execution units working on registers just like any other instruction.

R0b0t1 6y ago
> Not all AVX instructions trigger a cooldown to build voltage Ok, but the AVX instructions that do trigger the cooldown may be the ones you wish to use, and the statement is still valid. I never mentioned AMD. > not "standalone units" I didn't say they were standalone, in fact, the opposite. It just depends on what you want to call a boundary. The tighter integration is what is driving these special purpose units but up until now they've not affected package TDP so heavily.
- paulmd 6y ago
  > Ok, but the AVX instructions that do trigger the cooldown may be the ones you wish to use, and the statement is still valid. I never mentioned AMD. Again, it's not that AVX instructions inherently must be clocked down for. It is an implementation detail of Intel's processor. AMD already has different boost behavior for AVX2 and will probably have different boost behavior for AVX-512 when they implement that. > It just depends on what you want to call a boundary What you were saying is that they are units attached over some "fabric", which isn't the case. > The shared memory fabric allows fast offload the the extremely specialized processing units. They aren't "multiple processors that share memory", it is an execution unit within the Streaming Multiprocessor. It is no different from an AVX instruction, they are driven by registers, not by a fabric and not by memory. At risk of reigniting the old Bulldozer "what is a core really anyway", AVX and tensor cores don't really represent an independent core, they are an execution unit within a larger core. They can't operate independently without the other execution resources provided by the core.