4 ms·
> Since you mentioned this in the same breath as GPUs, I feel I have to point out that according to some reverse engineering paper, Nvidia's Turing is a VLIW ar
by dragontamer 7y ago
> Since you mentioned this in the same breath as GPUs, I feel I have to point out that according to some reverse engineering paper, Nvidia's Turing is a VLIW architecture. (I'm talking about the actual hardware here, not PTX.)
NVidia Volta / Turing has been reverse engineered here: https://arxiv.org/abs/1804.06826 https://arxiv.org/abs/1804.06826 . EDIT: I'm talking about the actual hardware, the SASS assembly, not PTX.
It doesn't look like a VLIW architecture to me. Page 14 for the specific instruction-set details. I realize this is mostly a matter of opinion, but... those control-codes are very different from the VLIW that was implemented in the Itanium.
> On Volta, a single 128-bit word contains one instruction together with thecontrol information associated to that instruction.
So NVidia has a 128-bit instruction (16-bytes) with a LOT of control information involved. The control information encodes read/write barriers, but there is still one-instruction per... instruction.
The "core" of VLIW was to encode more than one instruction-per-bundle. Itanium would encode maybe 3-instructions per bundle for example.
What NVidia is doing here is having the compiler figure out a bunch of read/write/dependency barriers so that the GPU won't have to figure it out on its own (I presume this increases power-efficiency). The only thing similar to VLIW is that NVidia has a "complicated assembler" which needs to figure out this information and encode it into the instruction stream. Otherwise, it is clearly NOT a VLIW architecture.
> Presumably they have some reason for that that's unrelated to increasing IPC, since AFAIK their GPUs aren't superscalar.
NVidia Turing can execute floating-point instructions simultaneously with integer-instructions. So they are now superscalar. The theory is that floating-point units will be busy doing the typical GPU math. Integer-instructions are primarily used for branching / looping constructs (and not much else in graphics-heavy code), so they are usually independent operations.