2 ms·
Some GPU assembly advancements... * Bpermute and permute. (Pshufb is like permute, a gather operation. Bpermute is the opposite, like a scatter. Bpermute Doesn
by dragontamer 6y ago
Some GPU assembly advancements...
* Bpermute and permute. (Pshufb is like permute, a gather operation. Bpermute is the opposite, like a scatter. Bpermute Doesn't exist on x86 yet)
* __shared__ memory crossbar: every simd unit can read, or write, to shared memory in parallel per clocktick. The crossbar can also broadcast 1-to-all each clocktick.
* Butterfly permute: the fundamental pattern in permutations for a variety of operations. Most noticably for scan, and FFTs. Butterfly networks are closely related to pext and pdep implementation (showing how common that particular permute is).
* 8+ way hyperthreads / SMT. GPUs have very bad latency, but very high SMT counteracts that problem well in practice.
* PCIe Atomics: perform those compare and swap negotiations over I/O, allowing tight CPU and GPU memory integration.
* Crazy RAM. 1000GBps on HBM2. 800GBps over GDDR6x thanks to 2-bits transferred per clock ticks.
* Crazy networks. AMD Infinity fabric pushes over 100GB. NVlink is 600GBps. A GPU network link has more bandwidth than a typical CPU's DDR4 RAM bandwidth.
* NVidia SASS has the craziest instruction set, the compiler figures out read / write hazards and publishes them in the SASS assembly itself. NVidias ISA decoder + assembler team is doing something crazy here, the likes I haven't seen in any other instruction set ever.
* "Ballot" instructions. Its... really hard to explain why these are useful. They just are, lol.
Just a few cool concepts I've seen in the GPU world recently. Sure, matrix multiplications get the headlines because of tensors / deep learning. But don't sleep on the obscure stuff.