jhj

16 ms·

121.
by jhj 10y ago
There's no real analogue to a CPU core (or thread) on a GPU, there are warp schedulers (Nvidia) and functional units with varying throughput rates. The closest analogue to a CPU thread is the GPU warp (Nvidia), which shares a program c
122.
by jhj 10y ago
"CUDA cores" is a misleading term. When they quote "CUDA cores", they've been counting float32 fma functional units; e.g., Tesla K40 has 192 float32 fma units per SM x 15 SMs => 2880 "CUDA cores". http
123.
by jhj 10y ago
see https://github.com/facebook/fbcunn/blob/master/test/Referenc... for a loop nest that shows what the forward pass of a 2-d image convnet convolution module does. That's gussied up with convo
124.
by jhj 10y ago
Valid-only convolution (in the MATLAB sense) by itself reduces the dimensionality of the input; for images, it will go from (h x w) to (h - kh + 1) x (w - kw + 1) per each plane. You can think of a convnet as a series of feature transformat
125.
by jhj 10y ago
For someone that knows a thing about CUDA and parallel programming already, the best reference is Paulius Micikevicius’ presentations. If the words in it mean something to you, these 100+ slides explain more about the hardware and programmi
126.
by jhj 10y ago
&& / ?: may not be “branching operations”, the compiler could determine that the operands have no side effects (q->score is used in both branches and q must be a valid or invalid pointer in both) and collapse that down to j
127.
by jhj 10y ago
That's just a combination of GPU model and data parallelism, which have been supported for about 2 years in Torch among others. You can add in the outer product trick for fc layers as well. https://github.com/soumith&#x
128.
by jhj 10y ago
the hardware performs some coalescing, but it's complicated... memory accesses in a warp do not necessarily have to be contiguous, but it does matter how many 32 byte global memory segments (and 128 byte l1 cache segments) they fall in
129.
by jhj 10y ago
the warp scheduler can dual-issue instructions; it has to because otherwise 192 fp32 alus can't be used by the 4 warp schedulers (4 x 32) effectively. increased ipc can do this for you. this was maxwell though; the ratio of resources t
130.
by jhj 10y ago
It wouldn't because as you say the ALUs are the same, it just would change the unit that the warp scheduler deals with (and relevant changes to kernel occupancy that it would entail).
131.
by jhj 10y ago
Bit math (bfe.*, logical bit ops, ...) are nowhere near as fast as float32 ops on their GPUs by a factor of 2-4x; there are many fewer functional units for them. Maxwell doesn't have support for many 64-bit word bit ops as well (they g
132.
by jhj 10y ago
Warp size is still 32 in Pascal.