3 ms·

This is a nice start, but what would really help is for someone who understands the GPU programming model in-depth to give this a shot, with the goal of reducin

This is a nice start, but what would really help is for someone who understands the GPU programming model in-depth to give this a shot, with the goal of reducing DRAM bandwidth and fitting the layers exactly onto a GPU's memory hierarchy (cache levels, local memory and registers). Basically, sizes of a target HW platform's memories should be hyper-parameters of the architecture.

machinekob 3y ago
Indeed it could be a target, but newer and newer co-processor for pure MatMul or even NPU are created and cache/register are changing fast from one architecture to another so it still wouldnt be optimal for most users.