3 ms·
What about cards with higher memory bandwidth, like Groq’s LPUs? Would that help with data branching?
by earthnail 2y ago
What about cards with higher memory bandwidth, like Groq’s LPUs? Would that help with data branching?
- hansvm 2y agoData branching in general, no (pulling from 32 places is still 32x as expensive in that architecture, but you might be able to load bigger chunks in each place). For a KAN, a bit (it shifts the constants involved when I was talking about smaller vs bigger splines above -- sparsity and dropout will tend to make the GPU tend toward that worst-case though). You still have the problem that you're heavily underutilizing the GPU's compute.