2 ms·
4x speedup at batch size 1 is really terrible. GPUs are terrible at small batch sizes. And that is comparing a research project with production hardware. Hones
by IshKebab 6y ago
4x speedup at batch size 1 is really terrible. GPUs are terrible at small batch sizes. And that is comparing a research project with production hardware.
Honestly if you look at any of the other AI hardware startups they all advertise much more significant speedups.