3 ms·
According to Ilya Sutskever in a podcast that I heard, GPUs are already pretty close to ASIC performance for AI workloads. Nvidia can highly optimize the full s
by huijzer 3y ago
According to Ilya Sutskever in a podcast that I heard, GPUs are already pretty close to ASIC performance for AI workloads. Nvidia can highly optimize the full stack due to their economies of scale.
- josephg 3y agoRight. As I understand it, training on gpus isn’t limited by the speed of matrix multiplications. It’s limited by memory bandwidth. So a faster ASIC for matrix operations won’t help. It’ll just sit idle while the system stalls waiting for data to become available. That’s why having 96gb (with another 480gb or whatever) available via high speed interconnect is a big deal. It means we can train bigger models faster.