3 ms·
I'm not deeply familiar with this space, but isn't training and inference on CPUs gaining some traction[1][2][3]? It's still surely orders of magnitude slower,
by imiric 4y ago
I'm not deeply familiar with this space, but isn't training and inference on CPUs gaining some traction[1][2][3]? It's still surely orders of magnitude slower, but as CPUs get more powerful and ML frameworks get more efficient, making this cheaper and more accessible would be a major breakthrough.
[1]: https://arxiv.org/abs/2206.10034 https://arxiv.org/abs/2206.10034
[2]: https://www.intel.com/content/www/us/en/developer/articles/technical/ai-inference-acceleration-on-intel-cpus.html https://www.intel.com/content/www/us/en/developer/articles/t...
[3]: https://www.usenix.org/conference/atc19/presentation/liu-yizhi https://www.usenix.org/conference/atc19/presentation/liu-yiz...
- freeone3000 4y agoCPU training was and continues to be one-tenth of the speed, dollar for dollar. It's not even close. The inescapable trend is that bigger models are better models. So if you're doing this professionally, you're going to need GPUs not only for training, but increasingly for inference in order to get decent latency.
- geysersam 4y agoOn the other hand, it's likely competition will catch up to Nvidia. They were early and good. But the space is too lucrative, and the technology is not special enough that they will be able to keep their dominant position for long.