3 ms·

NVIDIA is so far ahead of the competition that people really do not care about the poor API that ROCm offers. Intel is not really a thing to even remotely care

NVIDIA is so far ahead of the competition that people really do not care about the poor API that ROCm offers. Intel is not really a thing to even remotely care about for GPUs, they are still obviously important for CPU end. All the other players matter even less. I badly wanted something decent as an alternative to CUDA for quite a while, but even after leaving AI myself, it is clear they are not near the standards that NVIDIA has been offering for nearing a decade now already. We don't even have to get into all the issues consumers have gotten with AMD's GPUs.

imiric 4y ago
I'm not deeply familiar with this space, but isn't training and inference on CPUs gaining some traction[1][2][3]? It's still surely orders of magnitude slower, but as CPUs get more powerful and ML frameworks get more efficient, making this cheaper and more accessible would be a major breakthrough. [1]: https://arxiv.org/abs/2206.10034 https://arxiv.org/abs/2206.10034 [2]: https://www.intel.com/content/www/us/en/developer/articles/technical/ai-inference-acceleration-on-intel-cpus.html https://www.intel.com/content/www/us/en/developer/articles/t... [3]: https://www.usenix.org/conference/atc19/presentation/liu-yizhi https://www.usenix.org/conference/atc19/presentation/liu-yiz...
- freeone3000 4y ago
  CPU training was and continues to be one-tenth of the speed, dollar for dollar. It's not even close. The inescapable trend is that bigger models are better models. So if you're doing this professionally, you're going to need GPUs not only for training, but increasingly for inference in order to get decent latency.
  - geysersam 4y ago
    On the other hand, it's likely competition will catch up to Nvidia. They were early and good. But the space is too lucrative, and the technology is not special enough that they will be able to keep their dominant position for long.

imiric 4y ago

I'm not deeply familiar with this space, but isn't training and inference on CPUs gaining some traction[1][2][3]? It's still surely orders of magnitude slower, but as CPUs get more powerful and ML frameworks get more efficient, making this cheaper and more accessible would be a major breakthrough. [1]: https://arxiv.org/abs/2206.10034 https://arxiv.org/abs/2206.10034 [2]: https://www.intel.com/content/www/us/en/developer/articles/technical/ai-inference-acceleration-on-intel-cpus.html https://www.intel.com/content/www/us/en/developer/articles/t... [3]: https://www.usenix.org/conference/atc19/presentation/liu-yizhi https://www.usenix.org/conference/atc19/presentation/liu-yiz...

freeone3000 4y ago
CPU training was and continues to be one-tenth of the speed, dollar for dollar. It's not even close. The inescapable trend is that bigger models are better models. So if you're doing this professionally, you're going to need GPUs not only for training, but increasingly for inference in order to get decent latency.
- geysersam 4y ago
  On the other hand, it's likely competition will catch up to Nvidia. They were early and good. But the space is too lucrative, and the technology is not special enough that they will be able to keep their dominant position for long.