4 ms·

People don’t like to be vendor locked anymore. For that reason, they really do seek options to get out of NVIDIA monopoly.

by nicce 4y ago

pjmlp 4y ago
Except Intel and AMD so far failed to provide a good development story for libraries, polyglot programming, and IDE tooling. Unless one is happy using vim and emacs to target GPUs, coding in C or C++, with either pixel debugging or lame printfs. Making fire with stones to avoid a proprietary lighter.
- jiggawatts 4y ago
  > Making fire with stones to avoid a proprietary lighter. I'm stealing that quote!
latency-guy2 4y ago
NVIDIA is so far ahead of the competition that people really do not care about the poor API that ROCm offers. Intel is not really a thing to even remotely care about for GPUs, they are still obviously important for CPU end. All the other players matter even less. I badly wanted something decent as an alternative to CUDA for quite a while, but even after leaving AI myself, it is clear they are not near the standards that NVIDIA has been offering for nearing a decade now already. We don't even have to get into all the issues consumers have gotten with AMD's GPUs.
- imiric 4y ago
  I'm not deeply familiar with this space, but isn't training and inference on CPUs gaining some traction[1][2][3]? It's still surely orders of magnitude slower, but as CPUs get more powerful and ML frameworks get more efficient, making this cheaper and more accessible would be a major breakthrough. [1]: https://arxiv.org/abs/2206.10034 https://arxiv.org/abs/2206.10034 [2]: https://www.intel.com/content/www/us/en/developer/articles/technical/ai-inference-acceleration-on-intel-cpus.html https://www.intel.com/content/www/us/en/developer/articles/t... [3]: https://www.usenix.org/conference/atc19/presentation/liu-yizhi https://www.usenix.org/conference/atc19/presentation/liu-yiz...
  - freeone3000 4y ago
    CPU training was and continues to be one-tenth of the speed, dollar for dollar. It's not even close. The inescapable trend is that bigger models are better models. So if you're doing this professionally, you're going to need GPUs not only for training, but increasingly for inference in order to get decent latency.
    - geysersam 4y ago
      On the other hand, it's likely competition will catch up to Nvidia. They were early and good. But the space is too lucrative, and the technology is not special enough that they will be able to keep their dominant position for long.
Vt71fcAqt7 4y ago
To add to this, CUDA doesn't run on mobile. That alone will force an (open, hopefuly) competitor to surface.
- saurik 4y ago
  You really think Apple is going to implement an "open" anything?
  - coldtea 4y ago
    You mean aside from their long history of developing open things, from Webkit (based on the KDE's open basic wkhtml and made into a full-blown browser engine) to Swift?
  - Vt71fcAqt7 4y ago
    I don't think Apple is the only mobile vendor. I think that between AMD, Intel, Qualcomm, Mediatek, Samsung and Google, they will realize that they are all being screwed over by Nvidia. Right now ML is still mostly research but when real products start surfacing (or, perhaps, for real products to surface to begin with) they will have to settle on something, wether it's ROCM, OneAPI or something else.
    - pjmlp 4y ago
      So far they have been screwed by Google, as they also don't have OpenCL anything on Android, and Vulkan compute is something for hardcore NDK users.
- pjmlp 4y ago
  Nor does anything Khronos related for compute APIs. While Apple initially promoted OpenCL, Android never supported it as official API rather Renderscript. So on iOS eventually Apple moved into Metal, usable from Objective-C and Swift, while on Android, Google has deprecated Renderscript usable from Java and Kotlin, pushing everyone to now learn Vulkan compute and do the integration on the NDK by themselves while learning C, C++ and GLSL on the process (as no big deal from their POV). Basically hardly anyone cares with such great usability story. /s
camel-cdr 4y ago
Isn't SYCL and for lower level stuff vulkan compute a good alternative?
- pjmlp 4y ago
  Vulkan compute is a joke in terms of current tooling, SYCL is for C++, not polyglot.

People don’t like to be vendor locked anymore. For that reason, they really do seek options to get out of NVIDIA monopoly.

by nicce 4y ago

People don’t like to be vendor locked anymore. For that reason, they really do seek options to get out of NVIDIA monopoly.

pjmlp 4y ago

Except Intel and AMD so far failed to provide a good development story for libraries, polyglot programming, and IDE tooling. Unless one is happy using vim and emacs to target GPUs, coding in C or C++, with either pixel debugging or lame printfs. Making fire with stones to avoid a proprietary lighter.

jiggawatts 4y ago
> Making fire with stones to avoid a proprietary lighter. I'm stealing that quote!

latency-guy2 4y ago

NVIDIA is so far ahead of the competition that people really do not care about the poor API that ROCm offers. Intel is not really a thing to even remotely care about for GPUs, they are still obviously important for CPU end. All the other players matter even less. I badly wanted something decent as an alternative to CUDA for quite a while, but even after leaving AI myself, it is clear they are not near the standards that NVIDIA has been offering for nearing a decade now already. We don't even have to get into all the issues consumers have gotten with AMD's GPUs.

imiric 4y ago
I'm not deeply familiar with this space, but isn't training and inference on CPUs gaining some traction[1][2][3]? It's still surely orders of magnitude slower, but as CPUs get more powerful and ML frameworks get more efficient, making this cheaper and more accessible would be a major breakthrough. [1]: https://arxiv.org/abs/2206.10034 https://arxiv.org/abs/2206.10034 [2]: https://www.intel.com/content/www/us/en/developer/articles/technical/ai-inference-acceleration-on-intel-cpus.html https://www.intel.com/content/www/us/en/developer/articles/t... [3]: https://www.usenix.org/conference/atc19/presentation/liu-yizhi https://www.usenix.org/conference/atc19/presentation/liu-yiz...
- freeone3000 4y ago
  CPU training was and continues to be one-tenth of the speed, dollar for dollar. It's not even close. The inescapable trend is that bigger models are better models. So if you're doing this professionally, you're going to need GPUs not only for training, but increasingly for inference in order to get decent latency.
  - geysersam 4y ago
    On the other hand, it's likely competition will catch up to Nvidia. They were early and good. But the space is too lucrative, and the technology is not special enough that they will be able to keep their dominant position for long.

Vt71fcAqt7 4y ago

To add to this, CUDA doesn't run on mobile. That alone will force an (open, hopefuly) competitor to surface.

saurik 4y ago
You really think Apple is going to implement an "open" anything?
- coldtea 4y ago
  You mean aside from their long history of developing open things, from Webkit (based on the KDE's open basic wkhtml and made into a full-blown browser engine) to Swift?
- Vt71fcAqt7 4y ago
  I don't think Apple is the only mobile vendor. I think that between AMD, Intel, Qualcomm, Mediatek, Samsung and Google, they will realize that they are all being screwed over by Nvidia. Right now ML is still mostly research but when real products start surfacing (or, perhaps, for real products to surface to begin with) they will have to settle on something, wether it's ROCM, OneAPI or something else.
  - pjmlp 4y ago
    So far they have been screwed by Google, as they also don't have OpenCL anything on Android, and Vulkan compute is something for hardcore NDK users.
pjmlp 4y ago
Nor does anything Khronos related for compute APIs. While Apple initially promoted OpenCL, Android never supported it as official API rather Renderscript. So on iOS eventually Apple moved into Metal, usable from Objective-C and Swift, while on Android, Google has deprecated Renderscript usable from Java and Kotlin, pushing everyone to now learn Vulkan compute and do the integration on the NDK by themselves while learning C, C++ and GLSL on the process (as no big deal from their POV). Basically hardly anyone cares with such great usability story. /s

camel-cdr 4y ago

Isn't SYCL and for lower level stuff vulkan compute a good alternative?

pjmlp 4y ago
Vulkan compute is a joke in terms of current tooling, SYCL is for C++, not polyglot.