5 ms·

Nvidia also has CUDA, which has surpassed all rival attempts at GPU programming frameworks in the last 2 decades. I remember learning OpenCL and rooting for it

Nvidia also has CUDA, which has surpassed all rival attempts at GPU programming frameworks in the last 2 decades. I remember learning OpenCL and rooting for it to become the standard but couldn't find a single job for that role.

cyberax 2y ago
CUDA is nice, but it's not a moat. Rocm exists, and even creating a totally new AI computer API is not that far-fetched.
- _zoltan_ 2y ago
  CUDA is totally a moat. ROCM exists but AMD is like 100k engineering years behind NVDA just in terms of how much time NVDA had to invest into CUDA and all the ecosystem around it. need dataframes and pandas? cuDF. need compression/decompression? nvcomp. need vector search? cuVS. and the list goes on and on and on.
  - bjornsing 2y ago
    > need dataframes and pandas? cuDF. need compression/decompression? nvcomp. need vector search? cuVS. and the list goes on and on and on. Sure, but that doesn’t mean I’m going to pay a billion extra for my next cluster – a cluster that just does matrix multiplication and exponentiation over and over again really fast. So I’d say CUDA is clearly a moat for the (relatively tiny) GPGPU space, but not for large scale production AI.
    - erinaceousjones 2y ago
      > Relatively tiny GPGPU space There's a lot of other things which are very GPU parallelizable which just aren't being talked about because they're not part of the AI language model boom, but to pick a few I've seen in passing just from my (quite removed from AI) job: - Ocean weather forecasting / modelling - Satellite imagery and remote sensing processing / pre-processing - Processing of spatial data from non-optical sensors (Lidar, sonar) - Hydrodynamic and aerodynamic turbulent flow simulation - Mechanical stress simulation Loads of "embarrassingly parallel" stuff in the realms of industrial R&D are benefitting from the slow migration from traditional CPU-heavy compute clusters to ones with GPUs available, because even before the recent push to "decarbonise" HPC, people were seeing the increase in "work done per watt" type cost efficiency is beneficial. Probably "relatively tiny" right now compared to the AI boom, but that stuff has been there for years and will continue to grow at a slow and steady pace, imo. Adoption of GPGPU for lots of things is probably being bolstered by the LLM bros now, to be honest. CUDA benefits from being early to market in those areas. Mature tools, mature docs, lots of extra bolt-ons, organizational inertia "we already started this using CUDA", etc.
      - bjornsing 2y ago
        Sure, all that’s interesting and highly realistic. But does it make Nvidia the most valuable company in the world?
  - cyberax 2y ago
    It's not really 100k years. A ROCm subset good enough to run an LLM can be done within a year or so. You can _technicall_ run it now with tinygrad. The silence from AMD has been deafening, though. I can't fathom why they're just ignoring the AI market.
  - HDThoreaun 2y ago
    There are billions of dollars being funneled into an open source CUDA replacement by all the big tech firms that are pissed about monopoly pricing from nvidia. I just cant see that never working.
- malux85 2y ago
  I’ve been using GPU accelerated computing for more than 10 years now. I write CUDA kernels in my sleep. I have seen 10s of frameworks and 10s of chip companies come and go. I have NEVER even HEARD of Rocm, and neither has anyone in the GPU programming slack group I just asked. CUDA is absolutely a moat.

cyberax 2y ago

CUDA is nice, but it's not a moat. Rocm exists, and even creating a totally new AI computer API is not that far-fetched.

_zoltan_ 2y ago
CUDA is totally a moat. ROCM exists but AMD is like 100k engineering years behind NVDA just in terms of how much time NVDA had to invest into CUDA and all the ecosystem around it. need dataframes and pandas? cuDF. need compression/decompression? nvcomp. need vector search? cuVS. and the list goes on and on and on.
- bjornsing 2y ago
  > need dataframes and pandas? cuDF. need compression/decompression? nvcomp. need vector search? cuVS. and the list goes on and on and on. Sure, but that doesn’t mean I’m going to pay a billion extra for my next cluster – a cluster that just does matrix multiplication and exponentiation over and over again really fast. So I’d say CUDA is clearly a moat for the (relatively tiny) GPGPU space, but not for large scale production AI.
  - erinaceousjones 2y ago
    > Relatively tiny GPGPU space There's a lot of other things which are very GPU parallelizable which just aren't being talked about because they're not part of the AI language model boom, but to pick a few I've seen in passing just from my (quite removed from AI) job: - Ocean weather forecasting / modelling - Satellite imagery and remote sensing processing / pre-processing - Processing of spatial data from non-optical sensors (Lidar, sonar) - Hydrodynamic and aerodynamic turbulent flow simulation - Mechanical stress simulation Loads of "embarrassingly parallel" stuff in the realms of industrial R&D are benefitting from the slow migration from traditional CPU-heavy compute clusters to ones with GPUs available, because even before the recent push to "decarbonise" HPC, people were seeing the increase in "work done per watt" type cost efficiency is beneficial. Probably "relatively tiny" right now compared to the AI boom, but that stuff has been there for years and will continue to grow at a slow and steady pace, imo. Adoption of GPGPU for lots of things is probably being bolstered by the LLM bros now, to be honest. CUDA benefits from being early to market in those areas. Mature tools, mature docs, lots of extra bolt-ons, organizational inertia "we already started this using CUDA", etc.
    - bjornsing 2y ago
      Sure, all that’s interesting and highly realistic. But does it make Nvidia the most valuable company in the world?
- cyberax 2y ago
  It's not really 100k years. A ROCm subset good enough to run an LLM can be done within a year or so. You can _technicall_ run it now with tinygrad. The silence from AMD has been deafening, though. I can't fathom why they're just ignoring the AI market.
- HDThoreaun 2y ago
  There are billions of dollars being funneled into an open source CUDA replacement by all the big tech firms that are pissed about monopoly pricing from nvidia. I just cant see that never working.
malux85 2y ago
I’ve been using GPU accelerated computing for more than 10 years now. I write CUDA kernels in my sleep. I have seen 10s of frameworks and 10s of chip companies come and go. I have NEVER even HEARD of Rocm, and neither has anyone in the GPU programming slack group I just asked. CUDA is absolutely a moat.