4 ms·

The CUDA C APIs are supported as much in C as in C++ using SCALE! Cuda-fortran is not currently supported by scale since we haven't seen much use of it "in the

The CUDA C APIs are supported as much in C as in C++ using SCALE! Cuda-fortran is not currently supported by scale since we haven't seen much use of it "in the wild" to push it up our priority list.

anon291 2y ago
It doesn't matter though. NVIDIA distributes tons of libraries built atop CUDA that you cannot distribute or use on AMD chips legally. Cutlass, CuBLAS, NCCL, etc.
- tama_sala 2y ago
  Correct, which one of the main moats Nvidia has when it comes to training
- ckitching 2y ago
  SCALE doesn't use cuBlas and friends. For those APIs, it uses either its own implementations of the functions, or delegates to an existing AMD library (such as rocblas). It wouldn't even be technically possible for SCALE to distribute and use cuBlas, since the source code is not available. I suppose maybe you could do distribute cuBlas and run it through ZLUDA, but that would likely become legally troublesome.
  - anon291 2y ago
    > SCALE doesn't use cuBlas and friends. For those APIs, it uses either its own implementations of the functions, or delegates to an existing AMD library (such as rocblas). And this is the problem. I guarantee you NVIDIA has more engineers working on cuBLAS et al than AMD does. The NVIDIA moat is not CUDA the language or CUDA the library. It's CUDA the ecosystem. That means things like all the high performance libraries; all the high performance libraries with clustering support (does AMD even have a clustering solution like NVLink -- everyone forgets that NVIDIA also does high speed networking); all the high perf appliances (everyone also forgets that NVIDIA sells entire systems, not GPUS); all the high perf servers (Triton inference server, etc). We can go on. I commend the project volunteers for what they've done, but I would recommend getting VC money and competing directly with NVIDIA.
- nickysielicki 2y ago
  NCCL is open source and permissively licensed.
  - anon291 2y ago
    Sure and it works with NVIDIA GPUs only AFAIK.
    - ckitching 2y ago
      Not any more ;)

The CUDA C APIs are supported as much in C as in C++ using SCALE! Cuda-fortran is not currently supported by scale since we haven't seen much use of it "in the wild" to push it up our priority list.

anon291 2y ago

It doesn't matter though. NVIDIA distributes tons of libraries built atop CUDA that you cannot distribute or use on AMD chips legally. Cutlass, CuBLAS, NCCL, etc.

tama_sala 2y ago
Correct, which one of the main moats Nvidia has when it comes to training
ckitching 2y ago
SCALE doesn't use cuBlas and friends. For those APIs, it uses either its own implementations of the functions, or delegates to an existing AMD library (such as rocblas). It wouldn't even be technically possible for SCALE to distribute and use cuBlas, since the source code is not available. I suppose maybe you could do distribute cuBlas and run it through ZLUDA, but that would likely become legally troublesome.
- anon291 2y ago
  > SCALE doesn't use cuBlas and friends. For those APIs, it uses either its own implementations of the functions, or delegates to an existing AMD library (such as rocblas). And this is the problem. I guarantee you NVIDIA has more engineers working on cuBLAS et al than AMD does. The NVIDIA moat is not CUDA the language or CUDA the library. It's CUDA the ecosystem. That means things like all the high performance libraries; all the high performance libraries with clustering support (does AMD even have a clustering solution like NVLink -- everyone forgets that NVIDIA also does high speed networking); all the high perf appliances (everyone also forgets that NVIDIA sells entire systems, not GPUS); all the high perf servers (Triton inference server, etc). We can go on. I commend the project volunteers for what they've done, but I would recommend getting VC money and competing directly with NVIDIA.
nickysielicki 2y ago
NCCL is open source and permissively licensed.
- anon291 2y ago
  Sure and it works with NVIDIA GPUs only AFAIK.
  - ckitching 2y ago
    Not any more ;)