3 ms·

You gotta wait until GGML and the likes repackage the model; early releases are almost always targeted at ML folks with dedicated GPUs.

londons_explore 3y ago
I think it's really lame that ML, which is just math really, hasn't got some system-agnostic language to define what math needs to be done, and then it can run easily on CPU/GPU/TPU/whatever... A whole industry being locked into NVidia seems bad all round.
- esafak 3y ago
  It's not Nvidia's fault that the competition (AMD) does not provide the right software. There is an open alternative to CUDA called OpenCL.
  - londons_explore 3y ago
    Nvidia has wrapped their cuda language in patents and licensing so tightly that there is no way AMD could release anything cuda-compatible.
    - kkielhofner 3y ago
      Yes but AMD could release a ROCm that actually works and then put actually meaningful resources into some of the countless untold projects out there that have been successfully building on CUDA for 15 years. There was a recent announcement that after six years AMD finally sees the $$$ and will be starting to (finally) put some real effort into ROCm[0]. That announcement was two days ago and they claim they started on this last year. My occasional experience with ROCm doesn't show much progress or promise. I'm all for viable Nvidia competition in the space but AMD has really, really, really dropped the ball on GPGPU with their hardware up to this point. [0] - https://www.eetimes.com/rocm-is-amds-no-1-priority-exec-says/ https://www.eetimes.com/rocm-is-amds-no-1-priority-exec-says...
  - tormeh 3y ago
    As sad as it is, this is true. AMD has never spent lots of money on software, while Nvidia always has, which was fine for traditional graphics, but with ML this really doesn't cut it. AMD could have ported Pytorch to OpenCL or Vulkan or WebGPU, but they just... can't be bothered???
- miloignis 3y ago
  https://onnx.ai/ https://onnx.ai/ sounds close to what you're thinking of, it's an open interchange format.

londons_explore 3y ago

I think it's really lame that ML, which is just math really, hasn't got some system-agnostic language to define what math needs to be done, and then it can run easily on CPU/GPU/TPU/whatever... A whole industry being locked into NVidia seems bad all round.

esafak 3y ago
It's not Nvidia's fault that the competition (AMD) does not provide the right software. There is an open alternative to CUDA called OpenCL.
- londons_explore 3y ago
  Nvidia has wrapped their cuda language in patents and licensing so tightly that there is no way AMD could release anything cuda-compatible.
  - kkielhofner 3y ago
    Yes but AMD could release a ROCm that actually works and then put actually meaningful resources into some of the countless untold projects out there that have been successfully building on CUDA for 15 years. There was a recent announcement that after six years AMD finally sees the $$$ and will be starting to (finally) put some real effort into ROCm[0]. That announcement was two days ago and they claim they started on this last year. My occasional experience with ROCm doesn't show much progress or promise. I'm all for viable Nvidia competition in the space but AMD has really, really, really dropped the ball on GPGPU with their hardware up to this point. [0] - https://www.eetimes.com/rocm-is-amds-no-1-priority-exec-says/ https://www.eetimes.com/rocm-is-amds-no-1-priority-exec-says...
- tormeh 3y ago
  As sad as it is, this is true. AMD has never spent lots of money on software, while Nvidia always has, which was fine for traditional graphics, but with ML this really doesn't cut it. AMD could have ported Pytorch to OpenCL or Vulkan or WebGPU, but they just... can't be bothered???
miloignis 3y ago
https://onnx.ai/ https://onnx.ai/ sounds close to what you're thinking of, it's an open interchange format.