3 ms·

The issue with ROCm is that its completely unaccessible for most users. It only support high end GPUs. While cuda works on a 1050ti.

vkaku 3y ago
Agreed. What they've also been doing is stalling/removing support on gfx803 and that literally could have allowed people to still use the GPU for doing many decent small nets.
kkielhofner 3y ago
Supported on CUDA 12 no less! To get an idea, the 1050ti is a card with an MSRP of $140 - almost seven years ago when it was released. Between the driver and CUDA support matrix it will likely end up with a 10 year support life. While it's not going to impress with an LLM or such it's the lowest minimum supported card for speech to text with Willow Inference Server (I'm the creator) and it still puts up impressive numbers. Same for the GTX 1060/1070, which you can get with up to 8GB VRAM today for ~$100 used. Again, not impressive for the hotter LLMs, etc but it will do ASR, TTS, video encoding/decoding, Frigate, Plex transcoding, and any number of other things with remarkable performance (considering cost). People also run LLMs on them and from a price/performance/power standpoint it's not even close compared to CPU. The 15 year investment and commitment to universal support for any Nvidia GPU across platforms (with very long support lifecycles) is extremely hard to compete with (as we see again and again with AMD attempts in the space).
- jacooper 3y ago
  To be fair, AMD does offer good long term support for cards, just not with ROCm.
  - kkielhofner 3y ago
    The 390.x driver series (updated 11/22) supports cards that are at least 13 years old[0]. From a quick glance of available drivers for AMD cards of that vintage the drivers haven't been updated since 2014/2015. When comparing the last driver release from Nvidia in the 2014-205 range it supports cards going back to at least 2004[1]. [0] - https://download.nvidia.com/XFree86/Linux-x86_64/390.157/README/supportedchips.html https://download.nvidia.com/XFree86/Linux-x86_64/390.157/REA... [1] - https://download.nvidia.com/XFree86/Linux-x86_64/96.43.23/README/appendix-a.html https://download.nvidia.com/XFree86/Linux-x86_64/96.43.23/RE...

vkaku 3y ago

Agreed. What they've also been doing is stalling/removing support on gfx803 and that literally could have allowed people to still use the GPU for doing many decent small nets.

kkielhofner 3y ago

Supported on CUDA 12 no less! To get an idea, the 1050ti is a card with an MSRP of $140 - almost seven years ago when it was released. Between the driver and CUDA support matrix it will likely end up with a 10 year support life. While it's not going to impress with an LLM or such it's the lowest minimum supported card for speech to text with Willow Inference Server (I'm the creator) and it still puts up impressive numbers. Same for the GTX 1060/1070, which you can get with up to 8GB VRAM today for ~$100 used. Again, not impressive for the hotter LLMs, etc but it will do ASR, TTS, video encoding/decoding, Frigate, Plex transcoding, and any number of other things with remarkable performance (considering cost). People also run LLMs on them and from a price/performance/power standpoint it's not even close compared to CPU. The 15 year investment and commitment to universal support for any Nvidia GPU across platforms (with very long support lifecycles) is extremely hard to compete with (as we see again and again with AMD attempts in the space).

jacooper 3y ago
To be fair, AMD does offer good long term support for cards, just not with ROCm.
- kkielhofner 3y ago
  The 390.x driver series (updated 11/22) supports cards that are at least 13 years old[0]. From a quick glance of available drivers for AMD cards of that vintage the drivers haven't been updated since 2014/2015. When comparing the last driver release from Nvidia in the 2014-205 range it supports cards going back to at least 2004[1]. [0] - https://download.nvidia.com/XFree86/Linux-x86_64/390.157/README/supportedchips.html https://download.nvidia.com/XFree86/Linux-x86_64/390.157/REA... [1] - https://download.nvidia.com/XFree86/Linux-x86_64/96.43.23/README/appendix-a.html https://download.nvidia.com/XFree86/Linux-x86_64/96.43.23/RE...