4 ms·

There's only mention of Nvidia GPUs on the web site, what about AMD?

• zzbzq 3y ago
  Don't really work for AI. There might be a weird experimental driver for linux or something, I never got it to work.
  • MediumOwl 3y ago
    Why not?
    • zzbzq 3y ago
      They suck at it basically. My understanding is it's really the software, Nivida supports AI first-class with the drivers (or whatever.) AMD has next to nothing nothing.
• lhl 3y ago
  On Windows, llama.cpp has OpenCL support (CLBlast) and MLC LLM (https://mlc.ai/mlc-llm/docs/ https://mlc.ai/mlc-llm/docs/) has Vulkan acceleration. On Linux, ExLlama and MLC LLM have native ROCm support, and there is a HIPified fork of llama.cpp as well.