2 ms·

I'm thrilled to see support for RX 6800/6800 XT / 6900 XT. I bought one of those for an outrageous amount during the post-covid shortage in hopes that I could

I'm thrilled to see support for RX 6800/6800 XT / 6900 XT. I bought one of those for an outrageous amount during the post-covid shortage in hopes that I could use it for ML stuff, and thus far it hasn't been very successful, which is a shame because it's a beast of a card! Many thanks to ollama project and llama.cpp!

mey 3y ago
Sad to see that the cut off is just after 6700 XT which is what is in my desktop. They indicate more devices are coming, hopefully that includes some of the more modern all in one chips with RDNA 2/3 from AMD as well.
- mey 3y ago
  It appears that the cut off lines up with HIP SDK support from AMD, https://rocm.docs.amd.com/projects/install-on-windows/en/latest/reference/system-requirements.html https://rocm.docs.amd.com/projects/install-on-windows/en/lat...
- throawayonthe 3y ago
  I’ve already been using ollama with my 6700xt just fine, you just have to set some env variable to make rocm work “unoficially” The linked page says they will support more soon, so i’m guessing this will just be integrated
  - Rising6378 3y ago
    which env variable did you set to what? currently I'm struggeling setting it up
    - Retr0id 3y ago
      HSA_OVERRIDE_GFX_VERSION=10.3.0 seems to be the magic words. You can invoke it directly via `HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama serve`, but I added this line to the systemd unit at /etc/systemd/system/ollama.service: Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"
- bavell 3y ago
  I've been using my 6750XT for more than a year now on all sorts of AI projects. Takes a little research and a few env vars but no need to wait for "official" support most of the time.