3 ms·

What I don't get is why they don't at least assign a dev or two to make the poster child of this work: llama.cpp It's the first thing anyone tries when trying

What I don't get is why they don't at least assign a dev or two to make the poster child of this work: llama.cpp It's the first thing anyone tries when trying to dabble in AI or compute on the gpu, yet it's a clusterfuck to get to work. A few blessed cards work, with proper drivers and kernel; others just crash, perform horribly slow, or output GGGGGGGGGGGGGG to every input (I'm not making this up!) Then you LOL, dump it and go buy nvidia et voila, stuff works first try.

wkat4242 2y ago
It does work, I have it running on my Radeon VII Pro
- Filligree 2y ago
  It sometimes works.
  - wkat4242 2y ago
    How so? It's rock solid for me. I use ollama but it's based on llama.cpp It's quite fast also, probably because that card has fast HBM2 memory (it has the same memory bandwidth as a 4090). And it was really cheap as it was on deep sale as an outgoing model.
    - Filligree 2y ago
      "Sometimes" as in "on some cards". You're having luck with yours, but that doesn't mean it's a good place to build a community.
      - wkat4242 2y ago
        Ah I see. Yes, but you pick the card for the purpose of course. I also don't like the way they have such limited support on ROCm. But when it works it works well. I have Nvidia cards too by the way, a 4090 and a 3060 (the latter I use for AI also, but more for Whisper because faster-whisper doesn't do ROCm right now).
    - halJordan 2y ago
      Aside from the fact that gfx906 is one of the blessed architecture mentioned (so why would it not work). Like how do you look at your specific instance and then turn around and say "All of you are lying, it works perfectly." How do you square that circle in your head
      - wkat4242 2y ago
        No I was just a bit thrown by the "sometimes". I thought they were referring to a reliability issue. I am aware of the limited card support with ROCm and I complained about this elsewhere in the thread too. Also I didn't accuse anyone of lying. No need to be so confrontational. And my remark to the original poster at the top was from before they clarified their post. I just don't really see what AMD can do to make ollama work better other than porting ROCm to all their cards which is definitely something they should do. And no I'm not an AMD fanboi. I have no loyalty to anyone, any company or any country.

wkat4242 2y ago

It does work, I have it running on my Radeon VII Pro

Filligree 2y ago
It sometimes works.
- wkat4242 2y ago
  How so? It's rock solid for me. I use ollama but it's based on llama.cpp It's quite fast also, probably because that card has fast HBM2 memory (it has the same memory bandwidth as a 4090). And it was really cheap as it was on deep sale as an outgoing model.
  - Filligree 2y ago
    "Sometimes" as in "on some cards". You're having luck with yours, but that doesn't mean it's a good place to build a community.
    - wkat4242 2y ago
      Ah I see. Yes, but you pick the card for the purpose of course. I also don't like the way they have such limited support on ROCm. But when it works it works well. I have Nvidia cards too by the way, a 4090 and a 3060 (the latter I use for AI also, but more for Whisper because faster-whisper doesn't do ROCm right now).
  - halJordan 2y ago
    Aside from the fact that gfx906 is one of the blessed architecture mentioned (so why would it not work). Like how do you look at your specific instance and then turn around and say "All of you are lying, it works perfectly." How do you square that circle in your head
    - wkat4242 2y ago
      No I was just a bit thrown by the "sometimes". I thought they were referring to a reliability issue. I am aware of the limited card support with ROCm and I complained about this elsewhere in the thread too. Also I didn't accuse anyone of lying. No need to be so confrontational. And my remark to the original poster at the top was from before they clarified their post. I just don't really see what AMD can do to make ollama work better other than porting ROCm to all their cards which is definitely something they should do. And no I'm not an AMD fanboi. I have no loyalty to anyone, any company or any country.