3 ms·
As others have mentioned, Ollama uses Llama.CPP under the hood and they recently released Vulkan support which is supposed to work with AMD GPUs. I was able to
by chown 3y ago
As others have mentioned, Ollama uses Llama.CPP under the hood and they recently released Vulkan support which is supposed to work with AMD GPUs. I was able to use llama.cpu compiled with Vulkan support with my app [1] and make it run on an AMD laptop but I was unable to make it work with Ollama as it makes some assumptions about how it goes about searching for available GPUs on a machine.
[1]: https://msty.app https://msty.app
- deleted 3y ago[deleted]
- Kelteseth 3y agoI got a Windows defender Virus alert after executing your app.
- chown 3y agoUgh! Probably because it’s an exe app? Not sure how to go around about that. I am looking into getting it signed just like the counterpart MacOS app. Thank you for the heads up and sorry about the false positive.
- rezonant 3y agoIronically Ollama is also struggling with this sort of thing, see https://github.com/ollama/ollama/issues/2519 https://github.com/ollama/ollama/issues/2519 Code signing helps by having an avenue by which you can establish reliable reputation, and then using VirusTotal to check for AV flags and using the AV vendor's whitelist request form is the second part, over time your reputation increases and you don't get flagged as malware. It seems to be much more likely with AI stuff, apparently due to use of CUDA or something (/shrug)
- 65a 3y agoROCm is preferred over vulkan for AMD GPUs, performance wise. Using OpenCL or Vulkan should only be for older cards or weird setups.
- chown 3y agoThat’s good to know. Thank you!