4 ms·

I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but

I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but after two days of the compiler running it failed.

madushan1000 2mo ago
Two days sounds like a lot, both llama.cpp and openvino only takes a few minutes to compile on any decent modern cpu.
- walrus01 2mo ago
  the full set of llama.cpp binaries builds in under 5 minutes with an unmodified build workflow straight from their github page on a literally ten year old dual xeon.
- gnull 2mo ago
  It also even easier to get working and integrate into your system in a sustainable manner with NixOS. Do it by hand or throw an LLM at it, it will get you a declarative patch for your NixOS config that brings llama-cpp into your config that you can review and add under version control (no random `make install` build artifacts contaminating your system, no wondering "what was it that I ran? what are all these files? how do I do the same with a newer version?" a couple months later). There's also likely some build cache where Nixoids have already build what you want. I had a great experience with llama-cpp with Nvidia backend on NixOS. (Sorry for being that guy.)
numpad0 2mo ago
I don't know how it works, but does Vulkan not work on Arc?
cptskippy 2mo ago
Does the A770 use the Xe driver? If so then it might work with the scripts that I've been using to build llama.cpp with SYCL support for the Arc Pro B70. https://github.com/cptskippy/battlemage-llm-gateway https://github.com/cptskippy/battlemage-llm-gateway It's designed so that you can re-run the scripts to pull the latest updates. When Muse Glimmer was released the other day I just ran the 02 script to build the latest version of llama.cpp with support for it.
- dlcarrier 2mo ago
  Thanks for the link. I'm running an Arch-based system, by the way, so I won't be able to directly use the scripts, which are expecting Debian's package manager, but I'll look at what they're doing and see if I can get something working.
OpenArcBob 2mo ago
[dead]

madushan1000 2mo ago

Two days sounds like a lot, both llama.cpp and openvino only takes a few minutes to compile on any decent modern cpu.

walrus01 2mo ago
the full set of llama.cpp binaries builds in under 5 minutes with an unmodified build workflow straight from their github page on a literally ten year old dual xeon.
gnull 2mo ago
It also even easier to get working and integrate into your system in a sustainable manner with NixOS. Do it by hand or throw an LLM at it, it will get you a declarative patch for your NixOS config that brings llama-cpp into your config that you can review and add under version control (no random `make install` build artifacts contaminating your system, no wondering "what was it that I ran? what are all these files? how do I do the same with a newer version?" a couple months later). There's also likely some build cache where Nixoids have already build what you want. I had a great experience with llama-cpp with Nvidia backend on NixOS. (Sorry for being that guy.)

numpad0 2mo ago

I don't know how it works, but does Vulkan not work on Arc?

cptskippy 2mo ago

Does the A770 use the Xe driver? If so then it might work with the scripts that I've been using to build llama.cpp with SYCL support for the Arc Pro B70. https://github.com/cptskippy/battlemage-llm-gateway https://github.com/cptskippy/battlemage-llm-gateway It's designed so that you can re-run the scripts to pull the latest updates. When Muse Glimmer was released the other day I just ran the 02 script to build the latest version of llama.cpp with support for it.

dlcarrier 2mo ago
Thanks for the link. I'm running an Arch-based system, by the way, so I won't be able to directly use the scripts, which are expecting Debian's package manager, but I'll look at what they're doing and see if I can get something working.

OpenArcBob 2mo ago

[dead]