3 ms·
I also attempted to package ROCM on musl. Specifically, packaging it for Alpine Linux. It truly is a nightmare to build the whole thing. I got past the custom
by WhyNotHugo 6mo ago
I also attempted to package ROCM on musl. Specifically, packaging it for Alpine Linux.
It truly is a nightmare to build the whole thing. I got past the custom LLVM fork and a dozen other packages, but eventually decided it had been too much of a time sink.
I’m using llama.cpp with its vulkan support and it’s good enough for my uses. Vulkan so already there and just works. It’s probably on your host too, since so many other things rely on it anyway.
That said, I’d be curious to look at your build recipes. Maybe it can help power through the last bits of the Alpine port.
- lrvick 6mo agoKeep an eye out for a stable rocm PR to stagex in the next week or so if all goes well.
- sigmoid10 6mo agoInteresting how Vulkan and ROCM are roughly the same age (~9 years), but one is incredibly more stable (and sometimes even more performant) for AI use cases as side-gig, while the other one is having AI as its primary raison d'être. Tells you a lot about the development teams behind them.
- Hasslequest 6mo ago[dead]
- icedchai 6mo agoI've built llama.cpp against both Vulkan and ROCm on a Strix Halo dev box. I agree Vulkan is good enough, at least for my hobbyist purposes. ROCm has improved but I would say not worth the administrative overhead.
- seemaze 6mo agoI realize it does not address the OP security concerns, but I'm having success running rocm containers[0] on alpine linux specifically for llama.cpp. I also got vLLM to run in a rocm container, but I didn't have time to to diagnose perf problems, and llama.cpp is working well for my needs. [0] https://github.com/kyuz0/amd-strix-halo-toolboxes https://github.com/kyuz0/amd-strix-halo-toolboxes
- WhyNotHugo 6mo agoFWIW, Alpine now has native packages for llama.cpp (using Vulkan).
- seemaze 6mo agonice! will check it out edit: and thanks for the packaging work!