2 ms·
Nah, I tried Ubuntu too. The OpenBLAS package was missing on ARM, along with some other dependencies I needed for compilation. At the end of the day, even wit
by brucethemoose2 3y ago
Nah, I tried Ubuntu too.
The OpenBLAS package was missing on ARM, along with some other dependencies I needed for compilation.
At the end of the day, even with many tweaks and custom compilation flags, the instance was averaging below 1 token/sec as a Kobold Horde host, which is below the threshold to even be allowed as a llm host.