4 ms·
Is it any easier when using Ubuntu on ARM Oracle servers?
by jvickers 3y ago
Is it any easier when using Ubuntu on ARM Oracle servers?
- brucethemoose2 3y agoNah, I tried Ubuntu too. The OpenBLAS package was missing on ARM, along with some other dependencies I needed for compilation. At the end of the day, even with many tweaks and custom compilation flags, the instance was averaging below 1 token/sec as a Kobold Horde host, which is below the threshold to even be allowed as a llm host.